FoOlFuuka

at
FoOlFuuka

For the past 45 days we’ve halted the development for our flagship software, FoOlSlide, and worked on FoOlFuuka, a new 4chan archiver that is written in PHP instead of Perl. This caused more rage in the scanlation community (where is my daily update?!) than on 4chan in the first 12 hours since FoOlFuuka’s release. This means things are going pretty fine for us, and if that wasn’t cool enough, two people have already asked when they could switch to FoOlFuuka.

Let’s talk about it.

  1. What’s new?
  2. What’s missing?
  3. When will it be public?
  4. How will development proceed?

What’s new?

First of all, no HTML tables. We’ve removed them because tables are evil. This led us to a complete rewrite of the theme, which now looks more similar to Yotsuba than the original Fuuka.

We know that we broke all the userscripts like this, but we did some forward thinking. We’ve decided that we’d deal with userscripts ourselves, which translated into us rewriting the functionalities of the userscripts inside of FoOlFuuka, so they will always be updated without requiring a third party to fix your userscript.
Currently, the functions are:

  • Localization of the timestamps into your local time
  • Real time reading of threads
  • Quick reply (no reload when posting)
  • Quotes and backquotes show the post quoted/quoting when hovered
  • thread-O-matic 2.39b (name subject to change)

These are the most important userscript functions. We plan to improve these functions and add more. This also allows us to use the integrated API to make the lightest requests possible and give you the snappiest experience we can provide.

That’s right, we have an API. We’ll make a page with examples when we can say it’s solid enough (read: not anytime soon).

The page loading seems incredibly fast, considering we’ve fiddled a lot with the SQL and we have a ton of PHP threads running. The main page doesn’t show the latest threads anymore, but the threads with the newest replies. It also shows how many images have been omitted. There’s SauceNAO support, and the images are stored on server until they hit certain limits, keeping them up for a few days.

What’s next?

We still have to finish the statistics. I am currently considering which plotting Javascript plugin I should use for the graphic ones, while the rest is working already (though not publicly).

We’re still using the Perl fetcher. The Java fetcher coding came to a halt while we were focusing on the PHP part of the software. We’ll first get the PHP done, then work on the Java.

We plan to create the thumbnails on FoOlFuuka instead of fetching them from 4chan, because we can achieve better quality at a lower size if we use our own software.

When this is done, we’ll get some rest and work on FoOlSlide again for a while.

When will it be public? What is missing?

FoOlFuuka is already public – open-sourced under Apache 2.0 – but it’s still missing a few parts.

What is missing are a more solid database structure (so we don’t need to mess with it too frequently), the Java fetcher that I am working on, some solutions for creating more boards without having to do so manually, and a way to automatically update the Sphinx search definitions.

I suppose we could be done within a month. After you’ll have it installed, you can use the autoupgrade function from the admin panel to stay updated.

There’s only one column to be added to each of the original tables, and while we’re still filling it, the _local table is not used anymore. For the rest, FoOlFuuka can use two separate databases, one for the internal functions and one for the original tables, or it can keep it all in one database.
It is easy to fall back to the original Fuuka. Just delete a column.

How will development proceed?

We will use the same method as we do for FoOlSlide: the most mentioned bugs and the most requested features will be fixed or added first. This allows us to keep high quality software suited for the majority of users.

Go ahead and ask for features or notify us of bugs. Just a reminder: bugs aren’t there to irritate you, nor are they “nasty”. We are not withholding features to get reactions out of you. Try not being too personal when pointing bugs out or requesting things, and you’ll get the most out of us.

78 Responses to “FoOlFuuka”

  1. Anonymous says:

    >This caused more rage in the scanlation community (where is my daily update?!) than on 4chan in the first 12 hours since we had released FoOlFuuka.
    Do I see some denial there?

    • woxxy says:

      Rather, scanlation makes 4chan look like a huge circlejerk. Bad shit, man.

      Let’s say that for some, with FoOlSlide, there’s money involved. Of course not money I will see, so it’s not in my interest to develop something when I am all hyped to code something new.

    • woxxy says:

      You’re right on this, I should put some more credits to him, and Andrey, the guy who coded Fuuka. The description on github doesn’t really name them, and the info file that is gone.

      I will deal with this sometime tomorrow. Their memory shall not vanish.

      I’d especially like to say that there was nothing wrong with Fuuka and that I actually learned some stuff off it. We just wanted to get some solution that would keep our server stable when there’s multiple pageviews per second. It was a scaling issue.

  4. Anonymous says:

    Where is the option to sort by thread creation date? That is the only reason that I use the archive to see newly created threads as they come in, just like many of the other users that use the site.

  7. Anonymous says:

    Your new fool fuuka (I refuse to type in alternate caps like a 10-year-old, sorry) looks and is horrible.
    In lieu of killing yourself to avoid dishonor, could you at least default to oldarchive and make your ugly new archiver be the one that has to be reached through a new URL? If it’s as good as you think it is people will just change their bookmarks.
    Easymodo (which I still use) redirects to the same URL, but that now points to your horrid new PHP monstrosity, so yeah.

  11. Jerosh says:

    At the very, very least, make a style sheet that makes it identical to the old archive. There’s no sense in having an archive look completely different to the source material. I get that you guys are into the whole Web 2.0 thing, but 4chan isn’t exactly the place for people who care about AJAX and jQuery and lord knows what else.

    • woxxy says:

      The archive looks more like 4chan than the original archive by now. The AJAX stuff is hardly noticeable, and helps us having lower server load by fetching the least data needed.

      Surely it’s not as bad as the first 12 hours it was up.

      And we might make a whole theme to clone the old Fuuka (a stylesheet isn’t enough). It takes time though, and Yotsuba theme will come first.

      • Anonymous says:

        >The AJAX stuff is hardly noticeable, and helps us having lower server load by fetching the least data needed.
        Silly “unstoppable” PHP dev, that’s not how you lower server load. You should look into optimising your caching mechanism instead, or using something like varnish if you feel too dumb for the task. Because once the request hits your app you are fucked and no amount of ajax will save your [i][u][b]ANUS[/b][/u][/i]

  12. Anonymous says:

    >storing full images
    >threadomatic
    Are you trying to become 4chan itself or something?
    Soon you’ll be crying for more money because “omg so many traffics!” whereas in fact all people are asking for is a simple archive.

    I don’t even know who I am quoting.

    • woxxy says:

      My idea is putting a donation button or an ad when you go viewing the image, since we could call this “premium service”. I see other services around 4chan asking for donations anyway, and I think this is the most precious service we could give right now.

      Otherwise, we’ll take down what we can’t keep up with, or ask help to other people with servers.

      Just as for scanlation “if you can’t keep up with the costs, don’t buy RAWs you can’t afford, and don’t blame the users for not shouldering your expenses”.

    • Anonymous says:

      It’s a 4chan archive, why the fuck should it look entirely different than the site it’s archiving? There is no point at all. And it looks fucking ugly.

      Also, most of us have scripts we would like to stay compatible.

  15. Ottre says:

    I want a way to locate imagedumps. IMO the best way of doing this is a button that lets you toggle between latest threads / latest replies / latest image replies. If that’s too difficult to implement, tag all threads with 100+ images.

    • woxxy says:

      I have some ideas, just keep an eye on the + button on top. I’ll add some indicator for heated threads.

      For now you can just keep an eye on the “Images: xxx” detail under each thumbnail.

    • Anonymous says:

      Being smart is against Woxxy’s religion.

      Quoted from a post on /jp/:

      >Woxxy is good at reinventing the wheel.
      >He rewrites manga readers, he rewrites archives,
      >he rewrites catalogs and he now tries to rewrite 4chan itself,
      >all that while begging for money.
      >Will woxxy’s “creativity” ever end?

  17. Anonymous says:

    Warosu, official archive of #bunbunmaru

    If you’re not too busy watching kids television can you change the redirect on archive.easymodo to point to fuuka.warosu
    which is up and not ugly as sin
    and so the fuuka wars of 2011 heated up
    Eh, I’ll make a selection stored with cookies later
    or something
    one line edit to your redirect, Eksi, that’s all I’m asking
    to pick between foolz, oldarchive foolz and warosu
    warosu
    foolz is thinking of monetizing already
    the bastards
    I don’t dislike what woxxy is doing, though, he’s advancing pretty fast and adding a bunch of stuff
    It’s ugly and I hate it
    Change it

  18. Anonymous says:

    Would it be possible to also archive /u/ since many typeset doujin seem to appear on that board? It’s not very fast, hence it shouldn’t require many resources.

  19. Anonymous says:

    Local timestamps are BAD. I want to know when a post was made in relation to 4chan time, not my local time -5 GMT or whatever. Please change!

  23. M says:

    YEAH SCREW YOU ALL I READ THIS POST 5 TIMES AND I STILL HAVE NO IDEA WHAT THE HELL IT’S TRYING TO SAY OR WHAT THE HELL FOOL FUUKA EVEN IS, AND THE SECOND PART OF THE NAME IS GAY AND MY BRAIN ALWAYS WANTS TO MISPRONOUNCE IT INTO THE F WORD, SORRY

  25. Anonymous says:

    You added one of the primarily image dump boards?
    Eh, guess it makes sense if you’re hosting the full images now.
    Enjoy your bandwidth rape.

      • Anonymous says:

        Since /u/ IS extremely slow like you say, what are solid justifications for why /u/ would need their board archived in the first place?

        /u/ is primarily an imagedumping board. Everything you can find in /u/, you can find on one of the boorus or pixiv.

        And before you go and say /u/ dumps manga sometimes, too; I’ll tell you that all the yuri manga dumped on /u/ can easily be found on sites like baka-updates where links to scanlators are readily available.

        Even the raws, themselves, can be found with a simple google search.

        For the moon-savvy, Japanese P2P software will provide you all the raws of yuri manga you can handle.

  26. Scriptin Us says:

    I’m not sure I quite understand the complaints. I just found this site and I was amazed at how great his archiver site looks.

    I compared the current look with the old one and I must say that this new one is infinitely better.

    If you are releasing the code, that’s just awesome. I can’t wait to set this up for myself.

    • Anonymous says:

      Somehow, the new archiver causes some performance issues on my PC (quad core i7, 4gb ram, firefox 8.0) if there are many windows and tabs open, hence claims like “infinitely better” have to stay in context. But it’s a nice alternative, that’s true.

  27. the unoriginal FoOl says:

    What is 4chan? What’s the archiver for? I wantz manghas not some batshit legend!

    (No, seriously, I don’t understand what the 4chan deal is)

  28. Anonymous says:

    Thanks for all your hard work, I only have one issue. The localization of timestamps is extremely confusing.

    I now have to calculate the EST time (4chan time) of each post I look at in the archive, account for time zones, daylight saving time, etc. Also, my ISP sometimes routes traffic via a proxy cache which geolocates to a different part of my country, with a different timezone. So I am presented with posts in two different timezones with no way to know which is being used unless I run a traceroute. This is a hindrance and not helpful.

    The archive should present the post data as it appeared on 4chan, it should absolutely not change it in any way. If localization of timestamps is absolutely necessary, could it at least be optional or implemented in javascript so I can disabled it

    Thanks again for your hard work running this awesome service

    • Woxxy says:

      It is javascript, and it’s based on your computer’s time. We changed the system to store the posts in UTC time.

      I guess we will give the option to show 4chan time.

      • fsbgfdgf says:

        That’s a horrible idea. Local times suck on the internet.

        Also, why the fuck are you using Facebook and Twitter on this blog? You guys are total homos. Why do you have to run after the latest internet and shit-technology trends? That’s completely un-/a/-like.

  29. Anon says:

    That’s all nice and dandy, but, woxxy, stop killing the scanlation scene. You’re the daiz of the scene. I’m pretty sure you’ll come with a 10-bit jpeg or some shit in a week or less.

  31. PuffyPrincess says:

    Can someone explain what the hell foolfuuka is to a normal person? All I got out from that wall of opinions was that it’s some sort of program or whatever and that is has something to do with 4chan.

  32. fsbgfdgf says:

    >Comment too long
    Trying to post 2 words on the old archive.
    >Error: You must wait at least 10 seconds before posting again.
    Trying to post a second post on the new archive. It has been at least a minute already.

    Why are you guys so shit? Please stop hosting the archive, so that someone else will jump in.

  33. perrinu says:

    Can we get the new archive to order by post date and not bump date? It’s really hard to see what the janitor is deleting ordering the threads by bump date. Either that or re-enable posting on the old archive. Also, on the new archive there needs to be an option to disable auto-update. That just gets in the way if you are trying to type a post in the post box. The post box will just keep moving down the page if people are posting in a thread.

  34. Anonymous says:

    There’s a problem with the Fuuka style on http://archive.foolz.us

    When clicking on someone’s quote in a post it treats it like url link rather than jumping to the quoted post.
    The problem is that the quoted post adds a / between the thread number and the post number.

    Like this: http://archive.foolz.us/a/thread/59555302/#p59560670

    It should be like this: http://archive.foolz.us/a/thread/59555302#p59560670

    The / at the end of the thread needs to be removed.

  35. Anonymous says:

    Remove the “local time” thing. It looks like some ghost posts were created before the thread they’re replying to. Just stick with 4chan time.

  38. Fadamor says:

    Wow. Lots of rage. Most sane people would walk with their feet if a service didn’t meet their needs. Here, people stand around and rage. If there IS no other option you can “walk” to, then perhaps you should be coding the much better archiver that exists only in your head rather than wasting everyone’s time raging at this one.

  39. Naeko says:

    Don’t forget that full sized images for older threads are available at chanarchive.org

    If the threads don’t get enough upvotes in their first week of appearing at chanarchive, they will not be permanently archived (auto-deleted).

Leave an Opinion

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>