Introducing: HUGEpic - a web app for showing massive pictures

So here's my latest little fun side-project: HUGEpic.io http://hugepic.io

It's a web app for uploading massive pictures and looking at them like maps.

The advantages with showing pictures like this are:

• you only download what you need
• you can send a permanent link to a picture at a particular location with a particular zoom level
• you can draw annotations on a layer on top of the image

All the code is here on Github and as you can see it's a Tornado that uses two databases: MongoDB and Redis and when it connects to MongoDB it uses the new Tornado specific driver called Motor which is great.

Before I get to the juicy client side stuff, let me talk about something awesome in between Tornado and Javascript, namely: RQ
It's an awesomely simple python message queue that only works with Python, Redis and on UNIXy systems. All checks. My only real experience with message queues has honestly been with Celery which is also great but a right pain compared to RQ. With RQ, all I do is reduce the heavy tasks down to pure python functions. For example, make_thumbnail() then I simply do this:

from utils import make_thumbnail
from rq import Queue
queue = Queue(connection=self.redis)
job = q.enqueue(
    make_thumbnail,
    image,
    width,
    extension,
    self.application.settings['static_path']
)

and that's it. Starting rqworker on the command line (from somewhere where __import__('utils.make_thumbnail') makes sense) and we're off!

You might think that using a message queue is all fancy pants and just something I need to bother myself with because of Tornado's eventloop nature. But no, it's so much more than that. When a massive 5 Mb JPG is uploaded, a little algorithm is figuring out roughly how many zoom levels that can be used and what ALL 256x256 tiles are going to be for each resized version of the original. Then it needs to generate a thumbnail to represent that JPG and all the tiles and the thumbnail need to go through an optimizer (I'm using jpegoptim and optipng).
Lastly, to be able to serve all tiles from a fast CDN I have to upload every single tile to a Reduced Redundancy Amazon S3 storage that the Amazon CloudFront CDN is hooked up to. This might sometimes fail due to network hickups and must be resilient to continue where it left off.

All of that stuff takes a very long time but it's made it much easier and much more comfortable thanks to RQ.

Now, on the front end. The genesis inspiration to this was a library called Polymaps which isn't bad but when I later switched to Leaftlet I was blown away. It was lighter, smoother running and has an absolutely stunning API that even I could understand.

And that led me to find another amazingly neat library, for Leaflet, called Leaflet.draw which makes it really easy to add tools for drawing on the pictures. You can draw lines, rectangles, circles, polygons and drop markers. And for all of them it was relatively easy to bind cute popup bubbles so you can type in comments like this or this.

And lastly, there's Filepicker. It's a brilliant web service that simply takes care of your uploads. Uploading a 8 Mb JPG through a little file upload form not only takes an incredibly long time, it's fragile and has no good default UX. Filepicker takes care of all of that and makes it possible to upload files the way you want it. For example, if you use Google Drive to back up your massie pictures, Filepicker can handle that. And Dropbox. And Box. And of course, regular drag-and-drop uploads but with a lovely progress bar indicator and thumbnail preview.

There's also upload by simply entering a URL. So, try find a picture on Google Images click on one, then in the right-hand bar right-click the URL and "Copy Link Location" and paste that into HUGEpic to test.

So for a weekend project that has taken only a couple of weeks I'm quite proud. My hopes for big success is nil but it has been a great learning experience mixing interesting client-side programming, web programming and intereting CPU bound and networking challenges.

Oh, and did I mention it works great on mobile too? Even the file uploading part. Thanks to Filepicker.