Playing with filestream_iterator

There are several ways of serving static files from Zope. The simplest way is to just do something like this:

size = os.stat(file_path)[stat.ST_SIZE]
REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg')
REQUEST.RESPONSE.setHeader('Content-length',int(size))
return open(file_path, 'rb').read()

The disadvantage with this is if the file is say >1Mb, the whole 1Mb needs to be loaded into RAM before it can be sent. This is surely garbage collected afterwards but if you serve many of these at the same time, there's obviously the risk that the RAM gets too full.

The alternative is to use filestream_iterator from the ZPublisher which is an iterator that sends out chunks of the file at a time. Note that to use the filestream_iterator you must set the Content-Length header first. Here's how to use it:

from ZPublisher.Iterators import filestream_iterator
...
size = os.stat(file_path)[stat.ST_SIZE]
REQUEST.RESPONSE.setHeader('Content-Type','image/jpeg')
REQUEST.RESPONSE.setHeader('Content-length',int(size))
return filestream_iterator(file_path, 'rb')

I did a little unofficial benchmark to test if there's any speed difference between the two approaches. I'm currently only interested in serving photographs (jpg) in this app and I'm sure other results would be found if you did bigger filesystem files. These jpgs are in average about 200 Kb. During the course of writing this I tought I'd also try using sendfile as well to see if that makes it any faster:

Serving 526 jpgs with...
function             average total time (5 tests)
  open()               5.07750701904
  filestream_iterator  4.32704296112
  sendfile             4.431672620774

Interesting. Using filestream_iterator shaves off a few milliseconds but it's over many many servings. Conclusion, the benefit of filestream_iterator is not speed, it's RAM (un)usage.

Note. From reading the LocalFS source code I noticed that they only bother with the filestream_iterator if the filesize is greater than 128Kb. I copied that too into my app now but don't understand why. Is there a disadvantage of serving tiny files with filestream_iterator() Anybody?

Worth noting that sendfile wasn't any faster but I suppose you only see a speed boost with sendfile on much larger files.