uWSGI, or why you don't need Varnish

As a web developer one of my main goals is performance. In this blog post I explain how we have boosted the performance of Crowdcrafting without touching the PYBOSSA code or adding any extra layer to our stack.

Boosting your web service performance

If you are developing a web service you know that you have to cache content in order to serve lots of requests quickly, right? You might be using a memory cache like memcached or Redis, like we do. However, sometimes this is not enough because the request stills go through all your pipeline only saving time from accessing the DB or computing a difficult value. Moreover, if you have a distributed load-balanced high-available cache (as we do), the request will take some time in retrieving the data from a node. Therefore you will end up summing some precious milliseconds to that request just for fetching a value that has been already computed (I always picture the requests like these skaters running to get to finish line).

When those milliseconds are precious, then you are looking for caching the whole request, not just some data in the DB.

If you are looking for a solution to this problem you will probably find Varnish a web application accelerator also known as a caching HTTP reverse proxy. There is a lot of documentation on the web about it, and just to be fair we consider it for some time but we decided to avoid it for a single reason: our infrastructure uses cookies to handle sessions (we use Flask-Login) and this makes things really complicated.

Looking for alternatives: uWSGI cache capabilities

As I’ve explained in previous blog posts I love to keep things simple, so after checking Varnish and all the issues that it will bring to our stack we decided to check the capabilities of uWSGI regarding caching (I even opened an issue on Flask-Login about not using cookies for anonymous users in order to use Varnish with no much luck).

uWSGI has a very powerful plugin system that allows you to customize how your web service will behave. For example you can use the internal rooting plus the cache route plugin for caching specific requests based on some rules that you configure.

In the uWSGI Caching Cookbook, the explain step by step how you can do it for almost every single scenario, however the examples are very generic and you will need to work out your own rules to fit your project.

An example config file for uWSGI where you cache all the pages would be the following:

[uwsgi]
plugin = router_cache
chdir = /your/project/
pythonpath = ..
virtualenv = /your/virtualenv
module = run:app
processes = 2
; log response time with microseconds resolution
log-micros = true

; create a cache with 100 items (default size per-item is 64k)
cache2 = name=mycache,items=100

; fallback to text/html all of the others request
route = .* cache:key=${REQUEST_URI},name=mycache
; store each successfull request (200 http status code) in the 'mycache' cache using the REQUEST_URI as key
route = .* cachestore:key=${REQUEST_URI},name=mycache

This set of rules are very simple. It will cache every request that returns a 200 status code in the cache. This config file is really nice for project where the site is delivering content and there is no much changing.

However our site has a mixture of both things, pages that do not change too much over time and pages that have to be adapted for each user (specially for registered users).

Dealing with cookies and sessions

As I’ve said before Flask-login place cookie for anonymous and authenticated users. Hence, all users have a cookie, but authenticated ones have an extra one in our project as this cookie is used to remember the session of the user for a period of time.

Thanks to this configuration we can know that a user is a registered one if both cookies exists, or the other way around: we can know if a user is an anonymous user if only the remember me cookie does not exist.

Using this knowledge we can instruct uWSGI to cache some URLs (i.e. front page, about page, etc.) only for anonymous users, as they don’t need tailored information. If they sign up then, instead of serving their cached request we will process the request as usual (remember that we’ve different levels of caches, right?).

To us, for the moment, the most important aspect to cache is what anonymous users see, as this segment is what’s driving most of the traffic to our site. Now that we can distinguish between authenticated and anonymous users, we basically configure the uWSGI like this:

route-if = empty${cookie[remember_token]} goto:cacheme
route-run = continue:

; the following rules are executed only if remember_token is empty
route-label = cacheme
route = ^/about$ cache:key${REQUEST_URI},name=cache2
route = ^/about$ cachestore:key=${REQUEST_URI},name=cache2

The above example caches for anonymous users the about page of our Crowdcrafting site. When the cache is clean, the first rule will fail, so it will process the request, stored it in the cache and then served it. Next time the same anonymous user or another one request the same URI, the cached request will be served boosting the performance a lot. Simple, right? Now you only adapt this snippet to your own URIs and web project and you will have an amazing boost in performance. Best part? That you don’t have to touch a single line of your source code. Amazing!

Registered users will never receive any cached request with this configuration. You could cache for every user each URI based on their remember_token cookie however that will require lots of memory and it will defeat the purpose of having a cache: that lots of requests are already served from the same data point. Having a cached item per user is useless on this regard, as you will be loosing performance. In this case it’s is much better to cache at the data level, as all the users would benefit from it: anonymous and authenticated ones.

Summary

Thanks to this solution we’ve improved our performance a lot. Before these improvements, the average response time of our servers were close to 250ms and now all of them are responding in average below the 50ms. Saving 200ms is incredible! Most importantly because we’ve not added a new layer or anything special to our own stack. We’ve just configured it better!

NOTE: The heading photo pictures the filament of a light bulb. To take the picture the photographer used a micro lens, and I’ve always pictured uWSGI as micro WSGI ;-)