Today I learned that nginx requires to setting the
proxy_cache directive to use the configured cache for reverse proxy setups. Without this directive, even with configured caches nginx will simply ignore the cache and proxy things directly. Along with this, I also learned about the usefulness of setting a caching header
add_header X-Cache-Status $upstream_cache_status;, which will send along with the request whether the Cache was successfully hit or not. If the header is not sent, the cache is not configured correctly.
I came across this while putting some more work into my libravatar-nginx project, which aims to implement the most important parts of the libravatar API without requiring any further software than nginx and statically pre-generated images. The answer was brought to my attention by a StackExchange answer.