[Read part 1 of Inside Jidometa]
We recently implemented a feature which greatly improves the browser experience in Jidoteki Meta (Jidometa).
In the past…
The Jidoteki Meta UI is a simple JavaScript layer on top of a PicoLisp REST API. The API returns JSON responses for every successful request, but occasionally we would see rather slow navigation between sections of the UI.
Certain responses tend to take more time as the size of data increases. There is no way around this, since many tables and entries need to be queried on every request, or so we thought..
The present…
To “solve” this problem, we decided caching responses for common GET requests would be a good approach.
Our idea was to only cache the most queried items (the list of all builds, and the details for each build). By caching this data, it makes it possible to quickly navigate between different builds and the full list of builds, without making any calls to the API.
Caching gotchas
Before implementing any caching solution, it’s important to fully understand the gotchas that accompany it. In most cases, the most important requirement is to never serve stale data. To invalidate the cache requires us to ensure any cached elements which are affected by a change, will immediately be evicted from the cache.
Another gotcha, is avoiding the temptation to add yet another layer of complexity to the stack. We always aim to do things as simply as possible with the least amount of software and overhead.
We needed the ability to easily disable caching or wipe the cache if needed, and we needed to ensure the browser’s own caching features didn’t interfere with ours.
Caching as simply as possible
Since our Jidometa virtual appliance (as well as our customer appliances) run entirely in memory, it was easy for us to assign a temporary directory for storing cached responses.
We store responses in /tmp/jidometa/cache/ - which is not actually “on disk”, but rather an in-memory filesystem (tmpfs). Those responses are regenerated on every GET request to the API. Fortunately there’s no overhead for that.
Each cached response is a simple .json file which is absolutely identical to what would have been served by the API. The files are named according to the build ID they represent, so there’s only ever one cached response per build.
It’s hard to get any simpler than that.
Serving the cached responses
We use Nginx as a frontend for static files and the API. Below are examples of the two Nginx directives we added for serving the above cached files:
location /cache/ {
add_header Cache-Control no-cache;
alias /tmp/jidometa/cache/;
try_files $uri @nocache;
}location @nocache {
rewrite /cache/builds/(.*)-details.json /builds/$1/details break;
rewrite /cache/builds.json /builds break;
proxy_pass http://jidometa;
}
As an example, a request will first try to serve the /cache/builds.json from /tmp/jidometa/cache/builds.json - if the file doesn’t exist, it tries the @nocache directive. That directive will then send the request to /builds on the API, which will return a fresh response (and generate a new cached builds.json).
We also send the Cache-Control: no-cache HTTP header to ensure browsers don’t cache the cached response… but here’s the thing, browsers WILL cache the response regardless, except the no-cache directive will force the browser to use the Last-Modified or ETag headers in future requests, to revalidate whether the cached entry was changed or not. Well, assuming browsers handle those headers according to the RFC specs ;)
If the cached file hasn’t changed, the browser won’t fetch the entire file again.
Invalidating the cache
Builds are linked to their parent build (if any), as well as their child builds (if any).
When a certain build is modified (ex: the build’s status is changed from unreleased -> released), its cached response needs to be invalidated. We decided the simplest and easiest way to invalidate a cached entry is to delete the build’s cached response, and all the other responses linked to it (its parent and children).
Here’s the code we use for invalidation (PicoLisp):
(de remove-cached-file (Filename)
(call ‘rm “-f” (pack “/tmp/jidometa/cache/” Filename)) )(de remove-cached-build (Build)
(remove-cached-file (pack “builds/” Build “-details.json”)) )(de remove-all-cached-builds (Builds)
(mapcar ’((N)
(remove-cached-build (; N 1)) )
Builds ) )
A call to (remove-all-cached-builds) with the list of Builds as the only argument, will handle removing each cached response individually. What’s nice is we can also use (remove-cached-file “builds.json”ˆ) to remove the entire builds list from the cache.
Of course, we wrapped all those calls in a simple (invalidate-cache) function which can be called anywhere in our code; anywhere that updates an existing build, or inserts a new one.
If a build has one parent and three child builds, then a total of six cached responses will be removed from /tmp/jidometa/cache/, including the builds.json (list of all builds). This is quite an aggressive cache invalidation strategy, but it ensures there are never any stale responses, and a new cached response will be regenerated as soon as a request is made for the “missing” file. Easy.
Issues with caching
We have yet to encounter any issues with cached responses, but if they do occur, it’s quite simple to identify exactly which response is problematic, as well as delete it manually from the cache directory. It’s also simple to completely wipe the cache from the system and start over from scratch. This happens after reboot anyways.
The main issue with a fresh cache is the first page load will be slightly slower than the others. It might take a few clicks to “warm” the cache with fresh responses, but considering the use-case for Jidometa (on-prem virtual appliance), we can live with that. Had there been a requirement to support hundreds of concurrent users, we would have considered an in-memory cache with persistence, such as Redis… maybe.
In conclusion
I would go into further detail, but really there’s nothing more to it. Serving cached JSON responses the way we do is somewhat of a “poor man’s solution”, but the lack of overhead and the simplicity of the implementation are a net win for everyone.
Stay tuned for part 3, where we’ll provide more details regarding the Jidometa internals.