In [part 2 of Inside Jidometa], we discussed our implementation of caching for JSON responses.
The initial goal was to solve the “slow page load” issues, but after a few months of running this in production, we eventually found ourselves battling more and more odd race conditions.
In retrospect, caching was a bad idea
I knew this from the start, but we were quite confident in our ability to implement a proper cache invalidation scheme to remove any possible race conditions.
The reality is that as our app grew, new features were added, and those features introduced more places where race conditions would surface.
Finding the root cause
I tasked myself with the goal of finding the exact root cause of our slow page loads. This required the disabling of caching, and then optimizing of SQL queries.
Our SQL backend was performing simple and multiple SQL queries per page load (~30 queries per load). This was extremely inefficient and un-necessary.
I identified every page which had such issues, and managed to reduce and combine queries down to 2 or 3 queries per page, thus significantly decreasing page load time.
Some numbers
Without caching, loading a page (or API call) with ~700 builds would previously take nearly 30 seconds.
After my optimizations, this was reduced to 300 milliseconds, or 0.3 seconds. Those changes were sufficient for us to completely remove our JSON caching implementation, and also fixed other random issues/race conditions as a side-effect.
The lesson here is: caching sucks. Don’t do it unless.. just don’t do it. It’s pretty much always a hack, and will eventually somehow someway end up biting you in the rear. Even if you’re a seasoned expert in race-condition handling and cache invalidation, other things which you didn’t even know exist will surface and cause headaches and random failures – because of caching. The typical approach to solving caching issues is to add more caching (see Intel CPUs), or to perform more ugly hacks to work around the caching issues.
I’m a fan of using no caching, and then moving on with my happy life ;)
Moving forward
We released Jidoteki Meta v1.8 at the beginning of June, which included these changes and a few other extremely useful features.
We’re continuously working to improve Jidometa in order to provide the best, fastest, and most reliable way of consistently building virtual appliances designed to run on-premises.
As usual, feel free to contact us if you’re in the process of or thinking of providing your app to enterprise customers. We’ll be more than happy to help.