Juicer: Optimizing a SaaS Web Application for Enterprise Scale

Challenge

Juicer is a software as a service (SaaS) product that allows its customers to aggregate their social media feeds. Customers, ranging from Fortune 100 companies to bloggers could connect their social media accounts and receive a Juicer embed code which they could place on their website to display all their social media posts from their different social networks in one single beautiful feed. From then on, every time a customer posted on their social networks, those news posts would automatically show up in the Juicer feed on their website.

Juicer had rapidly grown to the point where it was regularly pulling in hundreds of millions of social media posts per day. As a social media service it was also prone to uneven load distributions, particularly during huge real world events. For example, the Giro d’Italia, one of the most famous cycling tours in the world, had a Juicer feed embedded on their main website. During the 20+ days of the event, that single feed would get 10s of millions of page views each day and they were just one of Juicer’s 10s of thousands of customer feeds! On top of this, premium product features such as customers paying for high frequency post fetching, reordering posts or fetching any post with a common hashtag had the potential to place extreme demands on the service. If Juicer was going to continue to prosper it was critical to get ahead of these requirements and optimize the system for an enterprise level of scale. At the same time, it was critical that these optimisations be introduced so as not to disrupt the service for existing customers in any way.

Solution

As is often the case in software development, no one single thing will fix everything, instead you need a multi-pronged approach in an ongoing process of making incremental improvements which are tested, validated and deployed. The Juicer app was essentially a single endpoint where each customer’s website connected to get the relevant post data. From the first month of operating the service a simple step to reduce load issues was to cache the endpoint so that frequently requested data could be offloaded from the backend servers. However, this approach could only go so far, given that some Juicer users were paying to update their feeds every minute, any cache would rapidly become invalidated.

It was clear that in many cases data was going to have to be retrieved from the database; there would frequently be a high volume of database queries. Given that many users would pull in social media posts using a common hashtag like “#love” as a feed source, this could mean pulling 100 posts a minute, 144,000 posts a day, 4,320,000 a month for a single feed source, and Juicer had hundreds of thousands of feed sources. As such, the database storing these posts was hundreds of millions of entries, approaching terabytes of size. Making these queries and requests for data from the database efficient was critical. Without these optimizations even simple queries could potentially take forever. One slow query could grind the whole thing to a halt, hogging database connections while the app continued trying to add thousands of more posts every second.

One of the common queries, from customers would be to pull in posts in a specific order. Therefore, the first step in optimizing these queries was to create partial indices on the database. Basically you pre-order the posts table to handle the different possible “orders” a user might request. Each time you create one of these indexes you are making a whole new copy of the posts table and re-ordering it. Normally, you might run a migration to achieve this, however during this process all connections to the database are blocked, and this would mean a disruption for existing customers, whose feeds would no longer come in at a consistent rate. In Juicer's case we therefore ran asynchronous migrations, which ran alongside the active database and also kept pace with it, by also processing any new posts which were coming in in real time.

Extensive time was also spent in the PostgreSQL query planner, which is basically a tool that explains how the database is running a query and gives some insight into why some queries are running faster or slower. From there queries could be fine tuned by adding indices, or adjusting postgres settings such as memory management. For each change it was important to test, measure and see how the database was affected. As part of testing a suite of performance tests was created, in this way queries could be run on a development or staging environment and any changes in response times could be measured. This was an ongoing process as when new customers emerged with new feed query parameters further query optimisation was often required to satisfy these new use cases. Through this patient and consistent approach the query was ultimately restructured so that it was perfectly optimized to meet the needs of the business and its customers.

Result

Through this process, Juicer was able to achieve a level of peak performance where the database could handle anything thrown at it. From a business point of view this meant the team could rest easy knowing that the site could respond and deliver to all Juicer customers no matter what kind of feeds or load spikes might occur.

In terms of query speed, we managed to get our server response times for even the most complicated queries down to under 100ms, the threshold of perceptibility for humans. This allowed Juicer to provide its customers with a lightning fast user experience even when receiving hundreds of millions of requests.

The technology behind Juicer was now perfectly primed for growth and the team knew that the underling software wouldn’t limit them from scaling and growing the business further. This enabled the small team at Juicer to focus their energies on growing the user base. Able to push on with marketing and product features, Juicer was now able acquire new and exciting clients with enterprise scale requirements such as Lyft, Ralph Lauren, Sony Music and Porsche and the business went from strength to strength.