Many companies have been in the same boat. You launch your site, traffic slowly trickles in and then BAM! Your product is mentioned in a Buzzfeed article, traffic suddenly spikes and your site goes crashing down. Site crashes can cause serious customer experience issues and can prevent potential customers from coming back to your site later on.
So how can you prevent site crashes or slow site speed from happening?
The key strategy is to offload as much of the traffic. When you receive transactions or non cacheable requests, dynamically scale horizontally. This will allow scalability of the infrastructure to handle traffic spikes.
How to Offload Traffic
There are a few ways in which you can offload traffic from your site:
CDN (content delivery network) caching
A CDN (such as CloudFlare, Akamai and CloudFront) acts as a layer on top of your website. It stores a copy of the initially served web page, so every subsequent request will hit the CDN and in turn be served a cached page. Since you are not pulling the page from the infrastructure, it decreases the load on your system.
When applying this strategy, be careful that you only cache “static” pages. If you were to cache transactional content, it could lead to the CDN displaying another customer’s order. Which you, and your customers’, do not want.
Data layer caching
Typically what ends up happening in a traditional multi-layer infrastructure is a lean and fast web layer, but a database layer that gets hit from all sides. When you increase the number of web servers, your database is panicking and then decides to just fall over. The idea here, again, is to offload the requests to a more scalable layer.
One of the strategies within data layer caching is to add a value into your memory data layer. The idea is to pull data for a specific request and store it in the cache based on a key (a combination of parameters from the request). You then store the result set into the cache as the value. The next time the same request comes through your system, it will be pulled from the cache instead of hitting the database again.
The three most popular choices for data layer caching are memchached, redis or riak. If you are using an e-commerce package, check to see if it automatically allows you to specify one of these endpoints to use. Other more home grown systems will require that you add the endpoint into your data access layer.
Running on the Amazon Web Services or AWS Cloud is recommended as they provide a lot of tools to help scaling automatically based on load or schedule.
Make sure that your assets and media are centralized.
If you are running Magento or WordPress/Drupal or another CMS that stores its uploaded media on the server, you need to move that to either S3 or to an AWS Elastic File System (EFS).That way as you spin up new server instances, you will automatically have the media and content that is needed.
Make sure you are storing any session data in Memcache,Redis or a centralized data store.
Typically web servers like to store session data on the server. So if I have multiple servers and the load balancer pushes me to a different box, then all of a sudden my shopping cart is empty or I am logged out of the site. You can also enable session affinity on your load balancer, but that is not recommended for scalability.
Make sure you have an easy way to bring up a new server instance.
If it takes 20 minutes to bring up a new web box, that may not work out so well. AWS allows you to create Amazon Machine Images (AMI) of your server instance and spin up new instances based on that AMI. There are many fancy customizations you can do post launch as well, but that is a different blog post.
Create an AutoScaling Group and Launch Configuration as well as your scaling thresholds.
So that is the web tier… but wait… there’s more. You need to also consider using a cluster of caching servers (AWS Elasticache is a wonderful way to do this).You also need to possibly beef up your database layer. There are many ways to enhance your databases, such as using read replicas, multi-master setups, clustered setups, sharding, NoSQL solutions, etc.
As you can see each layer can be a paper all on its own. We hope this gives beginners guidance and a starting point for managing traffic spikes. There are also service companies who specialize in managing AWS, such as OneCloud.
OneCloud Systems LLC, is an AWS Certified Consulting Partner that provides Technology Architecture, DevOps Support, and Cloud based server management (cloud architecture, migration, infrastructure design, implementation, 24/7 monitoring and support, etc).Learn More About Our Services