Speed Boost - Part 2 - RPMWare + Amazon S3

It’s been a couple days since we pushed our latest performance update live and we’re at it again.

First a little on the update we pushed a few days ago… Our internal statistics are showing about 70% fewer database calls, and pages loading about twice as fast. We’re pretty happy with that, but always looking for that extra edge; we’re speed freaks after all … I’m sure you can relate.

Today’s Update

We just pushed another update live that should give us another bump in performance.

First we tweaked our last release. As much as we test, there is no substitute for real-world performance metrics. We scaled back on how much we were caching and lowered the time objects stay in the cache. We’re not expecting anything groundbreaking here, but every incremental improvement helps when you’re serving millions of pages a month.

Second, we’ve moved all our media storage and hosting to Amazon S3. If you’re not of the geeky persuasion feel free to gloss over the rest of this. The bottom line is your site will be faster because the servers that power your site are no longer also serving static image files. If you’re interested read on …

Amazon S3 is a scalable storage system that provides the storage for Amazon.com, the other Amazon websites and now thousands of independent sites using Amazon Web Services. Think of S3 as a huge, redundant, fully backed up file server cluster with a huge pipe connecting it to the net.

When you (or we) upload images to RPMWare we generate 7 versions of that image (different sizes), save the largest to our servers and then send all 7 versions to Amazon S3 via a REST webservice. When customers are browsing your site the RPMWare servers generate your pages and the Amazon servers serve up the media files directly to their browser.

Because Amazon is HUGE and some of the other sites using S3 are HUGE we benefit from a number of economies of scale, the first of which is leveraging infrastructure that only HUGE companies like Amazon can afford. I have no idea how many servers are running S3 or how much bandwith they have available, but it’s more than we could ever dream of for RPMWare.

It’s completely redundant and the chances of losing anything are extremely small, as close to impossible as you can get.

Our servers have more resources to do what our servers do best. By transferring the bulk of our static file serving off-site we’ve got more power to throw behind powering the core features of your site. Everything will be faster.

It’s infinitely scalable. We could add 20 TB of data overnight without blinking an eye although we’d probably melt our credit card. While that’s far fetched the theory behind it is the same. We can add more and more images and media to our product catalog and not worry about where we’re going to store it all.

Changes

If you’re linking to RPMWare images offsite we’re going to continue to support the old images indefinitely. You’re offsite images will not stop working; however, to take advantage of the speed of S3 you may want to upgrade your links.

Moving Forward

The next step will be moving all your client files to S3 and increasing the upload limit from the current 1GB. Don’t expect this right away but it is on our todo list.

Questions - Comments

We’d love to hear your questions and comments regarding the latest changes. Feel free to leave a comment on the blog, shoot us an email or call us anytime.