Fanzter Tech 2011

By Joshua Warchol on 30 December 2011

What's changed since my first post on the technologies that power Fanzter's products? LOTS!

We often get asked by other entrepreuners, engineers, and even former co-workers, "what's your stack?". I thought that it would be interesting to give a high-level overview of the technologies we use and how our stack has evolved over the last few years. These technolgies helped us to serve over 20M users in 2011.

We've kept the best of what's worked well for us (Ruby on Rails, Solr, Memcache), improved on a few components, and brought some fresh players into the game as well. Our playbook builds on three guiding principles: agility, simplicity, and performance. We're a small team devoted to building the world's best consumer internet products. Our tools are the equipment we field to win the day. As such, we choose them carefully and evaluate their usefulness over the long-term.

No(t just)SQL

The star new player on the team is Percona Server with XtraDB. Their motto might as well be "We don't make MySQL, we make MySQL better." We've recently migrated our primary database over to Percona Server 5.5 and the results have been quite pleasing. The changes they've made make the performance characteristics of MySQL smoother and more predictable. And the improved diagnostic controls allow us to dig in and answer questions we previously had to guess at. I highly recommend anyone using MySQL check out Percona Server.

NoSQL databases systems have had a huge year in 2011, and our team has been working with these technologies throughout their gestation and growth periods. On the NoSQL side we've got Redis and MongoDB powering components of our operations. We've shifted much of the heavy lifting in our web apps into Resque worker jobs, and our in-house analytics (or as we call it, Fanalytics) is based on Sinatra and MongoDB. We like the speed and flexibility of these new players in the data storage space, but have been careful to prioritize shipping new features to our users over engaging in the lustful pursuit of new technologies for their own sake.

Taming The Cloud

Fanzter's products have always lived "in the cloud", but how we manage the servers that run our products has changed dramatically in the last year. We've pulled in the reigns of our server management and "in-sourced" it using Chef. We've been able to combine open source "recipes" from the Chef community with the system integration tools from OpsCode to simplify our server deployment and reduce our costs. Best of all, Chef recipes are written in Ruby, our preferred language.

Not every product is served from our own servers. We've moved some sites and services to PaaS juggernaut Heroku. The page you're reading is served off Heroku as a matter of fact. Working with Heroku's platform has been very interesting. While the platform has experienced some growing pains, stability and capability continue to improve.

Fanzter continues to focus our cloud efforts within Amazon Web Services (AWS). We've added their Simple Email Service (SES) and Simple Notification Service (SNS) to our acronym soup mix. We've even released a little ruby gem to make it easier to tie the God monitoring agent to SNS. Look for more open source software from us on Github in the future.

Monitoring & Performance

Like many of our peers, we're adicted to data. Graphs, live streams, slicing and dicing to distill knowledge from numbers. In the last year or so we've nurtured this habit with expanded tools and services to help us better understand our applications, servers and users. We've upgraded our subscription to New Relic's application performance monitoring platform and licensed it for even more servers. Every new server we launch with Chef is automatically added to their recently added Server Monitoring service as well. Being able to drill down and pinpoint performance or availability problems quickly is key to allowing us to sleep well at night, and to keep our operaitonal staffing lean and mean.

Besides New Relic, we've also added to our display components. My favorite is a set of Hubot-based interactive agents in our company Campfire chatrooms to report on iOS downloads, social media interactions, and we've built a live world-map that can show different types of activity from our various products as they happen, where they happened. Finally we've added the Graphite scalable realtime graphing system to our kit. This, combined with the extremely handy StatsD system allows us to quickly add metrics to tracking without adding configuration burden.

The challenge isn't collecting the data, it's having enough LCD monitors to display it all!

2012 is going to be a huge year for Fanzter and we've got a great, fresh toolbox to work with. Sound good? We're hiring!