If you have tried to check in yesterday, you might run out of luck as Foursquare was down for 11 hours. To many, it was way too long for a downtime. Foursquare wrote a blog post to explain what happened.
According to Foursquare, the database Shards did not perform as what they should. Instead of spreading the database across the Shards, it consolidated at only one.
The team tried various ways to rectified the problem but it became worst when they tried to add another Shard to the system. Due to unknown cause, the site went down.
The next (many) hours, the team tried their best to make the site up.
Click Here to Read the Full Story
Some new changes were made by Foursquare after this incident.
- Downtime and ‘we’re back up’ messages will be tweeted by @4sqsupport (our support account) and retweeted by @foursquare
- During these outages, regular updates (at least hourly) will be tweeted from @4sqsupport
- We’ve created a new status blog at status.foursquare.com, which will have the latest updates.
- A more useful error page; instead of having a static graphic saying we’re upgrading our servers (which was not completely accurate), we’ll have a more descriptive status update. Of course we hope not to see the pouty princess in the future…
Hopefully, the team can come up with backup plans in case such things happen again.