Tribune DataViz

Matters of interest, from the data reporters and developers across Tribune Publishing

Deploying ProPublica’s awesome table-setter tables to S3 using Fabric

with one comment

Yesterday, our friends at ProPublica released the source to their tremendously useful table-setter application. (Which our own Brian Boyer helped develop during his time there.) In brief, table-setter allows you to quickly generate beautiful interactive tables from CSV files or Google Spreadsheets. For a really great example, see this table documenting stimulus dollars spending (click through for the interactive version):

As you can see this table has sorting, highlighting, graphical components–the works.

These tables can be generated dynamically and hosted on a Rack-capable webserver or they can be built-out to static files. While we are perfectly capable of deploying a Ruby application, it was this second part that really peaked our interest.

Fast, robust and dirt cheap

After a bit of consideration, we decided to use the same solution we use for other static content: S3! By deploying the HTML and all assets to S3 we can serve these tables with no overhead to the rest of our operations. It takes five minutes to setup and will literally cost us a few dollars a month. And by doing it this way we dodge the twin bullets of configuring and maintaining a Ruby environment.

As we have shown before, when deployment is the question, Fabric is the answer. It was about an hour’s work to produce a script that accomplished the following things:

  • Rebuilds the tables
  • Removes the table index and makes tables individual files in the root directory, rather than index.html files in their own directory (this cleans up the URLs since S3 does not support default files for directories)
  • Gzips the tables and all non-image assets
  • Synchronizes the entire directory tree to our staging or production S3 instances

You can see the completed script on Gist. And because we have a DNS redirect for our production S3 domain, our URLs in end up looking really nice:

We would like to thank ProPublica for releasing this really tremendous tool for putting tabular data online. Gracious sharing such as this is what makes open source work.


Written by Christopher Groskopf

March 12, 2010 at 12:14 pm

Posted in Apps, Infrastructure

One Response

Subscribe to comments with RSS.

  1. […] a comment » We’ve blogged before about what an awesome tool Propublica’s TableSetter is. Its a fabulous way of presenting […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: