Tribune DataViz

Matters of interest, from the data reporters and developers across Tribune Publishing

Author Archive

I suck at this

leave a comment »

I’ve wanted to program computers ever since I was 14 or 15, and now that I’m 31, I can honestly say I have always known that I suck at programming computers. I’ve come to terms with this knowledge. It doesn’t bother me like it used to. I know I’m bad at my job, but I also know that it doesn’t matter what I think — it matters what I do. [Ed. note:  Abe is awesome at his job. He just doesn’t believe it.]

I take on tasks, close tickets, and steadily get better at not sucking quite so hard. But since this feeling of inadequacy seems to be common among programmers, let’s take a tour through my insecurities! I’m not sure why you should come with me, though, it probably won’t be that good.

May 15, 2000 — This date is made up, but it takes us to junior year in high school (go Dolphins!) One of my best friends, Scott, is a year ahead of me in programming classes – he’s taking AP Computer Science, and I’m just taking a programming class. He shows me the code he’s working on, mostly hard-core graphics for video games, mostly in C++. I don’t show him the code I’m working on, mostly crappy websites (the Wayback Machine preserved my Geocities Starcraft fanfiction abomination!)

At this point, I’ve been a fan of programming for a few years, but it hasn’t really clicked for me yet. Everything I write is basically copy-pasted from an O’Reilly book. (Sidenote: remember books?)

May 15, 2005 — This date is also made up, but it roughly corresponds to the day of my one truly rock-star college programming moment: the day I took (and passed!) the final exam for CS 440, Topics in Artificial Intelligence.

Getting a B in that class was one of the hardest things I’ve ever done. I barely understood what was going on; I’d leave lectures in a daze after 45 minutes of a professor saying words that I didn’t comprehend. I’d turn in homework assignments that at best failed in interesting ways. I got celebratorily drunk the night of the final, drunker than I’ve ever been before or since (Note to college kids: Tequila and Red Bull tastes delicious but it is such a bad idea, oh my God).

May 15, 2010 — Let’s just keep doing May 15ths every five years. Now we’re in California, and I’ve moved from a non-technical job answering emails from news publishers to a slightly-technical one, writing Python scripts that don’t work very well to interact with a poorly-maintained database that I am largely in charge of.

I’m not on a technical team, but I’m surrounded by extremely accomplished developers on related teams, and every time I sit in a meeting with them I barely say a word, for fear that they’ll realize I don’t know what I’m talking about and should probably be fired. Every time I have to make a change to the database, I spend the day with my stomach in knots, just knowing that I’ll screw it up somehow and everyone will find out. I dread every time I submit my code for review, certain that each time will be the one they discover that I’m a fraud and need to be eased out of the company.

May 15, 2014 — We’re more or less at the present, and for the first time in my life, I feel comfortable and confident showing up every day to write code. I’ve been at the Tribune for a little over a year; for most of that time, every Monday I would wake up nervous, worried that when I got in and checked my email, disaster would be waiting in the form of angry emails alerting me to an embarrassing bug (or worse.)

Every Monday this would happen, and at my previous job as well. But now it’s stopped happening. I’m not quite sure why, but I think it’s because I’ve realized that insecurities are not the same thing as reality. I know I’m not a very good programmer, but as I get better I realize that things that used to seem immeasurably more complex than anything I could understand are not so daunting.

The code I write is still bad, but it’s markedly less bad than it used to be. And that’s all I can do: to try hard, to steadily get better and to accept that I am going to make a lot more mistakes, some of them embarrassing, some of them catastrophic. There’s no other way to improve. There’s no other way to do what we do.

Watching the World Cup, I was reminded that this is the mentality that athletes must have — the cameras are on, the entire planet is watching, and a defender makes a misstep that lets a striker get by him for a goal. His entire home country thinks he’s terrible now. But he can’t let that bother him.

The next time that striker comes down the field, the defender has to challenge him, confident in the knowledge that this time, the striker’s going to look like an idiot. That’s all he can do. That’s all any of us can do.


Written by Abe Epton

August 20, 2014 at 3:08 pm

Posted in Uncategorized

From weekend hack to actual project

leave a comment »

Sometimes, you get an idea so good you can’t wait for Monday to code it up and show your friends. (If you’re reading this, you have weird friends.) You tell yourself the idea is simple, pretty straightforward to implement and you have some time on Sunday afternoon for a fun project. Maybe you just want to get a prototype built to explore the idea, confirm it makes sense, demo it for your boss and team members. After building a few of these, you find yourself repeating common mistakes, coding yourself into the same corners over and over again. But it doesn’t have to be that way.

If you’re like me, your hard drive is littered with folders bearing now-obscure names, the tombstones of weekend hack projects that never came to fruition. Which is fine: most ideas are bad and most weekend hacks won’t go anywhere. But this post is for the ones that make it past the haze of beer and sleep deprivation to actually become something real. I have two pieces of advice, seemingly contradictory, that will make your projects better and help you enjoy them more.

The first and most important thing I’ve learned is that it’s always worth doing it right the first time. The old carpenters’ adage applies to the craft of software: measure twice, cut once. By writing code correctly when you first get started, you make it so much easier to maintain later. Generalize that set of lines you copy-pasted 3 times with slightly different variable names! Decompose that hundred-line function into smaller pieces you can reuse elsewhere! Take five seconds and write a comment explaining why that magic variable has to be set to 11 when the code first runs – you’ll be glad you did when you come back to it weeks later while tracking down a bug.

The reality is that not only does it rarely take that long to write clean code, it only gets easier and faster the more you do it. If you’re working on a passion project over the weekend, use the time to instill good habits that you’ll be happy to have later on. I know it’s easy to just copy-paste an answer from StackOverflow and be done with it, but take an extra few seconds – that’s really all it’ll take – to clean it up, harmonize the variable names and indentation levels and write a comment or two. The amount of time you’ll spend reading this code looking for bugs probably dwarfs the amount of time it takes to tap out a 10-word comment, and that comment could be a lifesaver later on.

That said, don’t worry about the boilerplate. When you have an idea and can’t wait to get started, it’s easy to lose momentum by dragging yourself through the muck of creating the right folders, installing libraries you won’t need until way later, and getting hung up on command-line syntax.

If you’re working on a weekend, or for fun, you have the freedom to explore what interests you. You’re not required to eat your vegetables before you get to dessert; it’s a Sunday! Start with the cake, and eat the carrots if there’s time.

Let’s say you want to learn the Twitter API, and you have an idea for a bot that tweets rugby scores. You can start by just trying to write code that sends a tweet. Then focus on the gritty scraper you’ve got to write in order to get the rugby data. That scraper is gonna be a huge pain, filled with corner cases and unexpected gotchas, and generally not that much fun to write. You might think you need it first – your tweets need data! – but spending an hour sucking down data and parsing it before you even get to the thing you actually wanted to do in the first place is just going to demoralize you. The energy you had when you first got started is valuable; don’t waste it on boilerplate code.

Those two points may seem contradictory, but they really boil down to the same thing: don’t be afraid to take a step back and prioritize. What are you trying to accomplish and what will help you keep your momentum going? Some tradeoffs are worth it – take the five seconds to document your weirdest code – but some tradeoffs don’t make sense. That scraper can wait.

Finally, don’t try to get it all done at once. Know how to tell when you’re at a good stopping point: when you’ve captured the basic idea you were trying to implement; when you’re about to start going down a rabbit hole of documentation for a new library; or when you just need to take care of a bunch of special one-off situations that you think won’t take that long, but inevitably suck up endless hours. Write up a bunch of notes to yourself in the comments, try to transcribe everything that’s currently stuck in your head, make explicit all the next steps you’ve got mentally planned out. You won’t remember them in the morning, because now the sun is coming up and you’re going to step back, go for a walk and try not to get burned out. You still have to show your coworkers the beautiful demo that, mere hours ago, was just a gleam in your eye.


Written by Abe Epton

April 7, 2014 at 9:37 am

Posted in Uncategorized

Scraping Illinois campaign finance data – for fun, if not profit

with 3 comments

From time to time, Illinois has experienced campaigns that have involved shenanigans. Whether it be the ancient (2008) days of “I’ve got this thing and it’s f***in… golden” or more recent tales of stuffed elk heads and Michael Jackson’s fedora, the state’s political history is dotted with politicians who’ve come up with unique and sometimes illegal ways to raise and spend campaign money. It hasn’t always been easy to quickly track where the money comes from.

That’s why today we’re releasing the first version of a Twitter account and Python library designed to make it easy to keep track of Illinois campaign contributions. The Twitter account, @ILCampaignCash, is powered by the Python library,, which in turn takes advantage of data collected and released by the Illinois State Board of Elections. At the moment, the Twitter account is exclusively focused on A1 contributions – donations of $1,000 or more that campaigns are required to report within 5 business days – and so the library is optimized to process those reports, although it can also provide some details on quarterly D2 reports as well.

The Twitter account is designed to both notify readers of any A1 report filed, no matter how small – including those of the absolute minimum size required by law – and to highlight individual large contributions. Any $10,000 or greater contribution gets its own tweet.

We plan to keep working on the library and improving it as necessary for future projects, and so we welcome any feedback or suggestions from the community.

Written by Abe Epton

January 6, 2014 at 4:41 pm

Posted in Uncategorized

From Google News to the Chicago Tribune: Observations after a month in the newsroom

with one comment

In my five years at Google News, I’d barely ever heard anyone use the phone or raise their voice. But the newsroom at the Chicago Tribune, a 165-year-old urban stalwart, is a much more boisterous cubicle suite than anything at the Googleplex.

During my first month at the Chicago Tribune, the city hit 500 homicides for the first time in several years. When the 500th came, police first ruled it a homicide but then reclassified it as a “death investigation,” a middle ground that didn’t quite meet the murder threshhold in a city trying to tamp down murders. The noise that followed of reporters questioning sources was remarkable.

It doesn’t have free food (unless you steal from the fridges, or someone buys you a round at the Billy Goat), but the people watching is much more interesting.

And the sense of connectedness to the outside world is much stronger, particularly as the dozen reporters within earshot of me are all trying to make sense of breaking stories — running theories and possible sources by each other, badgering contacts and occasionally, yes, oh my heavens, yes, committing acts of profanity.

The cultural differences run deep, but there are some intriguing parallels between my current and past employer: They’re both obsessed with information and their respective roles in civic and democratic life, and fantastic places to write code and learn about the world.

While things were still fresh, I wanted to think about the things that strike me about my new, old-media gig and why I think people interested in code and civic society should really think about joining a newspaper — yes, a newspaper (or any media outlet) — in 2013.

To begin with, the newsroom is one of the most connected places I’ve ever seen. Information flows through it constantly, via phones and police scanners and loud clackety-noise-making machines of indeterminate purpose (not the nearby typewriter), as well as the ubiquitous Tweetdeck and Chartbeat screens. We have our morning meetings on a television stage elevated in the middle of the room.

But it’s the informal networks, the overheard chats among reporters and editors and the one-way conversations with sources, that provide a unique and fascinating education on how stories are put together and what’s going on in the city.

There’s an obsession with data here as extreme as any I witnessed at Google, but the relationship reporters have with stats is complex. Visible everywhere in the newsroom, and even during the meetings that decide what makes the front page every day, there’s a constantly-updating Chartbeat readout of that second’s top-performing stories.

It’s part of the cosmic microwave background of the paper, but intriguingly, performance metrics are only allowed to dictate so much. If the editorial decisions were guided solely by where users click and readers read — they are not — then we’d run 50 pages of sports and obituaries every day, with the occasional crime story thrown in, especially if it’s weird. (Hello, prison break! Is that you, mysterious cyanide poisoning what is bulimia of the chemicals nowadays of a lottery winner?)

A huge amount of effort is expended on totally ephemeral products: Stories that retain their interest for a day, at best, and then never get read again. The yearly article whenever the first snowfall hits is the same every year but somehow different each time — different sources, maybe a reason for a different angle or excuse for a pun early on. Someone has to spend a couple hours putting that story together, only to be read the next day — if it runs at all, if it snows at all — and then forgotten.

It’s humbling to imagine the work of the mammoth printing presses, churning out millions of pages of paper every day to sell, with the sole intent of creating something people read once and abandon.

In contrast to Google, where everything was tested to within an inch of its life and often built entirely in-house, at the Tribune, we run almost entirely on off-the-shelf open-source stuff (with some exceptions) and maintain an active Github repo of the stuff we build. That means I’m playing with more industry-standard technology than I ever did before.

It also means that it’s much easier to break things here, since changes to something live don’t require signoff from multiple reviewers and a complex testing and deployment process that take months to begin to understand. The difference between these two approaches is enormous, and both have significant benefits.

It’s comforting to know how hard it is to break something at Google, but it’s frustrating to wait forever before anyone ever uses your code. At the Tribune, I had some Javascript running in public by the end of my first week. And I barely know Javascript, but the demands of a continuous news cycle require a lot of flexibility, which also means a lot of variety in what you work on.

Today alone, people on my team are working on, among others, a crime project, amicrosite devoted to the Book of Mormon musical and a package looking back at the Tribune’s bankruptcy. Aside from being a constant stream of new stories and technologies to learn, these all represent something fascinatingly different from the missions of most Silicon Valley tech companies: individual storytelling.

At Google, an ostensibly-neutral platform provider, getting a project approved that had any specific focus was nearly impossible. The concern went beyond not wanting to alienate users by taking a stand on controversy; we were constantly trying to work on projects that had the largest possible userbase.

If a billion people wouldn’t be able to use what we were building, it didn’t make sense to keep building it. At the Tribune, not having to please a sixth of the world’s population makes it possible to spend an afternoon working on projects like analyzing underutilized schools and kids with custom tote bags in Chicago, which can be much more interesting and fulfilling than internationalizing a little-used contact form into 40 languages.

But what’s most exciting to a politics geek with some coding experience is how immense the opportunity is. Millions of people will read the stuff being written across the cubicle aisle from me, and the technical infrastructure that supports it is as primed for disruption as any I’ve ever seen. And no matter how dire the state of the journalism industry (by the way, it’s not as bad as you probably think it is), more people are reading the news than ever before.

Journalism is in the midst of being reinvented, and the products built in newsrooms like this one, all across the country, will wind up driving public debate and understanding about important topics for decades to come. I look forward to being a part of the transformation.

Written by Abe Epton

January 23, 2013 at 4:06 pm

Posted in Uncategorized