Analytic tools for a presidential campaign:

Building a custom BI tool in one sprint

Real screenshot of working service

In a nutshell

This is a story of how we built a working BI tool in one sprint.

It shows how easy it is to screw up if you don't ask questions (even when building for yourself). It also shows that you don't always need to solve all problems. Creating a tool that simplifies the process can do the trick (and then some). Assisting the human brain rather than replacing it was key to the success of this project

Role: Product Designer
Timeline: 2 weeks
Teams & parties involved: CTO, Designer, Deputy Campaign Manager, Project Manager
Context & Goals
How to screw up
Discovery
Reframing
Solution
Outcomes

Context

Early in 2018, we were a couple of months into a presidential campaign. We had opened 20 or so regional offices with over 60 to go. The main goal of the early stages of the campaign was to register at least 350,000 people (but no more than 7,500 per region) to support the nomination.

We needed a way to track how many supporters we had registered, their locations, and if we had enough time.

Goals

Create an easy-to-use BI tool that will allow campaign management to track campaign KPIs, identify trends, and make data-driven decisions.

1
Make key campaign performance metrics easily accessible
2
Provide data to back up strategic campaign decisions

How to (not) screw up

Overconfidence can cost a lot. Always do your research.

This is a happy story about how we almost screwed up (bigly). As a team, we had experience of working together on a bunch of smaller political campaigns. And for a long time, we’d been discussing the need for a proper campaign CRM down the line.

Ideas were circulating, we routinely discussed random features we definitely knew we wanted to build. Wireframes were occasionally passed around, everyone seemed to have some kind of shared vision of a system. The scary part was that the system looked like an extremely complex product to build.

Initial scope & key features:

  • Email and SMS campaign management tool, supporting time and action-based chains of emails and texts

  • Audience segmentation based on geography, user actions, previous engagement and communication

  • Online and offline activity tracking
  • Campaign performance tracking

  • Feedback collection tool
  • Multiple roles with multiple levels of access
  • High-end security and reliability

As a team, we were sold on this vision. But purely out of curiosity, (and as part of my UX-research self-training) I decided to do a set of formal user interviews.

How talking to users can change everything

To better understand our needs and to be able to prioritize possible features, I set up a series of user interviews with people who would potentially use our analytic tool. Because only a few specific roles would have access, I was able to interview almost every single user.

Campaign manager

in charge of vision and strategy

At any given moment I want to see how the campaign is performing overall. What are our current KPIs. What regions are performing particularly well or poorly. What cities look the most promising for the next campaign office. I’m ok with ugly tables, just show me the numbers.

I want us to track all interactions and any engagement with supporters and volunteers so that we can have targeted campaigns later.’
Deputy campaign manager

responsible for finance and operations

I need one place where campaign management can see the current overall state of a campaign at a glance.

If there is a crisis, I want to know about it. If some of the regional offices are failing, I want to know.

I want to see how many real signatures we can expect to get in each city. It would be nice to track the budget and its conversion to our key metrics (registrations and signatures). But I doubt we can build it quickly enough.

I’m ok with raw data and numbers.
Regional manager

works in HQ, responsible for 10-15 regional campaign offices

I don’t know. I probably want to know how many people signed up in each region, how many volunteers we have and if we can contact them
Regional coordinator

works on the ground, in charge of a regional campaign office

I want to know how many people signed up so that we can contact them and convert them to volunteers or donors
Campaign creative director

in charge of outreach and engagement

I want to know how our outreach campaigns affect the key metrics.

If we do local context campaigns, do they perform better than federal for specific regions.

Can regional campaigns affect the federal context?
The candidate

yep, the candidate

I want to understand what works and what doesn't. What actions increase the amount of website registrations and what actions don’t. I want you to be able to tell me this. This is the only thing I care about
It quickly became clear that a sophisticated contact management system was not going to answer any of those questions and that we needed something completely different.

Reframing the problem → New set of questions to answer

Reframing the problem

As much as we wanted to have a mighty CRM, it was more useful and urgent to have a dashboard with key metrics and a system that could track campaign activities and how they affect the number of registered supporters.

Product goals and key questions

  • One-stop place to track campaign performance

  • Understand how various campaign activities affect key metrics (number of people registered on the website, number of volunteers and donors)

  • Understand how many signatures we can expect to get from every region (track ratio of confirmed/unconfirmed registrations, see if supporters live close enough to the campaign office, etc.)
  • Track performance of regional offices
  • Help identify locations for next campaign offices

Solution

Step 0. Setting up the process

After we all agreed on the updated product vision and set of questions, we had one more problem to figure out — scheduling. Political campaigns tend to be messy to manage at the best of times. Being in the very beginning of a very aggressive political campaign in a very aggressive environment while juggling a bunch of projects can be overwhelming.

We knew that the task at hand called for focused work and ideally some headspace, but it was hard to imagine any of that happening at HQ.

Step 1: Turning questions into features

Step 2. Setting up expectations and priorities

It was a time-restricted project for an internal product with only a handful of potential users. We had only one chance to make it happen. We all agreed that our main goal was to deliver a usable product by the end of the sprint.

The dashboard showing the metrics would be our minimum, and any ‘analytics’ on top would be a great success. For the sake of speed, we also agreed on two principles: ad-hoc development, and 0 additional investments in UI polish.

You can see our UI for this internal analytics tool (left screenshot) and campaign website UI (right) showing very similar data.

Step 3. (Mock it) → Build it → Try it → Break it → Repeat

Jumping ahead, these wireframes were our final design mocks. The rest was built ad-hoc; we were sitting in the same room with our CTO and pairing on styling and product issues. Below I'll briefly describe our solutions for each requested feature.

Wireframes and project specs

Track and show key up-to-date metrics

We had 3 main metrics to track: the number of fully verified registrations on the campaign website, the number of registered volunteers, and the number of emails collected.

Showing the chart tracking our key metrics over time was more or less easy (though we had to overcome some technical challenges).

We went through some back and forth trying to understand what visualisation would be the most useful and settled on combining two charts:

Chart showing daily registrations (grey line), cumulative total (red), and all registrations for the selected interval (greyed our segment)

With these charts we were able to see the general state of the campaign (the number of registrations) and the overall trend (if the registration rate is slowing down or speeding up) as well as make rough predictions if we had enough time. It was also easy to spot exact days/weeks when something happened that caused an unusually high or low number of registrations.

As you can see, we also added a bonus feature: the ability to select a specific interval to see the exact amount of registrations during that time (to estimate the impact of certain events).

Track performance of campaign offices

To track the performance of campaign regional offices we added the ability to filter all data by region. This helped us to rate their performance, and to identify the best locations for future offices.

Predict total expected signatures from each region

Just showing the number of registered supporters would have drawn a far too optimistic picture. For a more realistic prediction, we showed

Screenshot of the product section showing the total numbers, key factors influencing the estimated numbers, and conservative estimations for our KPIs

Help to identify locations for next campaign offices

To rank suggested locations we simply sorted the regions without campaign offices by the number of registered supporters. To push it a little further, we added lists of best and worst-performing offices. Based on this data, we decided when and where to close or open new campaign offices.

Understand how different campaign activities affected key metrics

Figuring out the relationship between campaign events and the key metrics was the most challenging part of the system to build (or even to comprehend).

Every single day A LOT was happening. Even when we discussed possible ways of creating this feature, it sounded overwhelming: it was extremely hard to track all relevant events (online and offline), not to mention trying to analyze their possible effect on specific metrics.

Finally, we decided that we would leave two major actions up to humans:

Our product would simply show the timeline with key metrics and the list of events. To make it easier to draw conclusions, we added the ability to change the timeline scale and the ability to select a specific time period. This allowed users to view only relevant events happening at a specific time + corresponding changes in metrics.

Screencast of the real product in action (sorry for the low image quality)

To make it easier to tag events, I came up with a way to categorise all types of events that might be interesting to us and created a set of tags and instructions.

The key was not to build a ‘smart’ analytic tool that could track and connect every event to its measurable outcome. Even if such technology existed we had no time to implement and fine-tune it to make it useful. Luckily we had access to something better — human brains. Not trying to substitute the human brain with technology, and allowing human intelligence to be part of our product became our silver bullet.

Outcomes

  • We created and implemented a working analytic tool, tailored for our exact campaign needs in 2 weeks
  • It turned out to be so popular that we had to grant access to way more internal users than we initially expected
  • Campaign management used this tool daily for tactical and strategic decisions
  • As a team, we were able to make many discoveries that would have been impossible to make without the tool. These discoveries helped us better understand our audience and move more quickly

Lessons learned