Grow Smarter, Faster: How Axios drives engagement with user-level data
Would you consider your relationship with a reader that opens 100% of your emails the same as someone who opens 0%? Obviously not. Then why do we still think in the aggregate when looking at email list metrics? Most publishers remain mired at the campaign-level, ending their newsletter analytics exploration with list-level open and click rates. Are there ways to go deeper?
When we sat down to speak with Rameez Tase, Axios’ VP of Growth, we were treated to a law that beautifully encapsulates the problem and the solution.
Let’s call it Rameez’s Law:
“You don’t simply get a 50% open rate by having a 100% opener and a 0% opener. You get two distinct cohorts that you act upon in different ways.”
Simply put, some of your audience is engaged and some isn’t. So why do we treat them all the same? We should not measure success as an aggregate, but instead try to understand if the right people are highly engaged.
This is the essence of Rameez’s Law, and there is beauty in its simplicity. Think about your audience at a deeper level than averages and aggregates. By moving beyond campaign-level stats, down to cohorts and user-level data, you start to grow an audience while building lasting relationships, maintaining trust, and paving a solid, sustainable future for the company.
In this article, we lay out what Axios is doing to achieve this. We’ll cover a number of frameworks to help elucidate why this approach works: Goodhart’s Law, Andrew Chen’s Law of Shitty Click-Throughs, a DTC brand mentality, and much more.
So how does Axios do it? How does a media company leverage user-level data to rapidly grow, while still maintaining trust and loyalty?
Before we dive further, let’s take a look at who Axios is.
Who is Axios?
We’ll borrow our description of the company and its founders from a post we published last year:
“Jim VandeHei and his two-time co-Founder, Mike Allen, perfected the art of digestible, must-read daily emails with Politico’s Playbook. Axios takes the elements that made Playbook a success, and instills them across an entire company. These elements are summed up in the term ‘Smart Brevity’, which implies no wasted words, and no wasted space.”
Rameez brings experience from both the editorial and the financial worlds. He was an analyst at Deutsche Bank and The Blackstone Group before transitioning to a three-year stint as VP of Audience and Analytics at Mic. This mix of business and journalism is omnipresent at Axios offices – for example, skilled analysts know something about the business side, and people skilled on the business side know a bit about analytics. This creates a virtuous circle where everybody makes each other smarter.
The Two Funnels
A big part of Rameez’s job is to weed out the paid and organic levers that produce engaged newsletter subscribers. Web traffic is one funnel – these are site readers who haven’t subscribed. Out of the 10M unique monthly visitors they get on their site, Axios has 1.5M non-unique subscriptions across all 19 of their newsletters.
The other is social, where Rameez can find people who represent the ideal Axios reader, but have yet to encounter the brand. As social has plenty of tools to help isolate preferred demographics – lookalikes of Axios’ engaged users – this is the ideal channel for finding loyal readers.
Thinking about these as two very distinct funnels is another example of a segmentation mindset. One potential subscriber knows your brand and goes directly to you. The other you’re going out and locating yourself. The way you work to convert these two cohorts will naturally be distinct.
CPEEA: Cost Per Engaged Email Acquired
CPL, CPC, CPI… in Axios’ eyes, these measures fail to capture the true cost and ROI of acquiring an email.
Many media companies would agree. If they’re running social ads to get newsletter readers, they may measure CPEA (Cost Per Email Acquired). It’s a step in the right direction, but Axios goes one step deeper.
They think about CPEEA (Cost Per Engaged Email Acquired), which takes into account unengaged user churn. An email address isn’t just a line item in Axios’ book. To understand the true efficiency of your marketing spend, you should understand whether you’re investing in signing up engaged readers.
“Getting an email address is the lead, converting that email to an engaged subscriber is the goal,” says Rameez. Once you have them, you need to keep them.
Maintaining reader engagement
When Axios asks something, their community responds: surveys get a 10% response rate. Their vanity metrics are also way above industry standards (they average a 45% open rate). Engagement is, by all measures, through the roof.
But Rameez has to add a caveat: “one of the things with email analytics, they always tell you to do more.”
In other words, Rameez is saying that email stats will never tell you to do less. You’re not going to get metrics indicating you need to ease up on ads when click rates are high, or that you should push hungry readers less content if they’re consuming everything you send.
“You never know when you have eroded trust by that next metric,” says Rameez.
He notes that it takes intuition and an objective outlook to balance engagement and avoid losing trust. Measuring trust is difficult, and often materializes after a breach has taken place.
This is where we need to look at Goodhart’s Law.
Charles Goodhart is an economist whose famous adage states that “when a measure becomes the target, it can no longer be used as the measure,” according to Business Dictionary.
Wikipedia gives the example of a car dealership where the success metric is the number of cars sold. This inevitably leads to employees optimizing for sales at the expense of profits. People will alter their behavior, potentially for the worse, to maximize on what is measured.
A typical Axios newsletter has six to ten “cards”, each summarizing a noteworthy news event. People can skim until they hit stories they enjoy, read top-to-bottom, or anything they like. What’s important is that they’re engaged.
What’s less important is a metric like time spent reading – a bit of a holy grail in the newsletter world. As Rameez notes, “the easiest way to keep someone reading longer, is to make the content… longer.” It’s tongue-in-cheek, but it captures the essence of Goodhart’s Law.
That doesn’t mean Axios doesn’t pay attention to the overall list. List size and open rates are metrics advertisers care about, after all.
But when Rameez thinks about his role, he sees cohorting as a way to make human decisions more manageable.
Now what in the world does that mean? To answer this, we’ll first have to take a look at another law.
The Law of Shitty Click-Throughs
Numbers are great at Axios. They have some of the best stats in the industry.
But Rameez is well aware of the dangers of having a false sense of comfort. Newsletter consumption is linear – it takes 5x the amount of time to read five newsletters vs. one, and a reader has limited time. How can they push the business forward knowing this limitation?
At some point, Andrew Chen’s Law of Shitty Click-Throughs starts setting in: “Over time, all marketing strategies result in shitty clickthrough rates.”
Axios wants real brand equity. They know the age of plenty won’t last forever. Rameez’s team is doing everything they can to future-proof the business.
Part of this effort is obsessing over the correct measurement of lifetime value.
Focusing on User-Level Data
Acquiring engaged readers reaps continual benefits. For the most part, “an 80% open rate user begets another 80% open rate user” when they share the newsletter. For the sake of consistency, let’s call this Rameez’s Law #2. (Surprise! There’s two.)
“We really want to focus on who is this person, what value can we bring to them, what motivation can we provide them to share this product with their personal or professional networks,” says Rameez.
With user-level data, Rameez and his team are able to hone in on distinct groups with social retargeting, surveys, ambassador reward plans for signups, and other efforts to bring more highly-engaged readers to Axios.
Cohorting is a way to make human decisions more manageable. Can you imagine blasting these campaigns to your entire list? The zero-percenters and the hundred-percenters? It would be absurd.
User-level data and segmentation are there to bring the right people into the community and keep the business sustainable. So is a quality product readers can trust.
Axios caters to a premium group of loyal readers (decision makers) in an intimate setting (email). This means expensive sponsorships. Which allows them to afford slightly more sophisticated marketing efforts on the cost of acquisition side. Which grows their list. Which grows their business.
Thinking like a DTC brand
“We need to think much more like a direct-to-consumer brand.”
Some in the journalism profession might whip out the garlic and wooden stakes at this point. But hear Rameez out:
“[DTCs] are very sophisticated because at the end of the day, they need to sell a product. Casper or Brooklinen can’t really fake it. If they don’t sell a mattress, then they wasted that money acquiring the email. Media companies have been able to fake it when your customer and reader aren’t necessarily the same.”
A traffic-based or display-advertising-based media business isn’t held accountable for the things a DTC is. They can fake things for a while, arbitraging paid traffic against display ads, but this doesn’t translate to an engaged community that will stick around.
That’s both an opportunity and a challenge for Axios: the business’s future depends on a readership that reads. And people read Axios.
The challenge is, growing a portfolio of newsletters could decrease per-edition engagement because there are no economies of scale in newsletters. Recall that reading five newsletters takes five times as long.
So what does Axios do? They start crunching numbers.
Calculating reader LTV
Rameez often looks to the future. He’s already considering the cost of app downloads on Facebook, and a subscription plan:
“With a download on Facebook costing something between $20-25, and 20% of active subscribers might download, you’re adding $4-5 LTC to every subscriber. If you think about a paid subscription product, let’s say a $10 a month deal, renewed once, that’s $240 per subscriber. So if 10% of subscribers join, that’s adding an additional $24 of LTV.”
LTV can be a dirty topic for a journalistic publication, and a difficult one to measure in an advertising-based business, but this level of strategic thinking is mandatory. Growing an engaged community is the smartest thing Axios can do now.
Besides qualitative discipline – maintaining unique personalities across newsletters, creating content that doesn’t waste readers’ time, providing a diverse set of newsletters to choose from – Axios is able to grow an engaged community by leveraging user-level data.
Organizational data accessibility
Axios isn’t segmenting users for a marketing automation algorithm, but instead to make data more accessible to humans. Remember, they’re cohorting to make human decisions more manageable.
They’re dealing with a community in the millions, yet even on the growth and analytics side – which in most organizations is a relatively detached, numbers-only group – they are putting the human element first and foremost.
Axios manages to avoid the vicious circle that most organizations face when they don’t heed Goodhart’s Law, blindly follow the metrics, and disintermediate themselves from their audience (losing trust in the process).
Axios instead establishes a virtuous circle of organizational transparency, inter-team communication, a multidisciplinary workforce, and organizational DNA that is rooted in every team member.
The secret sauce is to “marry the the person with business context with the person with data infrastructure context, and they both make each other smarter…. Also, anyone can get the data they need,” says Rameez.
A newsletter data stack
Maintaining this virtuous circle is dependent on everyone having the necessary tools. But in the newsletter world, “all ESPs have a host of chokepoints because the technology of email is pretty archaic,” says Rameez.
When newsletters are your bread and butter, there are few tools on the market truly geared towards your goals. Everyone we spoke to for this series, from Tedium’s lone warrior Ernie Smith, to Smartbrief (a two-decade newsletter veteran) has custom-built tools to help them “newsletter” better.
Axios is no exception. They have a custom-built CMS called Eden. This hyper-personalized tool allows for smooth newsletter publishing, with sections like “Go deeper” available in the CMS editor as a button, and the ability to publish on the site and newsletter in one fell swoop.
Axios takes readability very seriously. According to Rameez, the CMS is built to force writers to write mobile-first. Apparently, some writers write entire 1,500+ word newsletters on their phones. You heard that here first, folks.
Axios still uses third party tools. Sailthru’s API is used to send emails via Eden, while also handling the data collection and aggregation. Axios uses Data Exporter for event- and user-level data.
“We [also] use BigQuery and Tableau. This allows us to move from campaign-level data, to breaking down building blocks and reassembling them into user-level data.”
Axios is a lot of old-school notions of journalism wrapped up in business savvy and the latest publishing tech. From early-AM news cycles and an unbending determination to hold up journalistic integrity, to custom-built tools; from user-level data analysis to a DTC brand’s mentality.
But for all the robust infrastructure, industry know-how, and quality writing, it all leads back to that simple adage:
“You don’t get a 50% open rate by having a 100% opener and a 0% opener. You get two cohorts that you act upon in different ways.”
Axios is successful because it’s able to see the obvious in something so obvious it goes over most media company’s heads.
You don’t treat your engaged users and unengaged users the same. Once you’ve internalized this, you can start thinking about your email list in completely novel ways. Rather than just checking a box for hitting a campaign-level open rate, you can start thinking about how to build deeper relationships with your engaged users. You can work to scale your audience, while maintaining the trust required to build long-term relationships and realize a higher lifetime value. It’s a critical mindset shift when you’re building a valuable, sustainable media business.