Why Do Marketers Test?

Jim Novo recently commented that few online marketers deploy testing the way it’s often done in the offline world. Jim speculates that the reasons for this include cultural issues and a lack of ideas about what meaningful tests to conduct. For me, this raised a more fundamental question:

Why do marketers test in the first place?”

You could argue that the answer to that is simple — to increase the effectiveness of marketing programs. But I think there’s another side of the coin: To improve the efficiency of marketing programs.

In the “old” world of direct marketing, where direct mail costs are significant, marketers test to determine who not to mail to. But in the online world, where the incremental cost of sending out one more email is practically non-existent, suppressing marketing messages is less of an issue.

So why should online marketers bother to test? If online marketing response rates are higher than direct mail response rates, and little opportunity to reduce campaign costs, then marketers will have little incentive to test.

The distinction — and balance — between effectiveness and efficiency is subtle. But marketers who recognize that their testing approaches have been more focused on efficiency than effectiveness will realize that they’re missing many opportunities to create and execute a strategic test and learn agenda that not improves effectiveness but drives marketing strategy.

The notion of a test and learn agenda isn’t new. But many database marketers’ agenda is undeveloped, underdeveloped, or misguided. Often, testing plans are focused on short-term and tactically-oriented questions like who should be mailed to, and which messages work better than others.

There’s a bigger opportunity here, specifically, to test to help answer more strategic marketing questions like:

  • What number of touches is best for which customer segments?
  • How can a sequence of messages help lift response?
  • How does time between touches effect response and conversion rates?

The opportunity to be more strategic with testing isn’t limited to marketing effectiveness. In many firms, it falls on the shoulders of market research to answer questions about consumer behaviors and attitudes. But far too often, market research is burdened with addressing tactical issues. Database marketers can step in and help here, and devise tests to help understand:

  • Which customer behaviors are most closely correlated with response and conversion and help define an “engaged” customer?
  • What the optimal spend per customer to increase customer engagement?
  • What is the profit per customer impact of increased customer engagement?

Database marketers — or online marketers, for that matter — won’t be able to answer these questions if their testing approaches are limited to figuring out who not to touch. Or if they don’t test at all, for that matter. Marketing needs a new mindset about testing, and break out of the confines of campaign-centric ROI test and measurement.

Technorati Tags: , ,


This Amp Goes To Eleven

In a DM News article called “Marketers can seal the deal with new high-tech envelopes”, the CEO of the Envelope Manufacturers Association said:

Next-generation technologies will make the envelope even more versatile and powerful. These include electronic ink [which] can turn envelopes into tiny billboards. The face of mail is going to change dramatically as these technologies are refined and deployed.”

Turn envelopes into tiny billboards? What are they now?

My take: This amp goes to 11. Nigel: “You see, most blokes will be playing at 10. You’re on 10, all the way up…Where can you go from there? Nowhere. What we do, is if we need that extra push over the cliff…Eleven. One louder.”

Technorati Tags: , ,

Marketing’s Civil War

Jacques asks if we’re in the midst of a culture clash between branding and measurement. The answer is yes — and in financial services, it’s more than just a culture clash — it’s a civil war within marketing. And there’s no question in my mind that branding is winning. How else could you explain the following:

Direct magazine reported that Scotiabank “figured out how to measure the impact of its brand advertising.” Since it “already had a handle on direct mail and email”, the bank measured the “success” of TV, print, and outdoor ads in driving consumers to its web site to sign up for a plasma TV giveaway. Since no direct mail or email was sent, site visitors who registered for the contest were credited to the TV spots. According to the article “Scotiabank has joined the parade of brands that’s trying to bring direct marketing’s accountability to brand advertising.”

No mention of how many sweepstakes entrants became customers, mind you.

At first, I couldn’t believe that someone at the bank would say it “had a handle on direct mail and email.” But then I realized that if it was a quote, it probably came from a branding warrior being dismissive of the direct channels.

But — as financial institutions increasingly move from gross to net measurement to gauge the success of their direct marketing efforts — it boggles the mind that Marketing at Scotiabank can get away with this kind of justification on its ad spend, and would even flaunt it as a measure of brand accountability.

Anybody can give away plasma TVs. N-E-Bah-Dee.

That branding is winning this war is somewhat of a paradox, when you consider that many execs — especially CFOs — want better quantitative justification for marketing investments. But they don’t “get” analytics and database marketing like they “get” advertising. After all, everybody is an advertising expert.

Can the measurement/database marketing army stay in the game and make it a fight? Sure. The “Competing On Analytics” Harvard Business Review article and book by Tom Davenport helps — but only slightly. While a number of analytic-types I know see this as proof (hope?) that analytics can rise to the strategic level, I’m not so optimistic. I think it’s more likely that the branding folks will find a way to hijack the term “analytics”, throw away the “brand equity” moniker, and start talking about “brand analytics.”

What will happen in some firms is akin to what happens in real-life civil wars — one side turns to outside assistance to help fight the war. Where database marketers will turn — or should turn — depends on whether their firm is marketing-, sales-, or finance-driven.

In sales-driven firms, smart database marketers will enlist Sales’ help. It won’t be easy, of course, if Sales isn’t happy with the quantity, quality, and distribution of leads generated today. Improving on these areas can be one way for the database marketers to gain Sales’ support to fight the battle with branding. In finance-driven firms, database marketing will align with the CFO organization. Again, the analytic side may have some credibility-building to do.

But enlisting outside help risks exposing the internal battles within Marketing, airing its dirty laundry to the rest of the organization. And credibility-building efforts may take more time than some have the patience for.

In financial firms that are already marketing-driven, I’m not at all optimistic that the analytics side can win the budget and justification battles. In these firms, it will likely take a drop in sales and profitability to force a change in culture and org structure to give analytics a shot at winning. And if that does happen, I’d still bet that the firm either brings in a new CMO from the outside, or promotes someone internally from a completely different function altogether.

On top of all this, remember — this is just one aspect to marketing’s civil war. It doesn’t even address the warring factions within the branding camp, who fight over the distinctions between (as Jim puts it) “branding” and “the brand.”

Technorati tags: , , ,

Notes From The DMA Financial Services Conference

At the DMA’s financial services conference in Florida last week, I got a chance to hear Martha Rogers, co-author of the One-to-One marketing books, kick off the second day of the conference. Some of my favorite quotes:

“There is little pure acquisition out there.” According to Martha, what firms do under the banner of acquisition marketing is “swap customers with their competitors.” It reminded me of the endless stream of credit card offers I get that are oblivious to the fact that I already have three other cards.

“Good customer service is NOT a relationship.”
Somebody recently commented on 1:1 magazine’s blog that “customer engagement” was just a buzzword, and that firms should just focus on building reliable products and providing great service. I don’t buy that — and I’ve written about why I think engagement is something different and important. And I interpreted Martha’s comment as supporting this view.

“Marketing can destroy value.” I didn’t capture all the math supporting this claim (cut me some slack — you try to make sense out of a bunch of numbers after three scotch and sodas). But I think what Martha was saying was that if non-responders were just .5% less likely to respond with each solicitation, after a few campaigns, the likelihood that they would respond would be near zero.

“Marketing has to address just one key strategic issue.”
And that, according to Martha, is to figure out what the firm needs to do to make customers more loyal and valuable, even though competitors will do the same thing. Be honest — when was the last time your marketing team addressed the question “what if our competitors did what we’re planning to do?”. Exactly — that’s never happened.

“It takes 8 quarters to make Return On Customer calculations accurate.” On the first day of the conference, DMA president John Greco said that the average CMO’s tenure is now 23 months. So, basically, if CMOs instituted ROC on day one in their jobs, it’s unlikely that they would be there by the time their firm got it right. (I think I’ll use this as a criterion for choosing between the flood of CMO job offers that I know will come my way — how long ago did my predecessor institute the ROC methodology? If it was less than 6 quarters ago, I won’t take that job).

I’d share notes from the other speakers….but I didn’t go to any other sessions. Too busy sunning myself on the beach, getting massages in the spa, and gorging myself at very expensive restaurants. Thanks for sending me, boss.

[Note: Return On Customer is a trademark of Peppers and Rogers. I have no idea how to cite that properly, so I help this keeps me out of trouble]

Technorati Tags: , , , , , ,

Stop Investing In Customer Retention

Target Marketing reported recently that marketers plan to shift their 2007 media budgets from customer acquisition to customer retention, relative to what they did in 2006.

If this is true for bank marketers, it’s a troubling statistic for two reasons.

First of all, media spending isn’t going to impact banks’ retention rates one single iota. Many banks report 15% to 20% annual attrition among their deposit accounts. Yet the percentage of consumers who intend to switch banks, by closing out accounts, is in the low single digits (Source: Forrester Research). The reasons for this discrepancy aren’t surprising: People move, get married, get new jobs — and, oh yeah, banks screw up from time to time. No amount of media spend is going to fix that.

But there’s another reason. When marketers say they’re refocusing on retention, I think what they’re really alluding to is cross-selling existing customers. But many of these efforts are doomed to fail as well.

Many bank marketers cite research from the BAI published that showed that bank customers were most likely to purchase additional products with their bank within six months of opening their initial account. If that’s true, then trying to sell more products to the vast majority of customers who have more than a year of tenure with the bank is destined to produce a disappointing ROI.

So what should marketers do?

Invest in customer engagement.

Many marketers consider engagement to be a buzzword. But engagement is a valid concept, if you use the term to describe the extent to which your customers interact with you in meaningful, emotional ways. Not just by checking their balances every day, but by relying on you for advice and guidance on how to manage their financial lives and make smart financial decisions.

The payoff is in increased purchase intention. Using market research data, I found that customers who are engaged with their bank are twice as likely to purchase more products from their bank in the near future than customers who aren’t engaged (click here to see how I defined engagement).


While the ROI may not be immediate, an investment in engagement is better than an investment in retention. The key to future profitability isn’t in simply keeping customers — it’s from deepening their relationships. And engagement is a necessary pre-condition for that to happen.

Technorati tags: , , , , ,

Integrated Marketing: A Different Perspective

I lamented a while back that the advertising folks had hijacked the term customer engagement. Unfortunately, that’s not the only one they’ve usurped.

In many circles, “integrated marketing” has come to mean consistent messaging across advertising channels. While that might be one example of integrated marketing, it’s not the only one, and may not even be the most important one.

There are three elements of marketing that are rarely well integrated in firms. The picture below captures the opportunities available for integrating them:


p.s. The graphic in this post was inspired by Jessica Hagy’s Indexed site. If you haven’t seen that site, then go check it out. She’s brilliant.

Technorati tags: , ,

One Small Step For The Credit Card Industry

Mintel Comperemedia reported that more than 9.2 billion credit card-related direct mail pieces were sent to US consumers in 2006.

So let’s see here…if each piece was, on average, one and a half inches high…and we stacked them all up, they’d reach 217,803 miles into the air…

Or 91% of the way to the moon.

Just 900 million more pieces in 2007, and we reach the moon. One small step for the credit card industry….

UPDATE: What was I thinking? The average envelope is nowhere near 1.5 inches high. Ok… look at it this way… the average envelope is 9.5 inches long. If each direct mail piece was laid end to end… it would wrap around the earth 55 times. And one more thing as long I’m updating this: Of the 9.2 billion pieces that were sent out last year… I think 9.2 million of them came to me alone.

Technorati tags: , ,

The Stories Loyal Customers Tell

Becky Carroll’s post on Stories and the Personal Touch reminded me of the stories I’ve heard from loyal financial services customers:

#1: A man in his late-50s, when asked by his bank in a focus group interview
why he was a loyal customer, hemmed and hawed for a few moments before saying “it’s because of Jenny, the branch manager where I bank.” When asked what made Jenny so special, he replied, “I don’t know. But one time I came into the branch to make a deposit, and the pen at the counter was out of ink. Although Jenny had a customer in with her, she somehow knew that pen was out of ink, and came out with a batch of new pens. That’s Jenny for you.”

#2: A magazine reporter and her partner were trying to adopt a child, and had
received word from the adoption agency that a child was available for adoption. But they needed a short term loan in order to make the trip to China to pick up the baby. According to the reporter, her bank “bent over backwards to approve the loan and get her the money in 24 hours” and for that she would “never leave them.”

#3: An IT executive traces his loyalty to USAA back to a single phone call. He called the firm to cancel a credit card and insurance policy. The rep said “I hope I’m not overstepping my boundaries, but we’ve found that many customer often cancel products because of events that aren’t related to USAA like a divorce or other family matter. We’ve set up a special department to help customers with these kinds of matters, is this something we might be able to help you with?” Since he was in the middle of the divorce, he took USAA up on that offer and has been a loyal customer since.

These may sound like unrelated stories, but there are lessons to be gleaned:

  • It takes more than just “great customer service”. I recently commented on the expectations that consumers have of the firms they do business with, one of them being “interpersonal excellence.” The man in story #1 is an example of this. It wasn’t any single interaction that drove his loyalty to the bank — it was the personal attention he received from Jenny and the connection he had with her.
  • Convenience isn’t enough. For banking customers, the added convenience of late branch hours or multiple ATM locations may be important, but the produce the stories that customers tell. In story #2, it was the bank’s operational excellence — its ability to turn the loan app around in 24 hours — than helped produce the story that woman tells.
  • It’s the high-emotion interactions that count the most. Examples #2 and #3 highlight the fact that stories are more likely to be formed during highly emotional situations — like a loan application or divorce. [Colin: This is why the JetBlue response to its Valentine’s Day travel disaster is so much more important than WordPress’ handling of down time. Sitting around an airport is much more stressful than waiting for your blog site to come up]. McKinsey calls these “moments-of-truth”. The challenge many banks — and other firms — have is recognizing these high-emotion interactions when they happen.

So what should Marketing do?

1) Strategerize its “test and learn” agenda.
That’s what USAA did. It posed the question: Why do customers leave? (NOT: What can we do to try and salvage a defection — when it’s probably too late to do so anyway)? Analytics execs should reexamine their group’s test and learn agenda to determine if they’re really asking the important strategic questions — or just refining their knowledge of campaign-level results. (This is a good example of Marketing focusing more on the “macro” and less on the “micro”).

2) Better integrate.
The advertising folks use the term “integrated marketing” to refer to ad campaigns that are coordinated (or the same) across channels. That’s all well and fine, but for many marketing departments the bigger challenge is internal integration — and one prime example is the need for integration between analytics and market research. The two groups need to work a whole lot closer to develop and test theories about customer behavior.

3) Redefine customer segments.
The stories that customers tell are clues into their expectations and the drivers of their satisfaction. Firms that continue to define customer segments by products owned or profitability miss these clues — clues that are more valuable to understanding how to sell and service customers than product propensity models that predict what to sell.

Technorati Tags: , , , , ,

Evaluating Database Marketing Vendors — 10 Questions That Might Not Be In The RFP

I see a lot of RFPs for database marketing assistance. While most go into lots of detail about the requirements the firm has, I’m often struck by what’s not included. Here are 10 questions that should be asked:

1. Operational efficiency. “Can you provide us with specific examples of how you’ve helped firms achieve ongoing marketing cost reductions and operational efficiency gains?”

2. Service levels. “We know you’ll commit to meet the service levels prescribed in the RFP, but can you show us scorecards you’ve delivered to clients that prove you’ve maintained high performance against SLAs?”

3. Data quality. “How do you create unique keys and prevent duplicates?”

4. Scalability. “How have you helped your clients scale beyond current volume — both from a processing as well as from an analytical perspective (i.e., scaling analytical data marts)?”

5. Analytical data marts. “Speaking of data marts, how have you improved extract time and reduced redundant processes involved with the design, development, and management of data marts?”

6. Modeling.
“We’re sure you have excellent capabilities for developing models, but tell us how you’ve improved your client’s models?”

7. Promotion history. “How well have you been able to track, manage, and analyze promotion history?”

8. Cross-channel capability. “What experience do you have in evaluating and incorporating Web analytics data in your work?”

9. Account team structure.
“Are your account team members dedicated to clients or shared across clients — and, if shared, how will that work?”

10. Management stability. “We know your financials are stable, but what’s the tenure at your firm of your senior and account management teams?”

Technorati tags: