Math By Mint

Someone once said that there are lies, damn lies, and statistics.

Personally, I think that there’s a hierarchy there. Anybody can lie. If you’re good, you can tell a damn lie. But you have to be really good to use statistics. Alas, I don’t think Mint.com makes it to the top tier.

At the Finovate conference in NY, Mint.com presented and shared some numbers that piqued my interest. It claims that it “manages over $12 billion in transactions” and that 50% of its users have changed their spending behavior by using Mint.

Manages transactions? Perhaps “tracks” would be the more accurate word. And how does Mint know that the behavioral changes have come as a result of its tools — and not the general economic conditions?

Then there’s Mint’s boast that it has identified more than $100 million in potential savings for its users. This is ridiculous. Imagine if Capital One claimed that it has provided more that $100 gazillion in consumer financing by adding up the credit lines offered in the marketing offers it has sent to customers and prospects.

And finally, according to its CEO, Mint.com currently has 500,000 users, enjoying a sign up rate that has “more than doubled” in the past 3 weeks.

Sounds pretty impressive. When Aite Group spoke with Mint in April, it had 220,000 users, and (we were told) was signing up 10,000 people each week. In its August 18th press release, it said it had 400,000 users. So in the eight weeks leading up to the Finovate conference, it added 100,000 users. Which, my calculator tells me, is 12,500 users per week, and just 25% more than the weekly rate from April.

I’m not calling Mint.com a liar. I believe that it has doubled its sign up rate. After all, with the economy going the way it has, who isn’t more concerned about managing their finances? It doesn’t surprise me at all that the sign up rate is increasing. The enrollment rate might have dipped during the summer for all I know.

But let’s put Mint.com’s numbers in perspective. It may very well be the largest “online personal finance service.” Online. On Javelin Research’s blog, Mark Schwanhausser implies that there are 9 million Quicken users. If Mint.com grew by 40,000 users per week from here on out, its user base wouldn’t equal the Quicken population until we’re ready to throw the next president out of the White House at the end of October 2012. At 20,000 users per week, the 9 million figure will be attained while you eat your Thanksgiving turkey — in the year 2016.

But this is a silly discussion. Enrollment isn’t the critical statistic — the percent of enrollees that are active users is the critical factor (any online banking exec will tell you that). Intuit will tell you that, too — it has sold a lot of copies of Quicken to people who don’t use it. At least the firm got paid for each copy it sold.

Here’s my request of Mint.com: If you present at the next Finovate, please don’t share BS statistics (that’s not “Bachelor of Science” by the way). Instead, tell us, of the people who have enrolled, how many are active? How many use the site on a daily or weekly basis? How many site features do they use? How many accounts have they aggregated? How does usage change over time? How many of the saving offers that have been presented to users have been accepted? We’d be a lot more interested in finding out how much Mint users have realized in savings from Mint offers — not just those that the site has identified.

The reality of the PFM space is that while money is really really important to us — and getting even more important with the economic uncertainties of the day — the process of managing our money is something that few people want to do, or enjoy doing.

Quicken has succeeded by finding a segment of the population that likes to track, trend, graph, and forecast their financial lives. Mint.com has done a great job of replicating a lot of that functionality, and going beyond Quicken by making it easier to get data in (through aggregation), provide offers for savings, and engage users with its blog.

This segment is small. Mint.com is deluding itself (and trying to delude others) when it says its tools are for the masses, and sites like Wesabe are for the hardcore PFM users.

The key behavioral change that will determine the success of the PFM space is not how many dollars in transactions it “manages.” It’s getting more people to become more disciplined about managing their financial lives. There’s no doubt that tools like those offered by the PFM vendors can help people become more disciplined. But this is not an inconsequential change in behavior. Effecting this behavioral change is as tough as quitting smoking, losing weight, or something like that.

Which PFM site or business model will be the one that is most successful in helping to effect this behavioral change? I have my bet, which I won’t share here. But I will say this — for all of Mint.com’s chest thumping, the PFM game is far from over.

Technorati Tags: , , , ,

17 thoughts on “Math By Mint

  1. Funny thing – I actually asked some of these very questions (% of active users, average number of log on’s per month per user, time spent on site per visit, usage trends, etc) at their booth after their presentation. I was politely told that Mint does not disclose that information. I am curious about how they decide which information to disclose, and which to keep close the the vest, and why.

    I missed the claim about “managing” transactions, but would probably put that off as a mis-speak.

    As for the claim that they have identified $100M in potential savings for its users, I can see this being quite doable, statistically speaking, depending on their definitions (and here is proof that statistics can often be misleading). For example, if they were to identify just $1.67 in savings per month for each of their 500K users, thats $10M in savings per year, and over a 10 year period (definitions!), thats $100M. True, yes. Meaningful, no. And like you said, “realized” savings is a much more impactful data point.

  2. Thanks for bringing Mint’s claims to our attention in this way. This reminds me of the early days of the web, when people talked about hits on their web site, which includes every single graphic element requested, and every robot spider crawl. Yes, these are indeed hits, but you could increase your hits by a factor of 20 just by slicing one graphic up into 20 different pieces. Yes, hits are a true metric, but equally as meaningless as Mint’s savings claim. And in 2008, I hope no one is bragging about their total hits.

    This is a meaningful as saving $1 million per second, each second I don’t purchase a beachfront house in Maine. Oops, just saved another $1 million. And another. Wow, this is amazing! In a fraction of the time it’s taken me to write this comment, I’ve saved more money than Mint has identified for its users!

  3. Some really good points. My uncle once told me “statistics are like bikinis – what they reveal is interesting, what they conceal is vital” – both funny and true.

    I’d agree that no one has any good metric today to measure success for PFMs – the only metric that might possibly have any meaning is a ratio of sign ins to registered users. I registered with Mint a year ago to check it out (after Finovate actually), and have signed in a total of twice since (as usual, I’m probably a edge case).

    It may be that many of these PFMs can help people reach the goal “to become more disciplined about managing their financial lives” – but while all the players are offering something for some type of user – no one has offered something compelling (yet) to “the masses” that just makes it so obvious, easy and pervasive that people flock to it and incorporate it in to their daily lives (like Google, the verb). I don’t think the problem is too big or complex – I think the _vision_ has been too complex. As an industry, retail financial services must provide consumers with easy, clear information – the new PFMs are getting closer – but not there yet.

    Not that I have an answer – if I did, I’d be pitching VCs 🙂 Or maybe I am…………..(I’m not).

  4. Thanks Ron for pointing this out. I was looking for this type of coverage from Finovate since I was only “virtually” attending.

    I think that features are a main telling point in PFM…what gadgets each company has that will give promise of “helping people help themselves” and how they spin the use of those things will make the differentiation. We all want to see what we spend, but we aren’t all the same. There has to be different players in this market (as you indicated in your post the different niches for Quicken and Wesabe) because peope are different; however if the niche gets too small, the group must be able to identify with the others in that group and be willing to pay the price to be in that group. If you are talking to the masses and focusing your efforts on a small niche, the ones you want to feel “special” and “targeted” will get missed.

    Sorry to blabber on…still getting used to posting my thoughts…I do enjoy your perspective…

  5. Hey Ron, I am one of their valued subscribers.

    I am also one of their subscribers who has not added their checking account because Mint.com does not support my bank.

    How many of their 500,000 members have bank accounts with banks that mint.com does not support.

    When I was beta testing quicken beam, the SMS balance checking service, my bank worked after a few days of technical difficulties with the beta beam service. When I upgraded my account to quicken online, my checking account was automatically included. Johnny Come Lately just may have their act together.

    @dmgerbino

  6. Ron and Morriss…thanks for attending Finovate. It was great seeing you again. And Paul, I don’t think we met but thanks too for attending.

    Wow, you covered a lot of ground here. Absolutely great point that getting people to change spending behavior is on par with quitting smoking. It’s a slow, sometimes painful process. But millions have quit smoking and I think the same will happen with PFM usage. Mass market? Probably not. Significant market? Yes. Especially once integrated tightly into online banking. (Personally, I just want a “what crap am I spending too much money on” button I can hit 2x per year).

    Regarding metrics shared on stage: No doubt Mint is sharing the numbers that make them look the best. No surprise there. I agree that it would be nice if they shared more metrics, but also with several dozen competitors in the space, I can understand why they wouldn’t.

    I disagree a bit on the irrelevancy of the “savings identified” metric. While the $100 million number is a bit like counting website hits, it’s not nearly the folly as Morriss’s (is that too many s’s?) second example, “not spending $1 million per second.”

    In what I’ve seen when using Mint, these are real savings someone could get by making a relatively simple transfer of money from one deposit account to another. As Ron suggested, the more-telling metric is how many savings suggestions are acted on. But that’s a core metric to Mint’s business model and something they understandably would shield from competitors.

    Ron, what did you think of Best of Show winners this time?

  7. Jim, thank you so much for having me at Finovate. I appreciate the opportunity.

    I’m not familiar in detail with Mint, but I know that some of the savings recommendations it makes are to credit unions that the viewer is not eligible to join. So the fact that they are counting such as “savings” puts their entire savings number into question. There are many dimensions to savings, or switching accounts and/or financial institutions, important vectors of which include convenience, reliability, service, ease of use, product tailoring to fit. If they recommend that you close your checking account and switch to one 3,000 miles away to save or gain $100-$300 per year, that gets counted as well. But is that really helpful? But I shouldn’t talk, I’m not totally familiar with how Mint works.

  8. The “statistics” quotation is Benjamin Disraeli. And you’ve got wonderful points going here. What I want to know is whose ass I have to kick to get them to update the URLs for my Credit Union. I’ve e-mailed twice and still no response. So, no, my money’s not being tracked.

  9. @Jimmy Thanks for your comment. I think the ass you HAVE TO kick is that of Yodlee’s, since that’s the firm behind the account aggregation.

    However, if you’re looking to kick asses, maybe we can chat offline, because I’ve got a list going that you might be able to help me with.

  10. @Jimmy/Ron,

    Ok, so I’m not ready to have my ass kicked, but if you have a particular site that’s not available on Mint, my team can help.

    We’ve got a backlog of >500 sites that folks have asked for, and we’ve added 1430 new sites this year alone… so these things don’t happen instantly. But I can see if we can bump the queue a bit for you.

    Ron, since you own the page, feel free to email us together or send Jimmy my email address.

    Regards

    Peter Hazlehurst
    SVP Products
    Yodlee

    (ps… and no, I’m not going to comment on the statistics stuff :))

  11. @Peter,

    thank you for commenting on this blog. I must apologize for my negative Mint.com and justthrive.com posts I have made here and at other blogs.

    It is Yodlee who is not up to speed with my banks website. Also, both Mint and Thrive still point to the wrong website. I have open cases with both, but still no solution. Time will tell.

    @dmgerbino

  12. Pingback: PFM: The New New Year’s Resolution « Ron Shevlin’s Marketing Whims

  13. First, thanks for a great thread, all – I’ve enjoyed reading the comments.

    Before diving in, I should introduce myself: I’m the behavioral psychologist at Thrive. I started working here precisely because of one of the point’s of Ron’s post: personal financial advice should be about behavioral change. And while we all here seem recognize that as the goal, I’m not convinced all the new personal finance companies do and I certainly think we disagree about how to do it.

    It is funny that Ron mentioned stopping smoking and losing weight, as I’ve run studies on both. I spent a good deal of time working in an eating lab and am entirely convinced, from personal experience, the wise guidance of senior colleagues, and from the vast amount of literature out there, that tracking alone isn’t the answer. Counting calories produces very little true change and any weight loss associated with it tends to die out very quickly. Watching where your money goes has the same problem: it stops working fairly quickly.

    Nor does research tell us giving you an endless supply of tips is the way to go. Tips are like fad diets: they work for some people, some of the time, for a little while. They’ll get you past a rough patch, but they won’t smooth out your future, and they remain fundamentally different than an integrative strategy. I’m taking tips different than advice here, to be clear: tips are generally specific and non-personal, where advice is based on an understanding of your larger personal context.

    Thrive’s approach of organize, advise, plan is just one of many ways that PF’s can work towards real behavioral change: I certainly am not trying to claim that no other one is valid. But we developed this based on the huge amount of behavioral psychology/economics research that is out there, as well as research from the rest of psychology, particularly self-control, judgment and decision making, and mental perception.

    And I can safely say it will only get better. I’m truly excited, both personally and academically, about some of the features we have planned and I only wish we had a vast team of engineers to make it all happen overnight. There are times in a field where products have covered all the bases and generating new innovation is the hard part: I think Thrive still has a long way to go before we hit that point. We have decades of research that has largely been ignored in PF and needs translating into appropriate advice and tools – this well is far from dry.

    re: stats – I think even “how many people took your advice” would fall down, in that having people switch accounts doesn’t necessarily produce lasting behavioral change – improving financial infrastructure is helpful but not sufficient. The real metric I’d like would be: has the spending and saving habits of your customer base changed since they joined your site? Mint focuses a great deal of effort on reporting – does that actually produce a quantifiable change in spending patterns?

    I can say that at Thrive, we’ve done a credible amount of internal testing on our private beta population to watch spending patterns change based on using Thrive. Not because it informs our business model but because we are interested in actually helping people. The co-founders lured me out of academia with one simple promise: you will get to apply what you know to help people who really need it. They delivered on that promise and I haven’t taken a day off in two months because of it – it matters that much to me.

    Back to numbers, I think we’ll need a critical mass of both members and months before I can feel confident in the metrics. Six months from now, this will be a clearer question and I hope that we can report based on a standard metric that anyone could use to evaluate their own PF site.

    And David, I’ve still got your case in my hopper – I’ve contacted the necessary folks but as Peter points out, it is a process and it takes time.

  14. @Matt,

    Thanks for the continued follow-up. For the record, most of my negative comments were for Mint.com. At least Thrive responded to my open case with an e-mail that included a phone number to call and surprise surprise my phone call was answered. From the customer service side you guys got it right. I never once doubted Thrive was on the case. Mint, on the other hand, I do not know.

    Regarding metrics, there are plenty standard ones out there. I am very fond of active customers. Let’s say a PFM company got 400,000 visits in September and signed up 25%. That is 100,000 new members. Let’s say only 85% could add their checking accounts. Now we have 85,000 customers. An active customer is one who comes back and logs in frequently. Do we want to say one a month, twice a month? Or, really serious members are those who sign in once a week and light users once a month? I would want to know how many continue to sign in or how many of those 85,000 customers have I turned into active repeating customers. I even think this blog’s friend, Jim Novo, considers recency a very powerful measurement. oh and these numbers are a lot less than the 400,000 we started with.

    @dmgerbino

  15. Re Matt @ thrive

    The concept of thrive is admirable, but fatally flawed.

    I think that Thrive has fallen into the some trap that the various ‘Virtual Doctor’ startups did in the tech boom.

    As a financial advisor with considerable experience, I have seen that people want advice from a real person when it comes to both their health and wealth.

    While you could argue that Chuck Schwab opened up stockbroking to the masses, the masses *thought* they new which stocks they wanted to buy (and they still had to go through a broker to do so).

    This is a non-advisory model, and completely differnt to the thrive model.

    I just can’t see how a computer generated algorythm can explain why people must pay off their credit card — and the importance of doing so, much less start an investing program. I am not convinced that this is a step int he right direction.

    At best this is a distraction that few people will take up.

  16. @ Sam: That’s certainly one viewpoint; time will tell which of us is right. But I have to say, since you’re a financial adviser, I’m surprised that you would say that a computer can’t tell people why it is important to pay off a credit card.

    Computers aren’t free-standing, but rather programmed, and in this case they are programmed by people with considerable experience in their own right. Want to convince people to invest? Show them a long time line, with the spikes and dips in the market smoothed out because of the length of the investment, and then compare that line with the same line for a savings account. That is fairly convincing to a number of people.

    Want to make it more convincing? Start basing it off their personal data, age now, age of retirement, predicted life events, etc. Contextualize it based on concrete information the person provides.

    I’m curious, as a financial adviser, do you talk to your clients about credit scores? Do you dismiss them because they are computer generated? How about insurance, where the rates are set by computers? Computers work by taking the model we create about who needs what kind of advice, and then insuring that those people get it. You’re doing this on one level by reviewing your clients information, but you’ll note that in almost every case where computer algorithms have been generated with the assistance of professionals, they have out-predicted the professionals themselves.

    And let’s not forget that we are free. Are you? It doesn’t actually matter if the computers can help people more than you can, because most people will never see you because they can’t afford it.

    I’ll be interested to hear your thoughts six months from now. Let’s revisit – Ron, want to organize a six-month rehash? *smiles*

Comments are closed.