When I talk to people about mobile trends one question that always seems to come up is, "Our customers are older and most don't have so smartphones so how important is mobile really to us?"
While there are many response to this question, the most important is that won't be true for long.
Sure smartphone penetration among 45-64 year-olds won't ever be as high more technologically savvy younger demos, but mom -- or grandma -- will likely have one soon.
According to this data from eMarketer, smartphone penetration in the 45-64 demo will pass 50% in next couple years. Like most mobile trends, I'd bet on it being sooner than later. Do you want to wait until then and have to play catchup?
App reviews provide valuable customer feedback, influence people to download, and even impact rankings in app stores, like Google Play. However, most people aren’t likely to leave a review. If they do there’s a good chance they’re pissed about a bug or missing feature.
So how do you get more reviews and higher ratings for your app?
My team and I tackled this problem for our Kodak Gallery Android app. Here’s how we went about it. What we learned will surprise you.
Our motivation for tackling this was two-pronged:
1. We had low ratings in the Android Market / Google Play. We had quietly released an alpha version of the app for testing purposes. It crashed a lot and got low ratings from people that stumbled across it in the market. When we released v1.0 the reviews were good; however, ratings don’t reset in with each upgrade in Google Play so we were stuck with a low “lifetime rating” (~3.0 out of 5.0).
2. Our Google Play ranking wasn’t as high as we thought it should be. We were stuck around #50 in the free photo category rankings despite some healthy download numbers. Like with Google search results, Google Play rankings are a black box but supposedly the number and quality of reviews is part of the ranking algorithm.
We felt if we had more reviews and better ratings our app would be more attractive to prospective users and do better in the market rankings.
The initial plan was to simply add a "rate this app" prompt in the app. This is a common tactic employed by lots of apps, but we wanted to see if it actually worked.
However, we were concerned we would get more reviews but they might not be very good. Ron, our newly crowned ratings guru, had the idea to test another version where we only prompted people to rate the app in teh market if we knew really the app.
Here's what our 3 test cells looked like:
1. Default. No user prompts to rate the app
2. Market Prompt Only. Users see a prompt asking them to rate the app in the Google Play market.
3. Local Prompt. Before asking users to go the market to rate the app we present a local prompt to rate the app. If the user selects "Love it!" or "Like it" they get the market prompt. People selecting "Not crazy about it", "Hate it", or Cancel don't get the market prompt.
Local Prompt Market Prompt
We rotated the test cells every day. Day 1 was default, Day 2 local prompt, Day 3 market prompt, and then repeat. We cycled this for 3 weeks to normalize for any influence the day of the week might have on people's responses.
Once people received the local or market prompts they did not see them again. "Later" on the market prompt in this case meant a future date that we hadn't determined. Like an afterschool special, no meant no.
We measure the results based on two success metrics:
1. Number of reviews per thousand visitors (RPMV). This eliminated the the impact the number of visitors had on reviews since more visitors = more prompts.
2. Average rating. Google Play ratings are based a 1-5 star rating.
Here are the cumulative results for the 3-week test that ran in April.
Number of Reviews
Winner: Local Prompt Screener
This was a landslide. The local prompt had 56% more reviews than the market only prompt while the average ratings were the same at 4.5. In each test cycle the local prompt had 17%-90% more reviews per visitor than the market only prompt so the results weren't affected by a big day.
An interesting sidenote is that the 14 days we ran the local and market prompt accounted for just under 30% of total reviews and increased the average overall rating from around 3.8 to 4.3.
Rankings Impact
The ranking in the free photo category jumped from the low 50's to the low to mid 40's.
If you're like us, these results are somewhat surprising.
We predicted that the Market Only prompt would drive the most reviews while the Local Screener would have fewer reviews but a higher average rating because we were filtering for people that really liked the app and making them go through more steps, which usually leads to lower conversion.
As you can see above, the local prompt had significantly more reviews. Why could this be?
1. People were invested. The local prompt was a super easy way to participate and once people did they had some stake in the game. When we then prompted them to rate the app in the market we were asking them to do the same thing they just did, but in a different place.
2. People really like the app. It turns out people really liked the app and we weren't filtering as many people as we expected. While 52% of people chose Cance", 94% of those that did rate the app locally said they "Love it" or "Like it".
The ~10 spot jump in the app rankings indicates the better reviews had an impact on the rankings but there could be other factors as well, like photo sharing seasonality. While it's hard to say if the increase reviews and higher rating affected the ranking algorithm we did move up and that was one of our goals so that makes us feel warm and fuzzy.
Instagram's Android app has only been out 2 days but it's already the #1 search result for "google play" -- higher than the market's homepage.
If you have an Android or iPhone I'm sure you've come across splash screens promoting a site's app when you visit using a browser on your smartphone or tablet. Creative treatments vary from site to site. So which one is best?
Well, I'm not sure what the best is but I did find out what's better by running some tests for the Kodak Gallery mobile app. Here's what I learned.
Our primary objective was to get more people to tap to get the app. A secondary objective was to minimize the exit rate.
Because we were looking for a significant delta, wanted to limit development hours, and had consistent weekly traffic, we ran a sequential instead of an A/B or A/B/C test. Each test candidate ran for one week from Thursday through Wednesday in February.
Our existing creative showed picture of a photo on a phone, some feature messaging, and a primary "Get it Now!" button with a "no thanks" text link.
It had been running for a few months but we didn't have any analytics on the page so we added some Omniture tags and ran this treatment as the first candidate.
"LinkedIn Style"
The 2nd candidate was inspired by the treatment LinkedIn uses. It's prettier, doesn't have any copy other than the name and CTAs, and has larger buttons with "not thanks" getting more weighting than the default. I figured LinkedIn must know what they're doing so we gave it a try.
"OS Style"
The 3rd test was based on treatments other sites use that look like a notification from your phones OS. It's pretty straightforward but we thought maybe people were on to something.
The LinkedIn Style performed slightly better than the default treatment but it wasn't the major change we were looking for.
However, when we ran the OS style we saw significant change in both directions depending on the platform. The OS style did 50-75% better in app downloads and dropped the exit rate to essentially zero on the iphone while Android download "taps" dropped by 60%; although, the exit rate was much better.
* If you looked closely you noticed that "Get the App" + "No thanks" for the iPhone OS style is greather than 100. Unfortunately Omniture has some data anomolies and this is one of them. The exit rate is a calculated metric (total visitors - get the app visitors - no thanks visitors = exits). Instead of showing a negative exit rate I just made it zero.
The OS Style was the clear winner for the iPhone but did horrible on Android.
Why did the OS Style do well on the iPhone?
My hyptohesis is the treatment appears more like an interaction with the iPhone than marketing content. iPhone users are used to seeing the prompt (do you want push notifications?, do you want to enable location services?, etc.) and won't bail. There also might be more trust. It also helps that the desired CTA ("OK" is pretty benign.
Why did the OS Style do so poorly on Android?
There are several reasons why it could have done poorly.
The next step is to test diffrent a treatment using "OK" for the CTA and putting it on the right side to confirm or refute the 2nd and 3rd bullet points.
If you've done any testing on app splash screens I'd love to know if you've seen the similar results or if you have something else that works better.
