12 Oct Make Data Driven Descisions
I’m the first to admit I’ve made plenty of decisions based on gut instinct, and scant evidence. In software development or product management though this is ill-advised, particularly where consumer services are concerned.
The fastest way to product/market fit is to get your product into as many hands as possible. Products always grow into product/market fit, they never shrink to fit. Consumers are the best test team you’ll ever have access to, and that’s why it’s important to make small iterations frequently, making use of a continuous feedback loop of analytics and consumer conversations to understand next best steps in your road map.
As I hadn’t expected many people to be using the skill, I hadn’t implemented any real analytics or reporting, but when I noticed a not insignificant uptake in it’s first few hours I quickly amended my python code to start logging some of the data. The results spoke for themselves.
Approximately 15% of users seemed to be struggling with the skill, or at least this is how it first appeared, as they were interrupting the skill using the ‘StopIntent’. Amazon doesn’t give you access to the full utterance from the user, an issue I’ve written about before, so I had to use a few work arounds to begin understanding what users were doing.
After a few more tweaks to my application, I was able to discover a few things that users were trying to use my skill for that I hadn’t thought of. These were real user requirements I hadn’t previously been able to capture!
A few more tweaks to the code (iterations) and I was able to add these user requirements and reduce the percentage of ‘StopIntent’ user interruptions pretty quickly.
Quite often standard analytics packages provide you with volume metrics and tell you where things might be going wrong, but not why. This is where creating some custom reporting will provide you with the evidence you need to fix problems. The data is often there you just need to made it accessible.
Without data I would have been flying blind just guessing what users really wanted, and so will you.