Concept Testing
How to quickly explore user context and validate assumptions to maximize product success
In a fast-paced world of product development...
If you are a product manager/owner/designer/user researcher, chances are that you may have found yourself in a situation like this once in your career:
Higher management: We have an idea about a product and we want to make it based on different requests from our users. Since our users need it now, let’s start building to launch as soon as possible.
When the pandemic upended all normal operations, global organizations and businesses scrambled to figure out business continuity measures. As part of those measures, my organization handed over an important and time-sensitive product. There was a prototype but not on par with the business’s idea of user requirements and we needed to improve and add those features they thought were necessary.
But there is a tiny problem- information about any market competitors or user research that helps me and my team understand the problem and build the product/feature effectively and efficiently is missing; either there was no time or it was considered unnecessary.
This was not the ideal product development scenario but we had to make it work.
But I have a lot of questions before I start…
I realized that there are some aspects I needed to question and some assumptions I needed to challenge/ validate before I went ahead with the development process and improve the existing prototype.
Since time was limited, I brainstormed with my colleague around the most important information that we were missing. This is what we listed:
What our users are doing to fulfill their needs in the absence of this product/feature? What will make them shift to ours? What is that threshold they will need to cross and why will they?
What is it that they are missing a.k.a what are their unmet needs?
How do we know that this product/feature is going to be successful with the majority of our users without knowing their context and their interactions?
What other tools, devices, and processes surround and interact with our users in absence of this product/feature or with it once they adopt it?
Finding answers and fast!
Time is crucial but so are the answers to those questions to ensure that the product development team is well informed about the market and sensitive to user needs.
It was time to don my user researcher hat. I addressed this situation by adapting and creating a quick research plan that considered the most important information needs for the project and since we also had a prototype- to do a quick user testing exercise. I also pitched it to the stakeholders for alignment.
I tried not to be very ambitious- I made it lean, to help me analyze and synthesize fast- two weeks turnaround is the shortest I have done.
I was collecting only the data that we needed.
Recruiting research participants in a short time span can be challenging but one creates methods to conquer as I move from one project to another.
After a number of projects, I have a user pool to dip into without causing survey/interview fatigue by repetitively engaging the same end users.
I have a template for the recruiting message that I tweak and send out quick email invitations to a number of users; keeping in mind a 10–30% conversion to these emails.
Analyzing and coding as I go about our interviews using pre-designed templates helps me derive bite-size actionable insights quickly.
I also do a short post-interview summary (or a brain dump) with the most important and interesting data that I have gathered in that session. One could create a tabular or card format to capture that summary.
Capturing the answers to the most important research questions in a spreadsheet and quantitative analysis gives us the significance of certain insight.
How many users feel the same about this task?
Users who are generous in their answers(sometimes it’s a challenge to steer back on the right track) help us understand other expectations and environmental implications. We capture those through high-level summaries and relevant recommendations.
Users also mentioned the different platforms they used to carry out their tasks. We created a benchmarking document where we compared these different platforms and their features- hence, conducting competitor analysis for the said product we were building.
Quick and dirty research or not?
User research is a very serious and important aspect of product discovery and development, it takes time and a lot of effort to set up comprehensive user research projects with plans and documentation but sometimes you are not afforded that luxury.
A large number of researchers do not approve of quick and dirty research but sometimes it helps to investigate research questions that are most crucial and get answers quickly so that we don’t build something and repent later.
We discovered nuances even in short interviews about different contexts in which our users operate and how their needs change and what may confuse or disappoint our users in a product concept. We synthesized these insights and pared down our product features based on time and importance, we reprioritized our product backlog and also our releases- Thanks to MoSCoW!
This exercise and the insights also helped convince the stakeholders and management that not all requirements gathered initially were actually the top necessity of our users.
I use my learning from these short exercises from different projects to build my user insights repository that becomes more enriched because of different aspects of the ecosystem we research in our short investigations. Sometimes, a few of them can trigger a deeper and longitudinal investigation to understand user behavior.