AI Product Management / 3. Start with the business goals
Part 3 of the AI Product Management series of blogs
Successful AI projects always start with a business problem. Start small and refine the scope once you start realising the value.
Defining the business problem is the hardest part.
As a product manager/business operator, you should focus on exactly what problem you are solving.
Different steps involved in a problem-first approach are:
Business problem definition: Define the scope of the problem being solved, Identify the major stakeholders, Quantify the value, Determine the priority and the investment required for solving this problem
Data Selection: Is the data required to solve the problem readily available? Are there any security or personal identifiable information (PII) issues? Does the data cover all the use cases? Can it be easily annotated? What is the strategy for data refresh and pipeline development so that the model is and continues to be accurate? These are some of the key questions that need to be answered at this stage.
Model Building by the data science/ML team involves feature extraction, hyperparameter tuning, benchmarking against different datasets and validations and comparison of results from candidate models.
Once you have built the model with the required accuracy, then deploy it so that you can test, measure and learn. Setting up ways to A/B test the model different versions of the model or tuning different outcomes and integrating it seamlessly into the business process.
When the model is in production, it is time to actively learn and tune. Is the bias appropriately mitigated, is there accuracy against not only your model but also the business problem you were solving, and monitoring these metrics is an ongoing process.
In this post and the subsequent ones, we will be going through each of these steps in detail.
Let’s start with the first step: Defining the business problem
Defining the business problem:
As an example, say your business has lots of repeat customers and wants to increase the lifetime value (LTV) of these customers. After doing a number of user interviews with these repeat customers and digging into the data you find that, there is a correlation between your customer repeat purchases and your customer feedback score (from customer enquiries).
Many customers are not giving you a good feedback score after inquiries with your customer service and they are upset that the customer service takes a lot of time to respond to them.
When you looked at the customer queries, you realised that 70% of all customer queries are asking for more information and within the informational queries, 65% of queries are related to bulk discounts and international shipping.
The breakdown of the business problem looks something like this:
What are the jobs to be done here?
Your company have a tiered pricing model when it comes to bulk discounts and international shipping, and have a number of rate cards depending on the total sales from the customer so far. Hence, when the customer service personnel gets a query on bulk discounts and/or international shipping, they look up the total sales from the customer so far and choose a rate card accordingly.
Different steps involved in the job of customer service personnel broadly follow the below steps below:
Identify the AI Value
Now let’s identify if and where the AI can add value.
In the above steps, steps 1-5 can be automated by AI.
As a product manager, it is crucial to communicate to the stakeholders what problem we are solving, and how does AI add value to this project?
In our example, AI can completely eliminate the manual handling of informational customer queries, thus reducing the customer response time (for informational queries) to seconds. This can lead to happier customers, higher customer retention rate and higher lifetime value (LTV) of customers.
Follow on benefits are customer service reps taking up complex queries faster than before hence increasing the overall customer feedback score; fewer customer reps required overall, leading to upskilling of those staff into other roles and improving the overall employee morale.
As much as possible, break down the business problem into actionable, measurable steps with clear metrics to aim for.
Do you even need AI?
It would be remiss of me to talk about starting with a business problem without acknowledging that not all business problems require artificial intelligence (AI) or can’t be solved by AI.
For example, continuing with the theme of informational customer queries, suppose you are a SaaS business and you are getting a lot of queries on your pricing even though you have a separate page dedicated to pricing. There can be numerous reasons for it and if customer service is getting lots of queries in spite of having a dedicated pricing page, a little bit more digging into the problem may reveal that this is actually a design problem.
Fixing the design issues may eliminate 99% of these pricing queries without the need for any additional technology.
In a nutshell:
Start with a business problem
Have a specific use case (with measurable business outcomes)
Not all business problems/projects are well suited for AI (or require AI)
In the next post, we will discuss how to choose the right problem a.k.a what are some of the key considerations for choosing the right projects for AI?