Measuring Success: What are useful metrics for Business Analysts?
We all know the importance of metrics. Metrics tell us if we're doing our job well, and metrics can tell us if there's something we need to pay more attention to, or fix.
At amaysim, our BA team have recently been debating which metrics would be useful for us to record. The irony of the BA role is that if it's being done well, it can almost be invisible. If a delivery team is struggling with velocity, is missing requirements or not delivering the right things, the first person to check in with is the BA. Similarly, if a delivery team is running well, is vibing with their product manager and is getting value into customer's hands quickly, you know there's a great BA in there somewhere. The challenge is how to really measure this.
The agony
Often the success of a BA is hidden by the achievements of others. Engineers can be measured on number of pull requests, numbers of defects, and cycle time. Project managers are measured on time and cost of delivery. But how do we as BAs measure our contributions? How do we apply quantitative metrics to something as subjective as stakeholder satisfaction? Well, the outcome only seems subjective, in reality BAs perform very specific functions. The key is to break those functions into their component parts, and then design a rating system for each of them. This will enable us to take emotion out of the process and base ratings on imperial metrics that can be tracked, measured and presented on a dashboard.
The real deal
As any group of BAs naturally would, we approached the issue by first taking a step back. What do we actually want to achieve from our metrics? What is our goal? After much debate, we came to the conclusion that measuring the time it takes us to break down our requirements wouldn't actually tell us that it was being done well. If we see ourselves as being critical enablers to speed of delivery, then let's measure our performance there. We're attempting to measure this by trialing a survey to the engineers in our delivery teams to ask them how clear the stories were to understand, how testable the cards were, whether it felt like the right size. We agreed the survey questions could be a numeric rating, so that the responses could be measurable rather than emotive.
The next thing we'd like to measure is our success in delivering value.
When stakeholders say they want initiatives delivered faster, what they really mean is they want the value faster. It's the BA who appreciates this nuance and (with their tech lead), keeps pulling at the thread "But what are our goals? What do we really want to achieve by doing this?" and then those golden moments of "Have you considered this? What if we tried to do it this way?".
BAs need to set time and budget aside and work based on value. BAs can drive this success percentage higher by ensuring they understand what outcomes the project sponsor wanted to achieve, aligning the delivery teams expectations with the project teams expectations, and working to cut the backlog in such a way that the sponsor is getting the value sooner. So, the question becomes, how do we measure that as BAs, we're doing a good job of making sure the right things are getting into delivery? How do we make the intangible, tangible? Again, we're trialing a survey to our stakeholders; the main "customer" in our initiatives whether it's a product manager, our customer service team, or another project sponsor. How satisfied were they that the value they wanted was delivered? How clear was the communication? Were the requirements clearly documented and visualised? How would you rate the subject matter expertise?
To summarise, the key metrics for a BA are:
speed to results and outcomes (rather than speed to project completion),
the right requirements implemented (rather than number of stories written), and
satisfied users (rather than number of defects)
But what do we do with this data once we have the scores? How do we then turn the metrics into an action plan for improvement? With imperial metrics we should be able to display them on a dashboard and track trends/changes over time.
The actions
There could also be 3 possible actions from the scoring:
Direct feedback for an individual to revisit in 1:1s with their manager, and also quarterly reviews
General feedback to workshop with the team
There may be some issues/topics that management could assist with in terms of training budget etc
What metrics do you use to measure BA success in your organisation?