Goal Question Metric (GQM) Model

I love metrics. Maybe it’s naïve to think that “numbers don’t lie” but they lie a lot less than hype, speculation and anecdotal evidence. The hard part is deciding what to measure. The “Goal Question Metric” paradigm is one tool that can help us figure that out.


The whole point of enterprise architecture is to “make things better”. But we need to make sure that we make the RIGHT things better and be able to prove that we’ve succeeded. We need to compare the before and after. And this requires metrics. The Goal Question Metric Model is one simple way of making sure that the metrics we collect (and the proof we produce) is closely tied to the business goals.

The GQM Model is a hierarchy with 3 levels: goals, questions and metrics.

In a nutshell the process involves 3 steps:

  • Identify high level (conceptual) goals. Goals identify what we want to accomplish.
  • Operationalise the goal by generating questions that will define the goal in an unambiguous and quantifiable terms. Questions help us understand how to meet our goal.
  • Specify the metrics that need to be collected to answer the questions

From experience I can say that the distinction between ‘question’ and ‘metric’ can be rather blurred. The question seems to be the metric. The point of the exercise is to go through the process – don’t worry so much about wasting time ‘making things fit’ the model.

GOAL
The first step in the process is to define the goal. A goal is conceptual not quantitative, and is made up of these components:

  • What? the product, process or resource under observation
  • What aspect?: the quality attribute of the object
  • Why?: motivation behind the goal (improve, reduce, minimise, maximise, understand, control)
  • Who?: perspective of the goal (who’s viewpoint)

Here are two examples of well defined goals.

Example 1
To improve the efficiency of customer relations staff in the call centre from the point of view of a the call centre manager.

What? Customer relations
What aspect of customer relations? Efficiency
Why? Improving
Who’s point of view? Call centre manager

Example 2
To improve the performance of the GetCustomer() web service such that service consumers are satisfied.

What? GetCustomer() web service
What aspect of the web service? Performance
Why? Improving
Who’s point of view? Service consumer

QUESTIONS
Questions help us move from the conceptual level to the operational level. Concepts such as ‘efficiency’ and ‘performance’ are abstract. Questions help people operationalise these abstract concepts.

Following on from the above examples, we could ask the following questions.

Example 1
The questions need to clarify the meaning of ‘improving efficiency’. It could mean any number of things, such as: increasing number of calls answered, reducing average call time, improving customer query resolution rates, increasing cross selling or reducing customer wait times in call queue.

Possible questions:

  • What is the maximum queue waiting time?
  • What sort of variations do we get in maximum queue times?
  • Are the peak periods? When?
  • Is the maximum queuing time increasing, decreasing or remaining the same over time?
  • What maximum queue time has the company committed to in the customer service charter?

From these questions we can see that the manager is looking to:

  • reduce customer queue times
  • reduce the maximum queue time for any one customer and not the average queue time across all customers.
  • minimise the occurrences of queue times exceeding the published wait times in the company’s customer service charter

Example 2
The questions need to clarify the meaning of ‘improving performance’. Improving performance of a web service seems pretty self-explanatory, but it still needs elaboration.

Possible questions:

  • What service level agreements (SLAs) are currently in place?
  • How often do we fail to meet SLAs?
  • What is the average response time for the web service?
  • Does the average response time vary?
  • When are the peak periods?
  • Are some customer’s details slower to retrieve than others?

From these questions we can see that:

  • We are only looking to improve performance ‘just enough’ to meet SLAs. Improving’ performance and STILL failing to meet SLAs won’t achieve the business goal. Conversely we don’t want to continually spend money to endlessly improve performance – eventually there are diminishing returns to the business.
  • We are looking to reduce the number of occurrences of SLA violations. We are not looking to improve the ‘average’ performance. In theory we could improve average performance and still have unacceptably high violation rates.

METRICS
Metrics help us answer questions. They also allow us to chart our progress towards achieving the conceptual goals. Note that some metrics can be used to answer more than one question. Collecting metrics requires time, money and effort. We want to collect the minimum number of metrics that will answer the questions we are interested in.

Example 1
To answer the questions posed above, we need to collect these metrics (and no more):

  • Total time spent in queue for every call answered.
  • Total time spent in queue for every person that hung up before their call was answered.

Example 2
To answer the questions posed above, we need to collect these metrics (and no more):

  • For each invocation of the service collect: timestamp, response time and customer ID requested.

So there you have it. The GQM method is a simple way of ensuring that whatever statistics we produce to prove that we’ve made things better, are relevant to the business’s goals.

Let’s take the Example 1 and elaborate further.

Let’s say you were to build a solution (e.g. a portal) that made it easier for customer service representatives to answer customer queries. Because the call centre operator can answer questions far quicker, it will reduce ‘average call time’. This will probably have a flow on effect to queue times – if the calls are handled quicker, more people will get through, and queue times will fall. That’s all well and good but if the business is interested in ‘reducing queue times’ don’t give them metrics to prove that your solution ‘reduced average call times’. Prove you solved their problem – don’t make them connect the dots for themselves. It sounds simple – and it is. It just requires a bit of up front thought.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: