You are here:

Probability & Statistics/metrics

Question
I work in a call center and I was hoping you could help me understand one of the metrics we are measured by as I feel as if one of the formulas is skewed. Our customers complete a survey and they are asked if they would refer the company to a friend on a scale from 1-10.  We have Promoters (which are scores of 9's and 10's) and we have Detractors (which include any score that we are given between a 1 and a 6); however, scores of 7's and 8's are considered neutral scores and not considered in the calculation of the numerator.  The raw RAF (Refer A Friend) score is calculated by subtracting the detractors from the promoters which equals the Net Promoter. Then, the final score is determined by the Net Promoter/# of surveys to get a RAF%

For example, say I received 12 surveys in a month with the following RAF scores:  10, 5, 7, 10, 10, 10, 9, 7, 10, 10, 10, 10.
Promoters (9s & 10s) = 9 surveys
Detractors (1s - 6s) = 1 survey
Net Promoter = 9 - 1 = 8
RAF = net pro/# of surveys
8/12 = 66.7% for RAF score

I do not understand how they can use the total # of surveys to calculate this RAF number when 2 of the survey scores (the two 7 neutral scores) were not used as part of the numerator calculation.  I know that I am so not on your level of mathematics genius...but, it just does not seem fair.  It seems that the formula should only use the number of surveys in the numerator calculation instead of all of them.  For example, 8/10=80% RAF.  Please help me to understand this better.  Thank you in advance for your assistance.  Rachel :D

What you are looking at is actually a sort of weighted average.

There is some function that assigns values to scores. If you were doing a normal average, it would be the identity function f(x)=x so that:

f(1)=1, f(2)=2, f(3)=3, etc.

Then you add up all the scores according to this function and divide by the total number of scores to get your average:

10+5+7+10+10+10+9+7+10+10+10+10=108

Divide by 12: 108/12 = 9.

However, someone has decided that the scores should be weighted differently, since in some sense, a 4 and a 1 are equally bad -- they are low enough to be considered bad.

So you have detractors:

f(1) = f(2) = f(3) = f(4) = f(4) = f(5) = f(6) = -1

neutral:

f(7) = f(8) = 0

promoters:

f(9) = f(10) = 1

Now your score is simply the average of your ratings, based on this function f.

f(10) + f(5) + f(7) + ... + f(10) = 1 + (-1) + 0 + ... + 1 = 8

Then divide to get 8/12 = 0.66%

Now, this is not a weighted average in the sense that the weights change the denominator -- so this average changes the scale (from 1-10 to -1 to 1). However, in any sort of averaging scheme, it generally does not make sense to ignore data points with weight 0, even if they do not contribute to the summation. You should still be counting them in the denominator.

Unfortunately that does mean if you have a high rating and then get a 7 or 8, your average will decrease slightly, but that is normal. Even though they are "neutral" they may bring down a good average.
Questioner's Rating
 Rating(1-10) Knowledgeability = 10 Clarity of Response = 10 Politeness = 10 Comment Thank you for your quick response and clear explanation!

Probability & Statistics

Volunteer

Clyde Oliver

Expertise

I can answer all questions up to, and including, graduate level mathematics. I do not have expertise in statistics (I can answer questions about the mathematical foundations of statistics). I am very much proficient in probability. I am not inclined to answer questions that appear to be homework, nor questions that are not meaningful or advanced in any way.

Experience

I am a PhD educated mathematician working in research at a major university.

Organizations
AMS

Publications
Various research journals of mathematics. Various talks & presentations (some short, some long), about either interesting classical material or about research work.

Education/Credentials
BA mathematics & physics, PhD mathematics from a top 20 US school.

Awards and Honors
Various honors related to grades, various fellowships & scholarships, awards for contributions to mathematics and education at my schools, etc.

Past/Present Clients
In the past, and as my career progresses, I have worked and continue to work as an educator and mentor to students of varying age levels, skill levels, and educational levels.