You are here:

Question
Hi Clyde,
I am looking to quantify the relationship between two numbers - a forecast number and the actual number. Price difference isn't very effective because when dealing with zeros, it cannot provide a measure of the difference in numbers. Another expert on this website recommended measuring the magnitude of change. I'm simply looking to measure the percentage by which the forecast analyst of was off in relationship to the actual data, as well as the percentage by which the actual number differed from the analyst's projections.

1) Is magnitude of change the best way to measure the change in numbers?

2) If not, what would you recommend?

3) For whichever method you use to quantify the relationship, can you explain it briefly and provide an example of the equation?

Thanks for the help,
Jesse

Unfortunately, measuring the validity or accuracy of predictions vs. observations is a complicated concept that may depend significantly on the context (what is being measured, how, when, how often, etc.).

There are two basic ways of measuring the "error." Assume you have a prediction P and a measurement M for some quantity. Then you can, as one person suggests, subtract the two:

(signed) absolute error = M - P

That's sometimes called "signed" absolute error of the prediction. It could be negative or positive (if M<P or M>P respectively). This shares the same units as the prediction and measurement (assuming this quantity has units).

However, if it isn't really important whether the prediction was too high or too low, only whether it was close or not, you can take the absolute value:

(unsigned) absolute error = | M - P |

By taking the absolute value, you always have a quantity that is positive. If you were just told "the unsigned absolute error is 0.1 mm" you would not know whether the prediction was too high or too low.

However, the absolute error also doesn't tell you much about how wrong it is, at least, not without comparing it to P or M. If M itself is 0.1 mm, then an absolute error of 0.1 mm is very bad!

Instead, you can divide the quantity by M to obtain the two types of relative error:

signed relative error = ( M - P ) / M

unsigned relative error = | M - P | / M

This quantity is unitless, and gives you some idea of relatively how good a prediction is.

If you are doing any sort of statistics (comparing many different pairs of M and P), you should use the unsigned absolute error. This is much more common, generally, for that reason (even when statistics are not being computed). Otherwise, you could have a set of very bad predictions (some too high, some too low) that averages out to a very small average -- but if you take the absolute values of them, you get a better idea of the average error of your predictions.

Of course, there are a million other things, depending on what kinds of science, experiments, predictions, etc. you are doing, that can influence how you account for error. But this is the basic place to start.

To summarize:

signed absolute error = M - P
unsigned absolute error = | M - P |
signed relative error = ( M - P ) / M
unsigned relative error = | M - P | / M
Questioner's Rating
 Rating(1-10) Knowledgeability = 10 Clarity of Response = 10 Politeness = 10 Comment Thanks Clyde, that seems pretty straight forward. I realize I can measure change by units rather than percentage to account for zero's and negatives in the data. I appreciate the help!

Volunteer

#### Clyde Oliver

##### Expertise

I can answer all questions up to, and including, graduate level mathematics. I am more likely to prefer questions beyond the level of calculus. I can answer any questions, from basic elementary number theory like how to prove the first three digits of powers of 2 repeat (they do, with period 100, starting at 8), all the way to advanced mathematics like proving Egorov's theorem or finding phase transitions in random networks.

##### Experience

I am a PhD educated mathematician working in research at a major university.

Organizations
AMS

Publications
Various research journals of mathematics. Various talks & presentations (some short, some long), about either interesting classical material or about research work.

Education/Credentials
BA mathematics & physics, PhD mathematics from a top 20 US school.

Awards and Honors
Various honors related to grades, various fellowships & scholarships, awards for contributions to mathematics and education at my schools, etc.

Past/Present Clients
In the past, and as my career progresses, I have worked and continue to work as an educator and mentor to students of varying age levels, skill levels, and educational levels.