Formulas for : CVRMSE and Net Mean Bias

In summary, the conversation discusses the use of Coefficient of Variation Root Mean Squared Error (CVRMSE) and Net Mean Bias or Abs (Mean bias) in the energy industry for conducting linear/multilinear regressions. The industry has been slow to adopt these measures, instead relying on R2 value and T-statistics. The formula for CVRMSE is similar to the ordinary coefficient of variation, with RMSE replacing standard deviation. The Mean Bias Error is also discussed, but the specific formula is unclear.
  • #1
paluee
1
0
Hi folks,

I am in the energy industry, and we do linear/multilinear regressions stats.
One of the concepts being introduced to supplement standard way of doing regressions is make use of:

(1) Coefficient of Variation Root Mean Squared Error

(2) Net Mean Bias or Abs (Mean bias)

Unfortunately, I can't find the explicit formula's.
Below is a list one goes thru to ensure a good fit/model,
some of the rules of thumb used in our industry:
*******************************************
1. R2 (Quality of fit): 0.75 or higher, with 1.00 as perfect fit.
2. CVRMSE Eng (Coefficient of Variation Root Mean Squared Error): 25 or lower for consumption meters.
3. CVRMSE Dmd (Coefficient of Variation Root Mean Squared Error): 35 or lower for demand meters.
4. Abs (Mean bias) - Must be equal to or less than 0.005.
5. Abs (T-Stats) - Positive 2.0 or higher for CDD and HDD, and greater than 2.0 or less than
-2.0 for any user variable.
6. HDD and CDD T-Stat > 0. This box must be checked to be ASHRAE and IPMVP
compliant

*******************************************
So in the context of our industry, say one take
a years worth of data, meaning 12 months of data, therefore 12 data points.
One performs a regression on this 12 data points to obtain an equation,
then one tries to interpolate these data points: a01=month1, a02=month2, etc.

Here is some output:
Data||ActualRaw||RegressInterpolatedValues
a01 51520 51345
a02 37120 37738
a03 33280 37512
a04 28800 28757
a05 17920 18641
a06 17280 17953
a07 19200 18965
a08 20800 18727
a09 27200 27243
a10 31040 31719
a11 38720 35605
a12 52480 51028

One of the methods used was to find the sum of each column and get a percent difference, and then get the differences of each month doing
=(actual-Regress)/actual * 100%
This was to get a measure of deviation of the regression to the actualdata.

But I must now use : CVRMSE and Net_Mean_Bias.

Does anybody have the explicit formula's for these 2 statistics.
Hope somebody knows these.

Here is an excerpt from our industry on this:
*************************************
users only heeded the R2 value and T-statistics, keeping them in
the range as detailed above. This was the recognized practice in the energy analysis industry. In the 2002, ASHRAE came
out with Guideline 14, which specified not an R2 value, but instead a CVRMSE and a Mean Bias Error (along with Tstatistics).
The energy analysis industry has been slow to catch on, still relying mostly on the R2 value. This was mostly
due to the fact that the software never had the CVRMSE and Mean Bias Error implemented—making it hard for energy
analysts to tune to these values

*************************************

SO hope some can tell me the formulas, or at least point me
to some source.



Paluee
 
Physics news on Phys.org
  • #2
The cvrmse is calculated the same way as the ordinary coefficient of variation, with the rmse replacing the standard deviation. that is, with

[tex]
c.v. = \frac{s}{\bar x}
[/tex]

you get

[tex]
cvrmse = \frac{RMSE}{\bar x}
[/tex]

There is some discussion of mean bias error in this document

http://www.tva.gov/sami/met/eval/ch5.PDF

although I'm not sure if it's what you are dealing with.

It would be interesting to see the statistical reasoning behind this: I've never worked with these before, but it seems rather sketchy (not a reference to your comments).
 
Last edited by a moderator:

FAQ: Formulas for : CVRMSE and Net Mean Bias

What is CVRMSE and how is it calculated?

CVRMSE stands for Coefficient of Variation of Root Mean Square Error. It is a measure of the variation in the data compared to the predicted values. It is calculated by taking the square root of the mean squared error (MSE) and dividing it by the mean of the observed values.

What does a high CVRMSE value indicate?

A high CVRMSE value indicates that there is a large amount of variability in the data compared to the predicted values. This means that the model is not accurately predicting the outcome and may need to be improved.

What is Net Mean Bias and how is it calculated?

Net Mean Bias is a measure of the difference between the average predicted value and the average observed value. It is calculated by taking the mean of the predicted values and subtracting it from the mean of the observed values.

How does Net Mean Bias differ from Mean Bias?

Net Mean Bias takes into account both positive and negative differences between the predicted and observed values, while Mean Bias only considers the overall difference without regard to direction. Net Mean Bias provides a more accurate measure of the difference between the predicted and observed values.

What is considered a good CVRMSE and Net Mean Bias value?

There is no set standard for a "good" CVRMSE or Net Mean Bias value, as it depends on the specific data and context of the study. However, in general, a lower CVRMSE and a Net Mean Bias close to 0 are desirable, as it indicates a more accurate model with less variation and bias in the predictions.

Back
Top