energykit.benchmark

ASHRAE-14 compliant forecast accuracy metrics.

energykit.benchmark — Standard metrics and benchmarks for energy AI tasks.

class energykit.benchmark.EnergyForecastBenchmark(actual, forecast, label: str = 'model')

Bases: object

Compute and display a comprehensive forecast accuracy report.

Parameters:

• actual (array-like) – Observed values.

• forecast (array-like) – Forecasted values.

• label (str) – Optional label for the model (shown in report).

Examples

>>> bench = EnergyForecastBenchmark(actual, forecast, label="LightGBM 24h")
>>> report = bench.summary()
>>> print(report)

ashrae_check() → dict

Check whether the forecast meets ASHRAE Guideline 14 thresholds.

Thresholds

Hourly: CVRMSE ≤ 30 %, NMBE ≤ ±5 % Monthly: CVRMSE ≤ 15 %, NMBE ≤ ±5 %

rtype:

dict with keys "hourly_pass" and diagnostic message.

summary() → DataFrame

Return all metrics as a one-row DataFrame.

Returns:

Columns: MAE, RMSE, MAPE (%), sMAPE (%), CVRMSE (%), R², Peak Coincidence, Load Factor Error.

Return type:

pd.DataFrame

energykit.benchmark.cvrmse(actual, forecast) → float

Coefficient of Variation of RMSE (%) — ASHRAE Guideline 14.

CVRMSE = RMSE / mean(actual) × 100

Returns:

CV(RMSE) in percent.

Return type:

float

energykit.benchmark.load_factor_error(actual, forecast) → float

Difference in load factor between forecast and actual.

Load factor = mean / peak. Higher = more uniform load, lower = peakier. A negative error means the forecast predicts a peakier load than reality.

Returns:

Δ load factor = LF(forecast) − LF(actual).

Return type:

float

energykit.benchmark.mae(actual, forecast) → float

Mean Absolute Error (same units as input).

energykit.benchmark.mape(actual, forecast, epsilon: float = 1e-08) → float

Mean Absolute Percentage Error (%).

Parameters:

• actual (array-like)

• forecast (array-like)

• epsilon (float) – Small value added to denominator to avoid division by zero.

Returns:

MAPE in percent (e.g. 5.2 means 5.2%).

Return type:

float

energykit.benchmark.peak_coincidence(actual, forecast, top_pct: float = 0.1) → float

Fraction of actual peak hours that are also forecast as peaks.

Relevant for demand charge management: missing a peak is expensive.

Parameters:

• actual (array-like)

• forecast (array-like)

• top_pct (float, default 0.10) – Top percentile to define “peak” hours (0.10 = top 10%).

Returns:

Coincidence rate ∈ [0, 1]. 1.0 = all peaks correctly identified.

Return type:

float

energykit.benchmark.r2(actual, forecast) → float

Coefficient of determination R².

Returns:

R² ∈ (−∞, 1]. 1.0 = perfect fit.

Return type:

float

energykit.benchmark.rmse(actual, forecast) → float

Root Mean Squared Error (same units as input).

energykit.benchmark.smape(actual, forecast) → float

Symmetric Mean Absolute Percentage Error (%).

Better behaved than MAPE when actuals are near zero.

Returns:

sMAPE in percent (0–200).

Return type:

float