AI & Machine Learning calculator

Brier score and probability calibration calculator

Calculate Brier score, Reliability, Resolution, and Uncertainty components to measure the accuracy of probabilistic forecasts and LLM confidence calibration.

Run this test live in StatLab npm i @statlab/core

When to use it

Use to evaluate the accuracy and calibration of probabilistic predictions.

Required Inputs

  • Predicted probabilities (0.0 to 1.0)
  • Actual binary outcomes (0 or 1)

Mathematical Formula

BS = (1/N) Σ (f_i - o_i)²

Reporting Cautions

  • Brier score ranges from 0.0 (perfect prediction) to 1.0 (worst prediction).
  • Decompose into Reliability and Resolution for deeper diagnostic insight.

Code Snippets (Python, R, TypeScript)

Python (SciPy / Statsmodels) 
from sklearn.metrics import brier_score_loss
bs = brier_score_loss(y_true, y_probs)
print(f"Brier Score = {bs:.4f}")
R Language 
library(scoringUtils)
brier_score(y_true, y_probs)
TypeScript (@statlab/core) 
import { brierScore } from '@statlab/core';
const bs = brierScore(yProbs, yTrue);

Developer Use Cases & Production Integrations

  • Evaluating confidence score calibration of LLM responses in VoxelAssurance quality audits.
  • Measuring risk prediction accuracy in production classifier models.

VoxelPulse Telemetry Control Plane

Continuous A/B testing, statistical telemetry analysis, and automated feedback loops for live backend services.

Explore VoxelPulse →

VoxelAssurance Release Readiness

Fixed-price AI release-readiness sprints, automated regression testing, and reliability statistical benchmarks.

Explore VoxelAssurance →

Related Statistical Test Calculators

Explore static calculators across the StatLab inference engine suite.