Performance & Telemetry calculator

Latency percentile (p95 / p99) significance calculator

Evaluate statistical significance of tail latency shifts (p90, p95, p99 percentiles) between software deployments using non-parametric quantile bootstrap confidence intervals and Mood’s median / Mann-Whitney tests.

Run this test live in StatLab npm i @statlab/core

When to use it

Use when benchmarking software performance where tail percentiles (p95, p99) matter more than average latency.

Required Inputs

  • Baseline latency run (ms array or percentiles)
  • Candidate latency run (ms array or percentiles)
  • Target percentile (p90, p95, p99)
  • Bootstrap iterations (e.g. 2,000)

Mathematical Formula

Quantile Bootstrap CI: Resample B times with replacement, compute q_p, estimate [q_α/2, q_1-α/2]

Reporting Cautions

  • Do not apply standard t-tests directly to tail percentiles due to heavy skew and non-normality.
  • Requires adequate sample size (N ≥ 1,000) to reliably estimate 99th percentile confidence bounds.

Code Snippets (Python, R, TypeScript)

Python (SciPy / Statsmodels) 
import numpy as np
def bootstrap_p95_diff(a, b, n_boot=2000):
    diffs = [np.percentile(np.random.choice(b, len(b)), 95) - np.percentile(np.random.choice(a, len(a)), 95) for _ in range(n_boot)]
    return np.percentile(diffs, [2.5, 97.5])
R Language 
library(boot)
p95_diff <- function(d, i) { quantile(d[i, "b"], .95) - quantile(d[i, "a"], .95) }
TypeScript (@statlab/core) 
import { percentileSignificance } from '@statlab/core';
const res = percentileSignificance(runA, runB, { percentile: 0.95 });

Developer Use Cases & Production Integrations

  • Automated SLA and tail-latency regression verification in VoxelAssurance release readiness sprints.
  • Continuous telemetry distribution shift detection in VoxelPulse infrastructure monitoring.

VoxelPulse Telemetry Control Plane

Continuous A/B testing, statistical telemetry analysis, and automated feedback loops for live backend services.

Explore VoxelPulse →

VoxelAssurance Release Readiness

Fixed-price AI release-readiness sprints, automated regression testing, and reliability statistical benchmarks.

Explore VoxelAssurance →

Related Statistical Test Calculators

Explore static calculators across the StatLab inference engine suite.