Adaptive Sharpness

A stateful Python library for real-time, no-reference focus evaluation.

The library accepts grayscale or BGR NumPy frames and returns:

• a relative focus score;
• raw values of six focus measures;
• adaptive per-metric weights;
• a heuristic measurement-confidence estimate;
• an optional spatial map of the sharpest textured regions.

The core library does not control a camera, lens or focus motor. Capture backends, the UI and the experimental tools are optional examples built around the core API.

Optional UI on a live Panasonic GH6. Green marks the regions the map found sharpest, dark marks soft ones, and untinted areas have too little texture to judge. The panel shows the winner, the decision margin, every metric with its current weight, and the timings.

• Scope
• Installation
• Minimal integration
• Input contract
• Output fields
• State, warm-up and reset
• How the score is built
• Configuration
• Optional: spatial focus map
• Optional: capture backends and UI
• Reference hardware benchmark
• Preliminary synthetic validation
• Limitations
• Research tools
• References

Given a stream of frames, it estimates how sharp each one is relative to the recent history of that stream, by combining six focus measures with weights that adapt to the measured conditions of each frame.

instantaneous_score and filtered_score are relative to the recent history of one evaluator instance. They depend on:

• previous frames and the current normalisation window;
• the scene and the ROI;
• which metrics are enabled;
• the order frames arrive in;
• the state of the temporal filter.

Scores from different scenes, ROIs, configurations or evaluator instances are not directly comparable. Two evaluators both reporting 0.8 do not necessarily see equally sharp images. Within one instance, on one stream, a rising score does mean increasing sharpness — that is the property a focus search needs.

python -m venv .venv
source .venv/bin/activate      # Windows: .venv\Scripts\activate
pip install .

Development install (editable, with tests and figure tooling):

pip install -e ".[dev]"

PTP camera support is optional:

pip install ".[camera]"

Python 3.11+ is required: configuration loading uses the standard library's tomllib. The Haar wavelet transform is implemented directly, so PyWavelets is not needed.

Both OpenCV bounds are load-bearing. The floor is 4.10 because that is the first release with wheels built against NumPy 2 — declaring >=4.8 alongside numpy>=1.24 let a resolver pick OpenCV 4.8 with NumPy 2, which fails at import with AttributeError: _ARRAY_API not found. The ceiling is 5.0 because a 5.0 run fails a substantial part of the suite, largely around CascadeClassifier availability. Older OpenCV still works if NumPy is pinned back with it, and CI keeps one such pair green.

git clone https://github.com/RE22GDV/adaptive-sharpness.git
cd adaptive-sharpness
./install.sh                # core + camera + dev
./install.sh --core         # library only

import numpy as np
from adaptive_sharpness import SharpnessEvaluator, load_default_config

evaluator = SharpnessEvaluator(load_default_config())

for frame in frames:                     # H x W x 3 uint8 BGR
    result = evaluator.evaluate(frame)
    if not result.ready:
        continue                         # normalisers have no scale yet
    print(result.filtered_score, result.confidence)

With a region of interest, in source-frame pixel coordinates:

from adaptive_sharpness import ROI

result = evaluator.evaluate(frame, roi=ROI(x=320, y=180, width=640, height=360))

Passing the focus actuator position through, so a recorded run stays analysable:

result = evaluator.evaluate(frame, roi=roi, motor_position=servo.position())

SharpnessConfig() gives the built-in defaults without touching the filesystem; load_default_config() reads the TOML shipped inside the package; load_config(path) reads your own.

evaluate() takes a numpy.ndarray or a Frame.

The frame is never modified. Violations raise ValueError/TypeError with a message naming the fix, rather than producing plausible but wrong numbers.

config = SharpnessConfig().with_overrides(
    pipeline={"color_order": "RGB", "input_max": 4095}
)

evaluate() returns a frozen SharpnessResult.

Use instantaneous_score when you want the raw per-frame measurement, and filtered_score for a control loop that wants jitter suppressed.

It is the geometric mean of six necessary-condition factors — edge sufficiency, SNR, exposure, contrast, inter-metric concordance, motion — capped by the weakest one. A value near zero means one factor collapsed, most often "there is nothing textured in this frame to measure".

result.as_row() gives a flat dict for CSV; result.to_dict() / to_json() give a structured form. Neither drops anything that was computed.

One evaluator instance corresponds to one video stream. The normalisers, the motion reference and the temporal filter all hold history.

SharpnessEvaluator and SceneEvaluator are not thread-safe. Evaluating two streams from two threads with one instance corrupts both.

Call reset() whenever the context changes:

• a different camera or stream;
• a different resolution;
• a change of ROI size or position;
• a scene cut;
• switching between whole-frame and ROI analysis (the analysis scale differs, so the raw metric ranges differ).

if roi != previous_roi:
    evaluator.reset()

Readiness needs observed variation, not merely a number of frames. A perfectly frozen scene drives every metric to its neutral value and never becomes ready — reported honestly as informative_fraction == 0.0 and a confidence near zero, rather than as a confident 0.5.

flowchart LR
    B[frame] --> P[Preprocessor<br/>validate, grey, ROI crop, downscale]
    P --> C[ImageAnalyzer<br/>noise, edges, motion,<br/>exposure, contrast]
    P --> D[six focus measures<br/>raw values]
    D --> E[NormalizerBank<br/>robust rolling scale]
    C -->|degradations| F
    E -->|normalised| F[AdaptiveEnsemble<br/>weights + score + confidence]
    F --> G[TemporalFilter<br/>innovation-gated EMA]
    G --> H[SharpnessResult]
    C -->|stats| H

Derived from classical operators, with documented modifications — they are not used unchanged. Larger always means sharper.

Division by $\bar{I}^2$ makes the energy measures invariant to a multiplicative illumination change. Three of the deviations are necessary, not cosmetic, and each was established by measurement:

• Without the $4^{\ell-1}$ divisor the wavelet measure rises with defocus: the orthonormal Haar LL band has a DC gain of 2 per level, so deeper levels are inflated fourfold.
• Without the Hann window the image border injects broadband energy that swamps the focus signal.
• With fixed Canny thresholds the edge-width measure is non-monotone in defocus — fewer edges survive as blur grows, and the survivors are the highest-contrast ones. Measured at four window sizes, fixed thresholds failed at every one.

$$S = \sum_i w_i s_i, \qquad \sum_i w_i = 1$$

Stage 1 — reliability. Each measure carries non-negative sensitivity coefficients $\kappa_{ij}$ describing how fast it degrades under condition $j$:

$$r_i = \exp\left(-\sum_j \kappa_{ij} d_j\right)$$

The five degradation factors $d_j \in [0,1]$ are noise, edge deficiency, clipping, motion and low contrast.

These are reasoned starting points, not values fitted to data.

Stage 2 — agreement. Measures that disagree with the reliability-weighted median are suppressed by a Gaussian kernel.

Stage 3 — weights. $w_i \propto \max(p_i r_i a_i, \varepsilon)$, renormalised to sum to 1. The floor keeps every measure marginally alive so the ensemble can recover when conditions improve.

Under noise the Laplacian's weight halves (0.18 → 0.09) while Brenner's rises (0.18 → 0.28); with no edges the edge-width measure all but disappears.

The gain depends on how surprising the sample is, so smoothing does not delay a real transition:

$$z = \frac{\lvert S_t - \hat{S}_{t-1}\rvert}{\sigma_r}, \quad g = \mathrm{smoothstep}(z), \quad \alpha_t = \alpha_0 + (1-\alpha_0) g$$

$\sigma_r$ is a robust estimate of the recent frame-to-frame variation of the input, so the gate measures surprise in units of the score's own noise.

1 frame to follow a step, against 6 for a plain EMA at the same baseline gain.

Full derivation: docs/ALGORITHM.md.

The shipped defaults live inside the package (adaptive_sharpness/data/default.toml) and are reachable through default_config_path(). Unknown keys and sections are rejected at load time.

from adaptive_sharpness import SharpnessConfig, load_config, default_config_path

config = SharpnessConfig().with_overrides(
    pipeline={"analysis_width": 240, "color_order": "RGB"},
    metrics={"enabled": ("laplacian", "tenengrad", "brenner")},
    temporal={"alpha_base": 0.5, "confidence_coupling": False},
)
# or start from the shipped file:
config = load_config(default_config_path())

SceneEvaluator adds a dense per-tile map and region proposals, answering which textured region appears sharpest in the frame.

from adaptive_sharpness import SceneEvaluator, load_default_config

scene = SceneEvaluator(load_default_config())
out = scene.evaluate(frame)

if out.has_subject:
    print(out.subject.label, out.subject.center)
    print("lead over runner-up:", out.separation())

The design turns on one fact: low gradient energy has two different causes — the region is defocused, or it has no texture at all. A blank wall, the sky and a heavily defocused object can be indistinguishable. Tiles below a contrast threshold are therefore marked invalid rather than reported as blurred, and FocusMap.valid_fraction says how much of the frame was measurable.

Ranking uses the map, not the ensemble: the ensemble score is normalised against a rolling history, which is the wrong basis for comparing regions within one frame.

Per-tile measure, chosen by measurement:

lap_over_grad — a ratio of two high-pass measures — starts best on clean scenes but collapses below the decision threshold at σ ≈ 20 and goes negative at σ = 26, meaning it picks the wrong half. grad_over_var holds its margin and is the default.

Details and caveats: docs/FOCUS_UI.md.

Neither is needed to use the library.

from adaptive_sharpness.capture import open_source

The UI has every algorithm stage switchable at run time, so each one's effect is visible directly rather than only in an offline table:

python3 demo/focus_ui.py --backend synthetic --headless --frames 120
python3 demo/focus_ui.py --backend gphoto2

These numbers describe one specific setup and do not generalise.

Raspberry Pi 5 Model B Rev 1.0 (8 GB, aarch64, kernel 6.12.75, Debian bookworm, Python 3.11.2, NumPy 1.26.4, OpenCV 4.13.0), analysis width 320×180, all six measures, Panasonic Lumix GH6 preview at 640×360.

The rate is limited by the camera's own 25 fps live view, not by the Pi.

Note that 240 px is slower than 320 px: at 240 the analysis height is 135, a poor DFT length, while 320 gives 180. Pick a width whose resulting height factorises well.

These experiments test implementation behaviour on one bundled synthetic scene generator. They do not establish accuracy on real optical defocus.

python3 tools/compare_methods.py --json data/comparison.json

Twelve methods on identical data with identical frozen normalisation, across nine synthetic conditions.

On the bundled synthetic sweep, all six implementations are non-increasing as simulated defocus grows. This is asserted as a regression test.

The criterion is hill-climb distance: how far from the true peak a greedy search ends up, averaged over both starting directions. It models what a focus loop does — it never sees the whole curve.

The adaptive scheme is the only one at 1.0 in every condition. On low_texture the plain and fixed means additionally pick up 2 false peaks each, where the adaptive scheme has none.

Nine published focus measures and seven standard fusion rules, on real frames (docs/COMPARATIVE_STUDY.md):

• Fusing the measures costs resolution. Ranked by how well they separate nearby real focus positions, all eight fusion rules — ours included — fall below nine individual measures. The arithmetic mean, the median, PCA and entropy weighting all do this; it is not specific to our rule. Averaging compresses the between-position spread faster than it reduces the noise.
• Our adaptive fusion is beaten by four of our own six measures on that criterion. Feeding tenengrad alone to a search separates positions 3.6× better than feeding it our fused score.
• Two published baselines beat everything: VOL4 (299.5) and TENV (164.6) against our best measure at 120.1.
• But fusion wins the other job. Under heavy degradation the ranking inverts: on low-texture noisy frames the adaptive rule locates the peak better than every alternative, including three offline rules that see the whole sequence in advance.
• Our two most expensive measures are our two weakest. fourier and edge_width are 76% of the measure cost and rank 21st and 22nd of 23.
• And there is a plain limit: on low-texture frames that are dark and noisy, every rule including ours lands 4.5–5.5 steps from the peak of a 13-step ladder. That is not a working signal.

The two jobs rank the methods differently — resolving nearby positions favours a single gradient measure, surviving degradation favours fusion. This project optimised for the second without measuring the first.

Speed on the reference hardware: six measures 3.99 ms, adaptive fusion 1.51 ms, against a 40 ms frame interval.

Five experiments on a recorded session rather than generated scenes, none of which needs an external ground truth (docs/REAL_FRAME_STUDY.md):

• Monotonicity transfers to real image content, including under halved exposure and σ=20 noise. Every metric except edge_width scores 1.00.
• The same frame can be scored 0.83 apart on a 0–1 scale purely because of the order it arrived in — a direct measurement of the relativity the API documents. Rank correlation stays at 0.91.
• The metric ranking differs from the synthetic one. By separability of focus positions on real frames, Tenengrad leads at 120× and the Laplacian comes fourth at 52×; edge_width is last at 17×.
• The disc PSF matches real defocus to within ~15% at sub-pixel blur — but only there. A handheld recording never holds a view steady long enough to test larger blur.

• On a clean sweep every method is equivalent. Any advantage claimed there would be noise.
• The criterion is fragile. The normalisation clips to [0, 1], so the three frames nearest focus tie exactly (peak_plateau = 3). An earlier version of this criterion stopped dead on that plateau and reported 20.0 for almost everything; the numbers above come from a version that traverses flat regions. Treat the criterion as indicative, not decisive.
• The temporal filter is neutral-to-harmful here. On an offline sweep every frame is a genuine change, so smoothing only lags; adaptive_no_temporal sometimes scores slightly better. The step-response test is the right place to judge the filter.
• Adaptive weighting costs repeatability. Over five independent noise realisations of the same sweep, every fixed method selected the same peak every time (std 0.000 steps) while the adaptive scheme moved between adjacent frames (std 0.800). Mean accuracy is unchanged — this is variance, not bias, because the weights are themselves computed from noisy measurements.
• The noise adaptation never engages on the reference camera. The GH6 live-view JPEG is denoised in-camera: measured noise σ = 0.0000 across 90 frames. Every noise result here is synthetic.

Each was found by an experiment, not by reading the code, and each has a regression test:

• No claim of scientific novelty. The six measures derive from classical operators. What is proposed is the combination scheme. Nine published measures and seven standard fusion rules have now been re-implemented and run on the same real data (COMPARATIVE_STUDY.md); the result is that a published baseline beats our best individual measure, and the adaptive fusion's advantage is confined to low-texture, heavily degraded frames. That is a measured position, not a claim of novelty.
• All comparisons use simulated defocus. A disc PSF models the circle of confusion, but a real lens adds aberration, vignetting, focus breathing and subject motion. No real recorded focus sweep has been analysed.
• The κ coefficients are reasoned, not fitted.
• One scene family, one camera, one lighting setup; five noise seeds is enough to notice a difference, not to bound it.
• Sharpness is not focus. Motion blur, a dirty lens, haze and heavy compression all reduce high-frequency content with no focus error.
• Only global translation is detected as motion. A subject moving inside a static frame is not.
• The scene map finds the sharpest textured region, not the important one.

Full list with reasoning: docs/LIMITATIONS.md.

Not part of the installed library; run them from a checkout.

Known gaps in the tooling, listed so nobody mistakes them for finished work: the recording analyser reads an image directory and ignores the collector's CSV, so real motor positions and sweep direction are not used; the collector reads the motor position after the frame, so their synchronisation is approximate; and the hill-climb criterion is fragile as described above.

Protocol for recording a real sweep: docs/EXPERIMENTS.md.

python -m pytest tests/ -q

338 tests. They cover the measure contracts and monotonicity, the noise and motion estimators against known ground truth, the weighting invariants and ablation switches, the temporal gate, the input contract, capture including stale-frame dropping, configuration loading, and the UI switches.

• Brenner, J. F. et al. (1976). An automated microscope for cytologic research. J. Histochem. Cytochem. 24(1).
• Donoho, D. L. & Johnstone, I. M. (1994). Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3).
• Ferzli, R. & Karam, L. J. (2009). A no-reference objective image sharpness metric based on the notion of just noticeable blur. IEEE TIP 18(4).
• Kautsky, J. et al. (2002). A new wavelet-based measure of image focus. Pattern Recognition Letters 23(14).
• Krotkov, E. (1987). Focusing. Int. J. Computer Vision 1(3).
• Marziliano, P. et al. (2002). A no-reference perceptual blur metric. ICIP.
• Pech-Pacheco, J. L. et al. (2000). Diatom autofocusing in brightfield microscopy: a comparative study. ICPR.
• Pertuz, S., Puig, D. & Garcia, M. A. (2013). Analysis of focus measure operators for shape-from-focus. Pattern Recognition 46(5).
• Yang, G. & Nelson, B. J. (2003). Wavelet-based autofocusing and unsupervised segmentation of microscopic images. IROS.

MIT — see LICENSE.