Characterization of heteroscedastic measurement noise in the absence of replicates

•Blackman windowed sinc filters are used to extract noise from measurement signals.•The high-pass filters can be designed to match signal frequency characteristics.•Pooling of multiple signals leads to error models without the need for replicates.•Variance models obtained offer insights into the origins of instrumental errors.

A method is described for the characterization of measurement errors with non-uniform variance (heteroscedastic noise) in contiguous signal vectors (e.g., spectra, chromatograms) that does not require the use of replicated measurements. High-pass digital filters based on inverted Blackman windowed sinc smoothing coefficients are employed to provide point estimates of noise from measurement vectors. Filter parameters (number of points, cutoff frequency) are selected based on the amplitude spectrum of the signal in the Fourier domain. Following this, noise estimates from multiple signals are partitioned into bins based on a variable that correlates with the noise amplitude, such as measurement channel or signal intensity. The noise estimates in each bin are combined to estimate the standard deviation and, where appropriate, a functional model of the noise can be obtained to characterize instrumental errors (e.g., shot noise, proportional noise). The proposed method is demonstrated and evaluated with both simulated and experimental data sets, and results are compared with replicated measurements. Experimental data includes fluorescence spectra, ion chromatograms from liquid chromatography/mass spectrometry, and UV–vis absorbance spectra. The limitations and advantages of the new method compared to replicate analysis are presented.

Graphical abstractFigure optionsDownload full-size imageDownload as PowerPoint slide