A comparison of least squares and maximum likelihood methods using sine fitting in ADC testing

•Least square (LS) and maximum likelihood (ML) sinewave fit ADC test were compared.•We evaluated error of SINAD of ADC under the test calculated from LS and ML fit.•ML fit was performed by gradient method in Matlab and differential evolution in LabVIEW.•ML fit results in more accurate SINAD values for increasing INL of ADC.

ADC test methods require the best possible reconstruction of the input signal of the ADC under test from the acquired, therefore erroneous, ADC output data. The commonly used least squares (LS) fit and the recently introduced maximum likelihood (ML) estimation are competing methods. This paper presents a simulation-based comparative study of these estimation methods with the goal to investigate the behavior of both methods and to determine their limits. Two alternative algorithms for the calculation of the maximum likelihood fit are considered (gradient-based minimization and differential evolution). The main finding is that while for low-INL (linear) ADCs the two methods (LS and ML) give similar results, for practical (almost always nonlinear) ADCs ML is definitely better.