کد مقاله کد نشریه سال انتشار مقاله انگلیسی نسخه تمام متن
394835 665856 2011 19 صفحه PDF دانلود رایگان
عنوان انگلیسی مقاله ISI
Δ-Entropy: Definition, properties and applications in system identification with quantized data
موضوعات مرتبط
مهندسی و علوم پایه مهندسی کامپیوتر هوش مصنوعی
پیش نمایش صفحه اول مقاله
Δ-Entropy: Definition, properties and applications in system identification with quantized data
چکیده انگلیسی

Recently, the minimum error entropy criterion, an information theoretic alternative to the traditional mean square error criterion, has been successfully used in the contexts of machine learning and signal processing. For system identification, however, the MEE criterion will be no longer suitable if the training data are discrete-valued, since minimizing error’s discrete entropy cannot constrain error’s dispersion. In this paper, to make the MEE criterion suitable for the discrete-valued data cases, we give a new entropy definition for the discrete random variables, i.e. the Δ-entropy, based on Riemann sums for finite size partitions. A probability weighted formula is established to calculate the average partition.This new entropy retains some important properties of the differential entropy and reduces to discrete entropy under certain conditions. Unlike discrete entropy, the Δ-entropy is sensitive to the dynamic range of the data, and can be used as a superior optimality criterion in system identification problems. Also, we present a plug-in estimate of Δ-entropy, analyze its asymptotic behavior and explore the links to the kernel based and m-spacing based estimates for differential entropy. Finally, the Δ-entropy criterion is applied in system identification with coarsely quantized input–output data to search for the optimum parameter set. Monte Carlo simulations demonstrate the performance improvement that may be achieved with the Δ-entropy criterion.

ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Information Sciences - Volume 181, Issue 7, 1 April 2011, Pages 1384–1402
نویسندگان
, , , ,