کد مقاله | کد نشریه | سال انتشار | مقاله انگلیسی | نسخه تمام متن |
---|---|---|---|---|
6865612 | 679059 | 2015 | 39 صفحه PDF | دانلود رایگان |
عنوان انگلیسی مقاله ISI
Neural Networks for Sequential Data: a Preâtraining Approach based on Hidden Markov Models
دانلود مقاله + سفارش ترجمه
دانلود مقاله ISI انگلیسی
رایگان برای ایرانیان
کلمات کلیدی
موضوعات مرتبط
مهندسی و علوم پایه
مهندسی کامپیوتر
هوش مصنوعی
پیش نمایش صفحه اول مقاله
![عکس صفحه اول مقاله: Neural Networks for Sequential Data: a Preâtraining Approach based on Hidden Markov Models Neural Networks for Sequential Data: a Preâtraining Approach based on Hidden Markov Models](/preview/png/6865612.png)
چکیده انگلیسی
In the last few years, research highlighted the critical role of unsupervised pre-training strategies to improve the performance of artificial neural networks. However, the scope of existing pre-training methods is limited to static data, whereas many learning tasks require to deal with temporal information. We propose a novel approach to pre-training sequential neural networks that exploits a simpler, first-order Hidden Markov Model to generate an approximate distribution of the original dataset. The learned distribution is used to generate a smoothed dataset that is used for pre-training. In this way, it is possible to drive the connection weights in a better region of the parameter space, where subsequent fine-tuning on the original dataset can be more effective. This novel pre-training approach is model-independent and can be readily applied to different network architectures. The benefits of the proposed method, both in terms of accuracy and training times, are demonstrated on a prediction task using four datasets of polyphonic music. The flexibility of the proposed strategy is shown by applying it to two different recurrent neural network architectures, and we also empirically investigate the impact of different hyperparameters on the performance of the proposed pre-training strategy.
ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Neurocomputing - Volume 169, 2 December 2015, Pages 323-333
Journal: Neurocomputing - Volume 169, 2 December 2015, Pages 323-333
نویسندگان
Luca Pasa, Alberto Testolin, Alessandro Sperduti,