کد مقاله | کد نشریه | سال انتشار | مقاله انگلیسی | نسخه تمام متن |
---|---|---|---|---|
4973726 | 1451681 | 2017 | 20 صفحه PDF | دانلود رایگان |
عنوان انگلیسی مقاله ISI
Acoustic model training based on node-wise weight boundary model for fast and small-footprint deep neural networks
ترجمه فارسی عنوان
آموزش مدل آکوستیک بر اساس مدل مرزی گرادیان گره ای برای شبکه های عصبی عمیق و سریع با پایه کوچک
دانلود مقاله + سفارش ترجمه
دانلود مقاله ISI انگلیسی
رایگان برای ایرانیان
کلمات کلیدی
موضوعات مرتبط
مهندسی و علوم پایه
مهندسی کامپیوتر
پردازش سیگنال
چکیده انگلیسی
Our goal for this study is to enable the development of discrete deep neural networks (NNs), some parameters of which are discretized, as small-footprint and fast NNs for acoustic models. Three essential requirements should be met for achieving this goal; 1) the reduction in discretization errors, 2) implementation for fast processing and 3) node-size reduction of DNNs. We propose a weight-parameter model and its training algorithm for 1), an implementation scheme using a look-up table on general-purpose CPUs for 2), and a layer-biased node-pruning method for 3). The first proposed method can set proper boundaries of discretization at each NN node, resulting in reduction in discretization errors. The second method can reduce the memory usage of NNs within the cache size of the CPU by encoding the parameters of NNs. The last method can reduce the network size of the quantized DNNs by measuring the activity of each node at each layer and pruning them with a layer-dependent score. Experiments with 2-bit discrete NNs showed that our training algorithm maintained almost the same word accuracy as with 8-bit discrete NNs. We achieved a 95% reduction of memory usage and a 74% increase in speed of an NN's forward calculation.
ناشر
Database: Elsevier - ScienceDirect (ساینس دایرکت)
Journal: Computer Speech & Language - Volume 46, November 2017, Pages 461-480
Journal: Computer Speech & Language - Volume 46, November 2017, Pages 461-480
نویسندگان
Ryu Takeda, Kazuhiro Nakadai, Kazunori Komatani,