Constrained non-negative matrix factorization for score-informed piano music restoration

In this work, we propose a constrained non-negative matrix factorization method for the audio restoration of piano music using information from the score. In the first stage (instrument training), spectral patterns for the target source (piano) are learned from a dataset of isolated piano notes. The model for the piano is constrained to be harmonic because, in this way, each pattern can define a single pitch. In the second stage (noise training), spectral patterns for the undesired source (noise) are learned from the most common types of vinyl noises. To obtain a representative model for the vinyl noise, a cross-correlation-based constraint that minimizes the cross-talk between different noise components is used. In the final stage (separation), we use the trained instrument and noise models in an NMF framework to extract the clean audio signal from undesired non-stationary noise. To improve the separation results, we propose a novel score-based constraint to avoid activations of notes or combinations that are not present in the original score. The proposed approach has been evaluated and compared with commercial audio restoration softwares, obtaining competitive results.