MERLIN — ModErn Restoration of Lost INformation in digital audio

MERLIN is an austrian–czech research project with co-funding from the two respective agancies, the austrian FWF and the czech GAČR.

Extended abstract: Locally degraded or even lost information is frequently encountered in signal processing. Some prime examples include corrupted time segments in damaged old audio recordings, missing data in compressed audio or lost blocks of time-frequency (TF) coefficients, also referred to as packet loss during transmission, e.g. in VoIP. In these scenarios, information is lost or unreliable in large connected regions in time, frequency or TF domains. Hence classical denoising or declicking methods that treat the whole signal or only isolated samples, respectively, cannot be applied. Automatic procedures to recover lost segments of signal data in either domain have seen increased attention in recent years and are often collectively referred to as inpainting.

However, the methods developed so far assume simplistic signal models that fail to capture the characteristic structures of speech and music data. Thus, the core objective of MERLIN is the development of novel and innovative inpainting methods through the combination of:

1. modern adapted TF representations,
2. appropriate signal models obtained from prior information and the reliable signal segments,
3. state-of-the-art signal processing methods, and
4. the consideration of perceptual indicators.

Signal completion in MERLIN is performed in several steps: Data-driven signal models will be learned, providing signal features and their temporal activations/variations. These models can predict the missing signal segment, initializing the restoration. Afterward, novel inpainting algorithms incorporating prior knowledge about the signal and its source are applied. For selected audio classes, more detailed modeling will be performed to further improve the restoration process.

The inpainting process will be driven by modern algorithms implementing e.g. (non)convex optimization schemes or image processing techniques, adapted to incorporate structural information obtained directly from the signal or meta-data. The methods conceived throughout MERLIN will be implemented in a software toolbox, freely available for research purposes in conjunction with an extensive database of real and synthetic test signals on which evaluation will be performed.