Appel à projets ANR - Programme BLANC édition 2009

PERceptual Scheme for 2D and 3D vidE(E)o coding

Context and positionning of the project:

Our digital age has seen a large deployment of video coding standards. The latest standard H.264/AVC follows a classical coding model, on the other hand a new impulse to research is brought by the emergence of new formats beyond HDTV towards formats for immersive displays allowing panoramic viewing, interactive and 3DTV (omni-directional video, free viewpoint video and stereoscopic or multi-view video).

A quantum leap in subjective quality is required, before these formats can enable a truly immersive experience for the viewer. In this competitive context, the project aims first at advancing the knowledge in perceptual modeling, in video processing and coding, and in computer vision, and second, at developing a content-based and perceptually driven representation and coding paradigm for 2D and 3D visual content.

Scientific and technical description:

Efforts are currently dedicated to the compression of multi-view sequences, and the ISO/MPEG has defined the MVC (Multi-View Coding) format which aims at capturing the redundancy between the different views. Apart from insufficient coding gain, MVC suffers from functionality limitations when virtual view-points have to be rendered at the receiver. It has also to be pointed out that visual quality is an even more crucial issue in 3D video than in 2D video. The targeted « perceptually-friendly » 2D and 3D representation and coding paradigm will build upon different models and techniques which have evolved in the last years and require further research.

To achieve the required next generation coding performance, we propose to work in the direction of a content-based and perceptually driven representation and coding paradigm using a clever combination of perceptual models, texture analysis/synthesis, waveform coding, and a rate-visual quality optimization framework. The first scientific objective of the project will thus be to define a representation of 2D and 3D visual content with the goal of best taking into account perceptual models and perceptual quality metrics rather than the ubiquitous mean square error distortion measure. The obtained framework would set the foundations for an efficient perceptual coding scheme for 2D and 3D (multi-view plus depth) visual content.

Organisation and programme:

The project involves a close collaboration between four complementary academic partners with recognized expertise in the field:

  • IRCCyN-Nantes, Image and Video-Communication team (perceptual modeling)
  • INRIA-Rennes, TEMICS team (Spatio-temporal texture analysis)
  • IETR-Rennes, IMAGE team (3D content representation and compression)
  • LTCI-TelecomParisTech, Multimedia group (2D content representation and compression)
The project is structured in 7 main tasks:
  • 1. Coordination
  • 2. Perceptual modeling for 2D and 3D video coding
  • 3. Spatio-temporal texture analysis and synthesis
  • 4. 2D content representation and compression
  • 5. 3D content representation and compression
  • 6. Integration within a common software platform
  • 7. Perceptual and subjective performance assessment

Results exploitation:

The scientific results will be disseminated as soon as they are available. The goal will be also to exploit the results through future collaborations with other partners of the "pôles de compétitivité".

The framework, integrating the most promising tools, is intended to set the foundations for technologies which could then be presented as candidates for international standardization (namely a reply to the ISO/ITU forthcoming call which should launch the H.265 standardization phase).