EU - MUSCLE - Multimedia Understanding through Semantics, Computation and Learning

MUSCLE aims at creating and supporting a pan-European Network of Excellence to foster close collaboration between research groups in multimedia datamining on the one hand and machine learning on the other in order to make breakthrough progress towards the following objectives: * Moving from modelling to learning: Harnessing the full potential of machine learning and cross-modal interaction for the (semi-)automatic generation of robust meta-data with high semantic value for multimedia documents. In particular, MUSCLE researchers will develop software tools and research strategies that enable users to move away from labor-intensive case-by-case modelling of individual applications, and allow them to take full advantage of generic adaptive and self-learning solutions that need minimal supervision. * Improving interoperability through understanding: Improving interoperability and exchangeability of heterogeneous and distributed (meta)data by enabling data descriptions at high semantic levels (e.g. ontologies, XML schemata) and adding inference schemes that can reason about them at the appropriate levels. To this end MUSCLE researchers will contribute to relevant international standards and protocols. * Creation of expressive and adaptive interfaces: In the same vein, improve the human-machine interface by exploring how machine learning can invigorate the creation of expressive, context-aware, and human-centered interfaces that will be able to effectively assist users in the exploration of complex and rich multimedia databases. With regard to these topics, MUSCLE research will contribute to viability studies and proof-of-principle demonstrators. * To stimulate cohesion, the NoE will set itself two grand challenges. These are ambitious research projects that involve the whole spectrum of expertise represented within the consortium. As such they also require the collaboration of a large number of groups and therefore act as focal points for the consortium: o Grand Challenge #1: Natural high-level interaction with multimedia databases In this vision it is possible to query a multimedia database at a high semantic level. This is an extremely challenging problem and will involve a wide range of techniques: natural language processing, interfacing technology, learning and inferencing, merging of different modalities, federation of complex meta-data, appropriate representation and interfaces, etc. o Grand Challenge #2: Detecting and interpreting humans and human behaviour in videos Many important applications of multimedia data mining revolve around the detection and interpretation of human behaviour. Applications are legion: surveillance and intrusion detection, face recognition and registration of emotion or affect, automatic analysis of sports videos and movies, etc. Again, success will depend heavily on the integration and interpretation of various modalities such as vision, audio and speech.