About the project

XTREME – “Mixed Reality Environment for Immersive Experience of Art and Culture” is an EU Horizon 2020 project that started in January 2024 and will finish in December 2026. XTREME will explore and provide a mixed reality (MR) solution to experience different forms of art. The project is in close collaboration with 14 different partners and together we will explore different alternatives to the traditional way of accessing music and art experiences.

View the project partners »

Human-centered mixed reality solution

Engaging with art and music has a positive effect on health, but limitations such as venue location, personal constraints, scheduling conflicts, and costs often hinder access to these experiences. With this in mind, in the XTREME project we aim to develop a human-centred mixed reality (MR) solution that transports concerts and performances to remote locations, removing physical constraints.

Shared virtual experience

This MR technology integrates real and virtual content, where users can share the experience virtually with selected social circles. Eliminating travel requirements, XTREME provides a greener alternative. This groundbreaking project provides virtual access to immersive art and music experiences breaking down traditional barriers.

Denmark

IT University of Copenhagen

The IT University of Copenhagen (ITU) was established in 1999 and is Denmark’s leading university focusing on IT research and education. The University delivers state-of-the-art teaching and research within computer science, business IT and digital design. ITU collaborates closely with industry, the public sector and international researchers and is characterized by a strong entrepreneurial spirit among students as well as researchers.

In XTREME, ITU is the coordinator partner and hence responsible for the overall management of the project, partner administration and reporting, together with financial, quality and risk management, and identification of user requirements. Besides that, ITU is responsible for research related to multiview scene synthesis, sound reconstruction and visualization, music content understanding, creation of computational synaesthesia, human computer interaction and experience design, as well as the intersection between different forms of art and new technologies.

Project coordinator and Principal Investigator: Prof. Sami S. Brandt

ITU.DK

Ireland

University of Limerick

The University of Limerick (ULIM) is a public research university institution in Limerick, Ireland. The University is located in the Wild Atlantic Gateway city of Limerick on the west coast of Ireland. ULIM is an independent research-led university offering undergraduate and postgraduate programmes across Arts, Humanities and Social Sciences; Education and Health Sciences, Science and Engineering and the Kemmy Business School.

In XTREME, the University of Limerick provides a cross-faculty Living Lab based in the Kemmy Business School consisting of researchers focusing on performance and mixed reality, AI/data and XR governance, law and ethics as well as assessing future commercialisation strategies. The Living Lab research is responsible for the participatory technology assessment, social evaluation and impact assessment, and building a Trustworthy-XR a framework to enable further positive societal impacts and opportunities. A key part of the Living Lab strategy lies in how it fuses research in the creative arts with technological governance and ethics to develop more humancentric and ethical innovation. The University of Limerick’s world renowned Irish World Academy of Music and Dance (IWAMD) and its researchers are a key feature of the Living Lab collaboration and research. The Irish Chamber Orchestra (ICO) (Ireland’s leading international chamber orchestra, funded by An Chomhairle Ealaíon (the Irish Arts Council) is based in the University of Limerick and will work with the Living Lab researchers to create innovative mixed reality performances for the XTREME project. A particular focus of the Living Lab concerns research on the potential social opportunities of mixed reality for music therapy (a use case concerns people with alzhiemers and dementia), and also social facing applications such as community music.

Principal Investigator: Assoc. Prof. Martin Cunneen

UL.IEirishworldacademy.ieirishchamberorchestra.com

Netherlands

University of Twente

The University of Twente (UT) was established in 1961 and is a public technical university located in Enschede, Netherlands. At the University of Twente, we are pioneers in fusing technology, science and engineering with social sciences to impact the world around us. The driving force for the students, scientists and educators is a deep sense of connection with people who share a curious, entrepreneurial spirit.

In XTREME, UT is responsible for the assessment of potential XR sets, audio and visual equipment, scene understanding and 3D Audio–Visual reconstruction of the scene.

Principal Investigator: Assist. Prof. Ville Lehtola

utwente.nl

Denmark

Marionette

Marionette AI-driven motion capture software gives creators the possibility of recording motion capture anywhere without the need for motion capture suits. After 3D animation is extracted from video, artist tools are available for making motion capture data ready to use in animation pipelines. The company is based in Copenhagen, Denmark.

In XTREME, Marionette will support motion capture with the AI-driven camera-based motion capture software.

Principal Investigator: Jennifer Anne Ipsen

marionettexr.com

Italy

Italian Institute of Technology

The Italian Institute of Technology (IIT) is a scientific research centre based in Genoa, Italy. Its main goal is the advancement of science, in Italy and worldwide, promoting projects and discoveries oriented to human-centric applications and technology.

In XTREME, IIT is responsible for 3D audio–visual reconstruction of the scene: mapping multichannel sound to an acoustic image stream, multimodal audio-visual detection and Integration of 2D audio-visual reconstructed scene with 3D stream.

Principal Investigator: Dr. Pietro Morerio

iit.it

Norway

MUNCH

MUNCH Museum is an art museum dedicated to the life and works of the Norwegian artist Edvard Munch. The museum was opened in 1963 and in 2021 it was moved to the new building at Bjørvika, Oslo. The museum building has been specifically created for unique art experiences and is a highly prestigious project for the City of Oslo. It is an impressive 13 floors modern building contracting: 11 galleries, over 200 works of Munch on permanent display, 4 500 square metres of gallery space and concert halls, cinema, and venue hire.

In XTREME, MUNCH is a space for XTREME audio-visual performance use case developed in a collaboration with an externally commissioned artist.

Principal Investigator: Birgitte Aga

munchmuseet.no

Germany

Leibniz University Hannover

Leibniz University Hannover (LUH), also known as the University of Hannover, is a public research university located in Hanover, Germany. It was founded in 1831. The spirit of Leibniz – as one of the nine leading universities of technology in Germany, Leibniz University Hannover is aware of its responsibility to contribute to identifying sustainable, peaceful, and responsible solutions to the key issues of tomorrow. The LUH’s expertise ranges from engineering and natural sciences to architecture and environmental planning, from law and economics to social sciences and humanities.

In XTREME, LUH is leading the work on developing the core technology, being also responsible for the human reconstruction and virtual presence: 3D Reconstruction of dynamically moving people, real-time motion capture of a person in a mobile environment, fine grain human pose estimation and audience motions and scene embedding.

Principal Investigator: Prof. Bodo Rosenhahn

uni-hannover.de

Denmark

KHORA

Khora is a leading Scandinavian Virtual Reality (VR) and Augmented Reality (AR) production studio, creating cutting edge content within multiple application areas.

Khora is exploring the value potential of VR and AR through meaningful collaborations and partnerships, working within multiple industries, with a deep understanding of the technologies and of how and why businesses and institutions should be working within these emerging mediums.

In XTREME, Khora is engaged in the user experience design and scenarios of the XTREME use cases, being also responsible for content design, production and adaptation for some of the productions.

Principal Investigator: Matias Katajavaara Seidler

khora.com

Netherlands

4DSOUND

4DSOUND is an Amsterdam-based studio focusing on spatial sound as a creative medium. Since 2007, 4DSOUND has been at the forefront of some of the world’s most creatively challenging and technically complex projects using spatial audio in the context of music, architecture, interactive installations, fashion, performing arts and research.

In XTREME, 4DS is responsible for the integration of a high performance and high definition spatial audio engine, combined with an intuitive interface for creation and cross media interactivity. A 4DS system will also be a part of the Audio-Visual Performance developed in MUNCH.

Principal Investigator: Poul Holleman

4dsound.net

Finland

AALTO University

Aalto University is a public research university located in Espoo, Finland. It was established in 2010 as a merger of three major Finnish universities: the Helsinki University of Technology, the Helsinki School of Economics and the University of Art and Design Helsinki. The close collaboration between the scientific, business and arts communities is intended to foster multi-disciplinary education and research.

In XTREME, AALTO is responsible for leading research on measuring audience experience in physical reality and in social XR.

Principal Investigator: Prof. Mikko Sams

aalto.fi

Greece

Bolt Virtual

Bolt Virtual is a market leader in the emerging technologies of eXtended reality, based in Athens, Greece. Since 2017 Bolt has been developing its own Virtual Reality, Augmented Reality and Mixed Reality applications for gaming, corporate marketing, promotion of cultural heritage, advertising tourism products, marketing tourism destinations, as well as for educational purposes.

In XTREME, BV is responsible for the definition of overall system architecture and development of application programming interface, content design, production, and adaptation of the audio-visual performance, as well as for the XTREME exploitation activities.

Principal Investigator: Yannis Parcharidis

boltvirtual.gr

Denmark

Immersive Stories

Immersive Stories is an award-winning production company with a focus on engaging storytelling and innovative creative concepts across multiple media platforms. Immersive Stories was founded in 2017 by producer and director, Ane Skak, building on her experience with editorial management, production and strategy planning in film, television, radio, podcast, digital content, and VR. Ane Skak has produced more than 50 VR experiences in close cooperation with institutions in Denmark and abroad.

In XTREME, Immersive Stories is engaged in the development of the creative concepts and experiences design for the virtual opera experience and the Irish traditional music concert.

Principal Investigator: Ane Skak

immersivestories.dk

United Kingdom

University of Nottingham

The University of Nottingham is a public research university in Nottingham, England. It was founded as University College Nottingham in 1881 and was granted a royal charter in 1948. The vision of a university devoted to discovery, enterprise and the advancement of the human condition, combined with his lifelong commitment to improving health and wellbeing, remains intrinsic to the culture of the University today and will continue to underpin our future purpose.

In XTREME, UN is responsible for developing the festival environment, as well as human computer-interaction for social experience, co-design the social bubbles service.

Principal Investigator: Prof. Steve Benford

nottingham.ac.uk

Finland

Finnish National Opera and Ballet

The Finnish National Opera and Ballet (FNOB) is an opera company and ballet company established in 1911 in Helsinki. The Opera House features two auditoriums, the main auditorium with 1,350 seats and a smaller studio auditorium with 300–500 seats. With around 550 employees, the FNOB employs 53 opera singers in soloist and choral roles, 74 dancers (additional 14 dancers in the Youth Company), and 111 orchestra musicians.

In XTREME, FNOB is a side for the XTREME virtual concerts and dance show performance use case.

Principal Investigator: Timo Tuovila

oopperabaletti.fi