Collaborative mixed reality: trialling a new immersive teaching format

Several participants – trainer(s) and learner(s) – are in the same room. At its centre is a reference point shared by everyone, which gives a “common” 3D overlay. Some people can use a virtual reality headset, others a phone or tablet.

Everyone sees the room on video, but 3D content is added to the real world: this is a mixed reality format. The level of immersion differs between a headset and a phone, as does each person's physiological tolerance to the device. The school or trainer can therefore choose which devices to deploy, while the software development stays the same (a single Unity project, several possible builds).

All participants are “connected” to each other through a transparent local network, with no setup required.

The trainer has controls over the module (switches to show or hide 3D parts, scale changes…) to act live on the teaching scenario. The trainer is at the heart of the module: they can talk, move around and gesture in the environment, since their silhouette and everyone else's are shown as video.

Participants therefore see each other, but inside a 3D environment. The trainer can, for example, use physical tools in a virtual environment or vice versa, or simply walk through a BIM model and point at elements as if they were really on site. The format can be combined with 3D scans (photogrammetry, laser point clouds…) for a high level of realism.

Benefits and what's new about the format:

• No 3D environment (set) to build in mixed reality: the focus stays on the teaching content in the training room.
• Limited interactivity: the trainer runs a simple but effective scenario.
• A less perishable immersive module, with no voice-over, text or interface that could date as laws or professional practices change: the trainer drives the content of the module.
• Hardware-agnostic (interoperable) development that can easily evolve over the years thanks to the free OpenXR, XR Interaction Toolkit and Netcode for GameObjects standards and frameworks.
• No third-party resources outside Unity that would be paid or risky. Runs entirely locally, GDPR-friendly thanks to privacy by design.