Cosmic XR – Lighting on Physical Surfaces

Long weekend, small experiments on Cosmic XR: explored light reflections on real-world surfaces using Scene Understanding. Claude quickly understood the architecture and APIs of Meta Mixed Reality Utility Kit (MRUK https://developers.meta.com/horizon/documentation/unity/unity-mr-utility-kit-overview/) and integrated the feature into the existing Unity project through Unity MCP. By leveraging semantic surface classification, reflections can now be selectively projected onto walls, floors, tables, and other real-world surfaces. While this doesn’t add much practical value, it always feels a little magical to see digital objects interact with the physical environment around you.

It reminded me of the first time I experienced HoloLens apps like Young Conker and Fragments back in 2015, which beautifully blended virtual content with the real world using spatial meshes and scene understanding. Back then, spatial meshes were coarse, full of holes, and often required significant custom processing before they became usable. It’s amazing how modern scene understanding can instantly recognize semantic surfaces and reconstruct clean geometry, and how SDKs like MR Utility Kit make these capabilities easy to integrate into real projects.

It’s one of those quiet advances in spatial computing that makes you appreciate how far the platform has come and makes building mixed reality experiences feel increasingly effortless.

Cosmic XR:
https://www.meta.com/experiences/cosmic-xr-space-in-your-room/9293405654102503/