Merging the physical and digital worlds in a seamless and natural way, requires that computers share our understanding of the visual world. 3D scanners are finding their way to consumers - and all of us will soon start scanning the world. Understanding scanned content and transforming it into Virtual Reality and Mixed Reality experiences, is a prerequisite for democratization of VR and MR and content explosion. The talk will cover advancements in computer visual perception, the work done on this topic at Resonai, and future applications in VR and MR.