Rec Room built a world using generative AI tools called Fractura.
Last year Rec Room added direct integration with Unity, letting creators build higher quality worlds in the Unity editor instead of being limited to the ingame shape-based creation tools.
Fractura is such a world, but mainly built using a pipeline of generative AI tools instead of fully manually.
Rec Room says ChatGPT was used to “develop the ideas” for Fractura’s “landscapes, flora, and lore”. Midjourney and DALL·E were then used to come up with concept art for the world’s design.
Concept designs from Midjourney and DALL·E.
But what about the actual creation of the world, not just ideas and concepts?
For the skybox, Rec Room says it used Skybox AI by Blockade Labs. Some Quest users have already been using this to make custom skyboxes for their Horizon Home environment. In Fractura, you can choose from multiple skyboxes generated by Skybox AI.
To generate the 3D assets for the world itself, Rec Room used CSM and Shap-E to populate Fractura with trees, ancient stone robots, floating land masses, and distant buildings. However, Rec Room found these tools generated models with suboptimal polycounts that weren’t conducive to realtime performance, so had to manually simplify the results. Rec Room also notes that the generation time for 3D models was very long, which was cumbersome.
You can visit Fractura in Rec Room now to see the result for yourself.
I tried out Fractura on Quest 3 standalone and found the performance was terrible, frequently dipping below 50FPS, but that’s a widespread problem with social VR platforms with user-created worlds like Rec Room and VRChat.
Still, performance issues on standalone VR aside, Fractura is an early primitive experiment that points towards what could be the future of user-created virtual worlds. As in the Builder Bot demo Meta showed off last year, generative AI combined with speech recognition could one day lead to a Star Trek like future where people can build virtual worlds from within VR, using just their voice.
