Nvidia introduced Magic3D🇧🇷 an AI model that can generate 3D models from text descriptions🇧🇷
After entering a phrase likeor “A blue frog sitting on a water lily”, Magic3D generates a 3D mesh model, with colored textures, in about 40 minutes. With modifications, the resulting model can be used in video games or CGI art scenes.
Nvidia frames Magic3D as an answer to DreamFusion, a text-to-3D model that Google researchers announced in September.
Just as DreamFusion uses a text-to-image template to generate a 2D image that is optimized on NeRF (Neural Radiance Field) volumetric data, Magic3D uses a two-stage process that takes a template generated at low resolution and optimizes it for a higher resolution.
According to Nvidia, the resulting Magic3D method can render 3D objects twice as fast as DreamFusion.
Magic3D also allows edit 3D mesh model on demand🇧🇷 Given a low-resolution 3D model, you can change the text to change the resulting model.
In 2022, we saw the emergence of templates capable of converting text to 2D imagessuch as DALL-E and Stable Diffusion, and rudimentary text-to-video generators from Google and Meta.
As for Magic3D, the researchers hope that it will allow anyone to create 3D models without special training.
Once perfected, the resulting technology could accelerate the development of video games (and Virtual Reality) and perhaps find applications in special effects for film and television. “We hope that with Magic3D we can democratize 3D synthesis and open up everyone’s creativity in creating 3D content,” say Nvidia engineers.