In recent months we have seen an avalanche of artificial intelligence systems that transform text into images. There are all kinds, for all tastes, and the results can be truly impressive.
Now Google announces DreamFusionthe generator 3D text to image last generation.
Available at imagen.research.google, they announced it via Twitter with several examples of what it can do, but before comparing it to other existing platforms, let’s take a closer look at this new system.
It is an evolution of Dream Fields, a 3D text-to-image generator that Google already showed us last year. What it does is create 3D images using partial 2D datasets, with a pre-trained text-to-image request model.
Unlike Dream Fields, which used OpenAI’s CLIP technology as the last pre-trained model, DreamFusion now uses its own: Image, Google’s DALL-E 2 competitor.
Ben Poole is the submitting research scientist, co-author of the proof-of-concept paper, who published:
We optimized a NeRF from scratch using a pretrained text-to-image broadcast model. No 3D data needed!
Because you don’t need 3D data, it’s much easier to train, so you’ll learn faster and get better over time.
It doesn’t have the photorealistic quality of DALL-E 2 image results yet, but the generated drawings make sense, with great depth and good boundaries, no abstract data randomly thrown by the drawing.
Google’s Dreamfusion vs Dall-E, imgcreator and Stable Diffusion
It is not yet available to the public, but there are some examples on its website that we can use to compare it with other tools on the market.
Let’s see how he draws the phrase “A brain mounted on a rocket heading for the moon”
In Imgcreator there is good quality in the result, although the brain is not mounted on the rocket.
With DreamStudio Lite we see the power of Stable Diffusion, where the brain only appears in one of the 4 published designs.
Here we have the example with the new Google project. Perfect, with all the elements and with the brain “mounted” on the rocket, just as the description indicates.
DALL-E has it backwards, instead of the rocket-mounted brain, we have the brain-mounted rocket…
It would still be necessary to test DALL-E, hopefully we can do it soon.