Adapt image-generating AI to make it compatible with 3D animations


Generative adversarial networks are a class of algorithms that have made artificial intelligence famous, due to the various recently known examples in which their ability to generate images similar to those of some predisposed data set.

Extending the capabilities hitherto attributed to these systems, new research presented a modification of StyleGAN2, a popular generative adversarial network primarily used for generate facesadapted to work with 3D graphics.

Generative multiplane imaging: Generating 3D images using artificial intelligence

A recent project, jointly developed by Apple researchers and the University of Illinois Urbana-Champaign, United States, addressed the problem of generating 3D geometry and its corresponding texture from single-view images.

The researchers based their work on StyleGANv2, whose code was released by Nvidia. On top of this project, they developed a new generating branch capable of producing a set of fronto-parallel alpha maps in a spirit similar to three-dimensional images.

According to what the researchers pointed out in the presentation of their project, this initiative only required two adjustments to make a 2D generative model compatible with 3D.

First of all, it was necessary to add a multiplane generator branch to StyleGAN, so that it can generate a depth map on the built plane. Secondly, the implementation of a discriminating system conditioned by the pose, conditioned by the position of the camera, was also required.

The researchers behind this project call the products generated by this system as “generative multiplane imaging” (Generative Multiplane Images, in English, or GMPI), which “Not only are they of high quality, but they also ensure display consistency, which makes GMPIs different from much previous work”.

The researchers hope that the simplicity of the proposed method will inspire future work to overcome the current limitations associated with its application.

A series of examples, such as those presented in the image that accompanies this note, can be reviewed in action, as videos, in the project website. It is recommended to review that portal from a computer, since the animations are processed from the browser.