In 2021, the Meta team previewed a new project in development that would animate any drawing, or doodle, with people or characters. Yes, animations from drawings.
Meta has not only made progress on this project, but is also sharing its code so that developers can create their own projects or bring new ideas to life.
This AI-powered tool creates animated characters from your drawings
In a previous article, we already discussed how the Meta developers were using the power of AI to animate the figures that were displayed in any drawing. It didn’t require any fancy drawing, as a simple child’s drawing was enough for the AI ​​to create an animation of the character:
Our researchers envisioned using object detection models, pose estimation models, and image processing-based segmentation methods to quickly create a digital version of the drawing.
To do this, you need to go to this link and choose one of the sample drawings, or upload your own drawing. And then you just have to follow a series of instructions. For example, for the tool to give the expected result, Meta asks us to choose only the part of the drawing in the selection box.
And once he scans the image and separates the drawing from the background, he highlights what the character’s joints would be to perform the movements. They may not have been in the right place, so the tool gives you the option to modify them according to the design of your drawing.
When you finish making all these changes to fit what the tool requests, you will see that the drawing is scanned again to start the action. You can choose a series of movements to bring your drawing to life from a variety of animations, for example, to make it look like it’s dancing, walking, jumping, etc.
Meta releases an open source version of its AI-powered tool
Since the launch of this tool, users have uploaded hundreds of thousands of drawings that have been used (with consent) to train the AI, so this project has a large data set.
And because of this interest, the Meta team considered releasing an open source version with the models and code used in this project:
By publishing the models and code as open source, the project provides a starting point for developers to develop and extend the project, fostering a culture of innovation and collaboration within the open source community.
Those who want to use this code just have to follow the instructions on GitHub for installation.
