The Danish clamping block manufacturer Lego is currently celebrating its 90th birthday. A research team from Stanford University has now trained an AI to read and implement the building instructions for Lego kits. The Lego instructions served the researchers as an example of the important human ability to learn to complete complex tasks under supervision. In the study, a deep learning network learned to translate the two-dimensional instructions into three-dimensional instructions for positioning and rotating Lego bricks.

For this purpose, the researchers developed a special Manual-to-Executable-Plan Network (MEPNet). It performed better and produced fewer error builds than two models used for comparison across all kits tested.

AI builds Lego model

Discussing their findings, the study’s authors point out that the biggest problem – determining the exact 3D position of an object from a 2D drawing – is alleviated by the limitations of building with Lego bricks. The model’s code and training data are available for download on GitHub.