In recent years, advances in brain-computer interfaces have been amazing. Researchers and entrepreneurs have been working on harnessing the potential of these interfaces to improve cognitive abilities.

In a recent study, published in the journal Intelligent Computing, University of Glasgow researchers Gao Wang and Daniele Faccio demonstrated that it is possible to connect a human brain and a computer to perform simple computational imaging tasks.

Reconstructing visual planes with ghost images

A ghost image is a technique used in signal and image processing to reconstruct an image of an object that is hidden behind a barrier or wall. The technique is based on illuminating the object with different light patterns and detecting the light intensity values ​​that are reflected by a surface close to this object.

Instead of using a traditional camera or image sensor to capture light intensity values, the visual cortex of the human brain is used as an image sensor. This process is done by measuring the electrical signals generated by nerve cells in the brain that respond to patterns of light reflected from the surface.

The electrical signal data is processed by algorithms that reconstruct an image of the hidden object based on the detected light intensity values. This ghost image technique may have applications in various areas, such as medicine, security, and scientific research.

Wang and Faccio used ghost images to computationally reconstruct images of an object hidden behind a wall. The researchers used the human visual system for two purposes: it not only detected visual cues, but also gave feedback in real time.

How the experiment worked

In Wang and Faccio’s experiment, flickering light patterns were projected onto an object and reflected from a secondary white surface. The reflected light intensity values ​​were detected by the participant’s visual cortex and captured as electrical signals by a single-electrode earphone.

The signals were processed on a laptop by an algorithm designed to reconstruct an image of the hidden object. “A first impression might be that the visual system simply acts as a different type of camera,” the authors explained.

To obtain higher resolution and more efficient images, the authors added an adaptive computational loop. With the help of real-time feedback provided by the human visual system, they could modify projected light patterns and “carve out” the least useful ones to improve both frame rate and image quality.

Challenges and next steps

The authors found in a follow-up experiment that when participants were asked to say or type their perceived light intensity on a keyboard while their brain signals were monitored, the quality of the reconstructed images dropped.

Wang and Faccio believe their paper lays the foundation for brain-computer systems to enable alternative forms of computation within and beyond images. They also noted that much more remains to be investigated, given the tension between conscious and non-conscious processing.