Long-term independent use of an intracortical brain-computer interface for speech and cursor control

A man with ALS used a brain-computer interface (BCI) to decode neural activity into text and computer cursor movements, according to data from the BrainGate2 clinical trial. The system utilizes implanted microelectrode arrays in the speech motor cortex and AI models to translate brain signals into communication, as reported in the trial findings.

The participant, identified as ‘T15’, was 45 years old at his 2023 enrollment. He suffered from severe dysarthria but retained limited neck and eye movements. Before the BCI, T15 communicated at 6.8 words per minute (WPM) via interpreters or 6.3 WPM using a gyroscopic head mouse.

In summer 2023, researchers placed four 64-electrode silicon microelectrode arrays, known as Utah arrays, into T15’s left precentral gyrus. These arrays, provided by Blackrock Neurotech, target the speech motor cortex and connect to a titanium pedestal on the skull.

Did You Know? The BCI system required a large wheeled computer rack to function, which T15 used throughout his home during the study.

How does the brain-to-text BCI translate thoughts?

The system records neural activity from 256 electrodes. This data is processed through two different neural network architectures: a Recurrent Neural Network (RNN) and a transformer-based model, according to the study.

Both models predict a distribution of 41 output classes, consisting of 39 American English phonemes, a silence class, and a blank token. On post-implant day 600, the researchers switched to the transformer-based decoder to achieve higher accuracy.

The transformer model uses a 12-block causal transformer with multihead attention. It was trained offline using labeled data from copy tasks and confirmed personal use trials, as detailed in the trial records.

How does the system control a computer?

T15 used the BCI to move a computer cursor and perform clicks. Initially, he was told to imagine moving his right arm and hand, though he later reported relying on “intuition” to move the cursor.

The system integrated a custom application called ‘BG Home’ to control mouse movements and key presses. This allowed T15 to use his personal computer through BCI control.

The study compared neural cursor control with gaze-based control using a Tobii Pro Spark eye tracker. In a 6×6 grid task, T15 achieved 1.67 bits per second with the neural cursor and 2.59 bits per second with gaze control.

Expert Insight: Samantha Carter notes that the participant’s transition from conscious imagery of movement to an intuitive control strategy suggests a significant adaptation in how the brain interacts with the BCI interface over time.

What are the performance and latency results?

Latency is divided into real-time feedback and sentence finalization. Real-time inference latency was estimated between 160 and 240 milliseconds, according to the data.

Sentence finalization was faster with the transformer-based encoder, which had a median latency of 1,791 milliseconds, compared to 2,657 milliseconds for the RNN. Latency increased with sentence length; sentences over 20 words took a median of 5,181 milliseconds with the transformer.

To maintain stability, both models underwent continuous fine-tuning during real-time use. New day-specific embeddings were created once 5 to 6 new sentences were collected to track neural changes across sessions.

What may happen next with this technology?

The current system relies on a distributed, multicomputer setup that limits portability. Future versions could be miniaturized into an embedded platform with a portable external interface.

Multiscale Semi-Markov Dynamics for Intracortical Brain-Computer Interfaces

Researchers may continue to optimize the transformer-based decoders to further reduce sentence finalization latency. The system could also be adapted for more widespread clinical use if the hardware footprint is reduced.

Frequently Asked Questions

What is the BrainGate2 clinical trial?
BrainGate2 is an FDA-approved, sponsor-investigator-led multicenter study assessing the safety of chronically implanted intracortical Utah arrays in individuals with paralysis.

How did the BCI system improve T15’s computer access?
Through the ‘BG Home’ software, the BCI translated neural signals into mouse movements and key presses, allowing the participant to control his personal computer.

What is the difference between the RNN and transformer decoders?
The transformer-based decoder, introduced on day 600, provided higher accuracy and lower median sentence finalization latency than the original RNN model.

Do you think neural-based cursor control will eventually replace eye-tracking for people with limited mobility?