Neuroscience: Translating handwriting brain activity into text

A method of communication for people with paralysis that uses a computer to decode attempted handwriting movements from brain signals is demonstrated in Nature this week. The approach may allow much faster communication than was previously possible.

Brain–computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. A major focus of research in this field has been restoring large movements, such as reaching and grasping. However, highly dexterous movements, such as handwriting or touch typing, might enable faster communication rates, which have previously been limited to a maximum of around 40 characters per minute using point-and-click typing with a 2D computer cursor.

Francis Willett and colleagues found that a participant in their study, who was paralysed from the neck down, was able to reach a writing speed of 90 characters per minute with 94.1% accuracy when using the new handwriting BCI. The authors instructed the participant to ‘attempt’ to write sentences as if his hand were not paralysed, by imagining that he was holding a pen on a piece of ruled paper. During this exercise, the BCI used a neural network, a type of machine learning, to translate attempted handwriting movements from neural activity into text in real time. The typing speeds achieved are more than twice as fast as those reported for any other BCI so far, and are comparable to typical smartphone typing speeds in people of the same age group as the study participant (115 characters per minute).

The proof-of-concept findings open a new approach for BCIs and suggest that a handwriting BCI is capable of accurately decoding rapid, dexterous movements years after paralysis. Nevertheless, further demonstrations of its longevity, safety and efficacy will be required before it can be put to widespread clinical use. In addition, the authors suggest that these methods could be applied more generally to any sequential behaviour that cannot be observed directly; for example, decoding speech from someone who can no longer speak. However, the technology “will need to provide tremendous performance and usability benefits to justify the expense and risks associated with implanting electrodes into the brain,” note Pavithra Rajeswaran and Amy Orsborn in an accompanying News & Views article.

After the embargo ends, the full paper will be available at: https://www.nature.com/articles/s41586-021-03506-2