Detecting Parkinson’s Disease using Deep Learning Techniques from Smart Phone Data

Identifying Parkinson’s Disease early is crucial for slowing the disease progression and a new tool developed by Khalifa University can now detect the disease using sensors on the average smartphone.

By Jade Sterlin

Parkinson’s Disease is the second most common neurodegenerative disorder, affecting more than one percent of the population above 60 years old. Often beginning as a barely noticeable hand tremor, over time, the disease interferes with movement, muscle control, and balance. Fine motor impairment (FMI) is progressively expressed in early Parkinson’s Disease patients but clinical techniques for detecting it may not be robust enough.

A team of researchers at KU including Dr. Leontios Hadjileontiadis, Professor of Biomedical Engineering and a member of KU’s Healthcare Engineering Innovation Center (HEIC), has developed a tool that can screen for early motor Parkinson’s symptoms and alert individuals accordingly via their smartphones.

In collaboration with researchers from Greece, Germany and the United Kingdom, Dr. Hadjileontiadis introduced a deep learning framework that analyzes data captured passively and discretely during normal smartphone use and published the results in Scientific Reports.

“Remote unsupervised screening via mobile devices can raise awareness for medical care, with daily data assisting diagnosis,” explained Dr. Hadjileontiadis. “User interaction with smartphones can unveil dense and multi-modal data to reveal patterns that can be connected with both motor and cognitive function. In particular, Hold Time, the time interval between the press and release of a key, offers insights to the probability of a subject suffering from Parkinson’s.”

The rate at which a person presses down and then releases a finger on a key indicates how quickly the brain can control the muscles. When the body needs to start moving, the brain’s motor cortex sends signals to the spinal neurons to activate the muscles. Dopamine is one of the neurotransmitters involved that ignites a chain of events resulting in a movement, a feeling or an action. For Parkinson’s Disease patients, dopamine-producing cells in the brain become inactive and the loss of dopamine leads to issues with movement. Symptoms of the disease become increasingly more apparent and the patient develops tremors, difficulty walking, and other issues with movement.

“Detecting these smaller tremors at the start of the disease can lead to earlier diagnosis and allow us to implement management strategies earlier,” explained Dr. Hadjileontiadis. “The standard medical practice in diagnosing Parkinson’s Disease requires years of expertise. Using a smartphone provides an unobtrusive way of capturing data as we link keystroke typing with an enriched feature vector to describe the keystroke variables.”

Additionally, acceleration values from the smartphone’s Inertial Measurement Unit (IMU) sensor are used to monitor for hand tremors. This also is a source of data captured passively and unobtrusively as users perform common actions with their phone, from placing calls to typing messages.

When combined with deep learning, these data could provide a novel tool for effectively remotely screening the subtle fine motor impairments indicative of early onset of Parkinson’s Disease. Deep learning has been previously shown to be highly effective in extracting useful representations from high dimensional information like images, and the research team showed that deep learning can be leveraged to quantify touchscreen typing based information that is strongly correlated with FMI clinical scores.

In screening for Parkinson’s, deep learning algorithms can detect the disease from MRI scans, tremors recorded on accelerometers and voice degradation from voice signals. Now, typing on a smartphone can monitor keystroke dynamics in everyday activities.

“We tried to detect Parkinson’s Disease using a multi-symptom approach that merges passively-captured data from two different smartphone sensors via a novel deep learning framework,” explained Dr. Hadjileontiadis. “Our method is inspired by the typical workflow of a neurologist, in the sense that it outputs a score for tremor and FMI, two of the most common motor symptoms, as well as a score for Parkinson’s Disease.”

Automated Parkinson’s Disease detection is not a new idea. Many sensors have been tested to capture specific aspects of different symptoms, such as IMU sensors for gait alterations, microphones for speech impairment, keyboards for rigidity, and writing equipment for fine motor impairment. The common denominator in these studies is that they attempt to infer Parkinson’s Disease from single symptom cues. This is inherently problematic as Parkinson’s manifests differently in different subjects, meaning any system that can reliably detect the disease needs to cover multiple symptoms. The research from Dr. Hadjileontiadis is multi-modal in this way, capturing data unobtrusively and ‘in-the-wild.’

Using deep learning techniques, the team achieved 92.8 percent sensitivity and 86.2 percent specificity for Parkinson’s Disease detection. Not only is their proposed framework performing well, but it can also be extended to include additional data in the same architecture, including speech information, for example.

“Performance-wise, our approach produced good classification results and this is the first work to address the problem of detecting Parkinson’s from multi-modal data,” said Dr. Hadjileontiadis. “This is a solid first step towards a high-performing remote Parkinson’s Disease detection system that can be used to discreetly monitor subjects and urge them to visit a doctor signs of the disease are detected.”

Read more about KU’s Healthcare Engineering Innovation Center (HEIC) here.