Meet the People Building the TV Controlled by Your Brain

Let's take a moment to think about some of the things many of us like to do with our downtime.

I'll start. Picking a glass of wine from an extensive list at a bar. Texting a friend to hang out. And, of course, browsing Netflix for something to binge-watch.

Now, imagine having limited mobility, and being restricted in carrying out what many of us considering to be mundane, day-to-day activities.

Selecting a movie to watch on Netflix, for instance, would be close to impossible with restricted motor skills, without the ability to manually scroll through options -- and in some cases, to dictate to someone else what we want to watch.

For many, that thought is a reality -- and now, a team of researchers is building technology to address it. Here's what we learned about the technology at this week's Samsung Developer Conference, and what's next for the people developing it.

Meet the People Who Are Building a Way to Control a TV With Your Brain

The name is Project Pontis: an initiative to develop a device that can control a TV only with the brain.

"How can we provide accessibility to people who cannot move?” asks Ricardo Chavarriaga, a scientist at Ecole Polytechnique Fédérale de Lausanne (EPFL) -- which is working in partnership with Samsung and the Swiss Paraplegic Centre. "We try to develop systems that allow people with severe motor disabilities to interact with their environments."

One of those interactions is something as seemingly simple as choosing a movie to watch, which was the activity demonstrated at today's event. "It will be an assistive tool for people with disabilities," says EPFL's Ruslan Aydarkhanov, "so that they can be independent and use the TV without external help."

At the center of Project Pontis is a physical prototype pictured below, which records electrical activity from the brain that the technology uses as signals that could indicate the user's intent.

Ruslan Aydarkhanov demonstrates the Project Pontis prototype

Then, using artificial intelligence and machine learning, those signals are converted into what Aydarkhanov calls "meaningful information": the details that translate into what the user wants to do.

This particular prototype uses a combination of the aforementioned brain signals and eye movements to assist with the activity of selecting a movie to watch.

"For the simple selections of menus, we just use eye movements, because it’s much more efficient," explains Aydarkhanov. "But when you want to find a movie, it’s a more complicated task, because you’re not sure exactly what you want to see."

That's where the artificial intelligence and machine learning come in.

"We use the reaction of the brain to see what is interesting to you," Aydarkhanov says, "and we show a sample of movies so that the system can, based on your reaction, understand what is this particular thing that you want to watch."

But what presents one of the biggest challenges to developing this prototype are the nuances of human interaction -- whether it's with other humans, or with machines.

Ricardo Chavarriaga speaks about the technology of Project Pontis

"A brain is not static, so every day, it’s changing little by little," Aydarkhanov says. "That means the signal that we record is also changing, and we have to adapt the artificial intelligence system to still make sense of the signal."

What the prototype is trying to replicate is that nuanced brain "signature" that is produced as a result of external stimulus. It's far from simple, and actually involves a combined process triggered by that external stimulus and what the user is experiencing internally.

"How does the user’s brain react when it has this ‘click’?" asks Chavarriaga. "I need to be sure that what I’m getting from the electrodes [read by the prototype] actually means what I think it means ... in this case, what the person wants to do with the TV."

Should that "click" be successfully replicated, Chavarriaga suggested, it could be carried over to other intelligent devices, such as prosthetics, which would likely be heavily reliant on technology that can understand user intent of movement -- something that also changes quickly and frequently, based on internal and external stimuli.

“Everybody is different. This variability has to be taken into account, [which is why we need] technology that is more intelligent,” Chavarriaga explains. “We have to find ways to properly match technology to the users, and we have to take into account the variability of the users.”

Project Prontis is one of Samsung's Corporate Citizenship programs, which works to develop technological advancements to meet various social and mission-driven needs.

The current trajectory of Project Prontis is to develop a second prototype into the first quarter of 2019. After that, the plan is to test the technology in hospitals and specialized institutions throughout Switzerland.

Featured image source: Samsung