Movies May Start Watching You

Tracking Audiences in Real Time May Change What you See on the Screen – In Real Time.


In the past few weeks, details of Disney’s recent turn to AI for audience tracking have begun to surface.  Disney is now using AI to actually capture and analyze moviegoers’ facial expressions as they watch a film. The end goal behind the media giant’s AI initiative is to produce something the audience wants, even if they don’t know they want it.

Sounds like science fiction once again becoming science fact. In this case, data-science fact.

How AI audience tracking works.

Discreetly placed cameras capture the individual audience members’ every reaction to the film. Even if there are hundreds of people in the theater, the cameras map each facial expression of each person. The more the merrier. For humans, smiling faces, huge guffaws, shocked expressions and teary eyes are easy to see and measure, but this technology goes much deeper to measure the slightest microexpression: minute ticks that happen in 1/25th of a second, all but lost to the human eye.

These subconscious, involuntary expressions reveal a person’s true emotion. Once captured,  the results of a viewing  are stored in a database of millions of facial expressions. Did you know that the upward curl of a lip is a world apart from a slight stretch of a grin? This is key information Disney (and others) want to gather to gain feedback on the movie, television show or even a song. Anything that can illicit an emotional response is something that can be measured and thus improved.

Clinical Psychologist and noted facial expression expert Dr. Paul Ekman has made a study of microexpressions. Ekman found that microexpressions often occur when someone is trying to conceal their feelings, like when a lie is being told. This he calls conscious –or intentional–suppression.. Just watch a few episodes of Lie to Me to understand what these expressions can tell. Dr. Ekman further explains that these microexpressions also occur when people are unaware of how they feel, called unconscious repression (unintentional). To an AI-supported camera, the cause doesn’t matter, only the resulting expression.

This is the power and potential of AI. Collecting the data is the easy, if not somewhat creepy, part. Analyzing the data and applying the results are the challenges. The idea is to measure the audience response to specific parts to a movie, all the way down to whether or not a line of dialogue resonated with the audience. If you combine the (microexpression) responses with the demographics of the audience you have the ability to craft and market new content that will resonate in the future, or in the case of Disney’s AI team, resonate later in the film.

What might the possibilities be?

It is easy to see this technique being used in focus groups and test audiences to measure creative choices by the filmmakers, such as choice of actors, action sequences, humor, imagery, etc. But these are willing participants. Unfortunately, knowing participants are slightly biased and produce slightly different results than raw data captured in the wild (audiences that don’t know or care their honest expressions are being measured). Yet the possibilities remain endless. Audiences can be treated to alternate endings, real-time changes in dialog, an increased presence of a likeable character, an increase (or decrease) of special effects, all based on a collective audiences’ response AS THE FILM IS BEING WATCHED! For this to happen, the producers need only prepare the alterations in advance. Taken a step further, the experience can follow you home as you watch a movie in your living room and the AI watches you. Yes, the camera saw you get up during a boring scene to make a sandwich. “Scratch that scene.”

Will this go too far?

We all know about facial recognition software and its capabilities. The software has become so exact in its analysis that your face will become your fingerprint of the future. It will allow you to stroll through security, purchase items without ID, even open your front door without a key. But will audiences want their face and its telltale expressions used for other things? Imagine getting an email that says, “We noticed how you laughed during the scene of Maui (Dwayne Johnson) singing the song, You’re Welcome. Would you like to buy the song or the Maui doll?”

Without the power of artificial intelligence, the opportunities to fashion content to an audience’s delight would not be possible. It behooves the studios to continue to pursue the technology in order to continue drawing larger audiences. In the near future, you may be telling your date, “Let’s go see a movie.” The studios will be saying, “Let’s go see an audience.”



Related articles:


Super-cool game tech will copy your facial expressions in real time


Facial recognition freak out: what the technology can and can’t do


Disney’s research publication