StoryFit Successfully Predicts Audience Sentiment
What if you could predict how an audience will respond to content before it’s actually produced and distributed to the public?
Sounds like a simple idea, but production studios and streaming services spend hundreds of millions of dollars every year estimating this answer, only to have their content underperform when it hits the screen.
StoryFit changes that.
StoryFit’s AI engine applies 100’s of proprietary models to measure 100,000+ character and story elements. Measured against billions of audience data points, our story intelligence works to simulate audience responses, uncovering what makes your story unique by producing powerful predictive insights into how viewers will respond before production.
To put StoryFit’s AI engine to the test, we partnered with emotional analytics company Kouo, to conduct a study to test how accurately StoryFit predicted the audience sentiment of two notable Crime-Drama-Mystery pilots – one which was a hit, the other a flop.
The study was structured so that StoryFit would measure the strength of these two stories from the scripts, producing powerful story intelligence insights in the pre-production stage. Kouo would then apply their emotional response biometrics data by measuring the response of real life humans watching the finished produced content.
The research revealed that StoryFit accurately predicted user engagement and showed significant overlap in viewer’s scene-by-scene emotional response to the two shows from Kouo’s data.
What makes for a successful Crime-Drama-Mystery? Killing Eve vs. Clarice
Is it the emotional intensity or suspense levels of the plot? How about the main character and their interactions? Or, likely a combination of all these things?
For our analysis, we selected two shows within the Crime-Drama-Mystery genre – Killing Eve, which proved to be a hit across all areas versus Clarice, which underperformed against expectations, resulting in a single season. But let’s let the data speak for itself.
Clarice received an audience score of only 59% on Rotten Tomatoes. After widespread negative feedback resulting in the “lowest-rated and least-watched show on the network that year,” according to Deadline, executives decided to cancel it.
Killing Eve, on the other hand, surpassed expectations. Met with a critical consensus of high reviews, the show earned a 96% on Rotten Tomatoes and became a NYT Critic’s Pick, along with receiving several other television accolades.
While we would expect to see some differences in both level of violence and language advisories (as Clarice premiered on cable T.V., while Killing Eve was produced for network T.V.), audiences would likely react similarly to the genre itself.
When we ran each of the pilot episodes through our story intelligence platform, our data displayed these same findings.
StoryFit’s Findings
We divided these metrics into three categories of data metrics for this analysis:
- Audience Perception – story elements that describe how a viewer would experience it
- Criminal Activity – themes that exist within each of these Crime-Drama-Mystery pilots
- Character Motives – what compels a character to act/react
In all three categories, Killing Eve came out on top, reaching the Impact and Prominent level of scores (top 10% and 25% of all shows within the genre) for every element.
Viewership Experience
Viewership experience predictors simulate an audience’s response to watching content. Three metrics that are commonly found to drive audience engagement are Enjoyable, Exciting, and Entertaining.
Not only does Killing Eve’s pilot surpass Clarice, it reaches the impact threshold (level of heightened viewer awareness) for all three tones.
Emotional Intensity
Emotional Intensity is often referenced by both critics and audiences as a common characteristic of great stories, but rarely is it actually measured.
The metric measures the level of a viewer’s engagement or attachment to a story at a particular moment or time. High Emotional Intensity correlates strongly with high viewer interest, making this an indispensable element to predict success.
Emotionally intense screenplays often have many peaks and valleys – moments that spike above and below the presence threshold. Rises and falls in this metric add to the overall audience’s investment, working to create tension throughout.
Our story intelligence platform registered Killing Eve as having double the amount of Emotionally Intense scenes as Clarice in their respective pilot episodes.
Kouo’s Findings
Kouo is a U.K. based emotion analytics platform designed to connect real customer emotions to key metrics, transforming the way businesses and companies interpret consumer behavior.
For this partnership, Kouo measured the physiological data and emotional responses of viewers wearing an Apple Watch while watching both pilot episodes. Only individuals unfamiliar with the shows participated in the viewing sessions.
Comparing StoryFit’s analysis with Kouo’s findings, the data overlapped reinforcing the validity of our observations.
When measuring focus, excitement, and stress levels in study participants, Kouo observed that moments of low emotional intensity were at least double for Clarice compared to Killing Eve across all recorded emotions.
High – number of scenes where the time spent in high emotional state is above one standard deviation from the average.
Similar to StoryFit’s pre-production findings, Kouo identified several measures to characterize a show. In particular, three measurements correlated closely with StoryFit’s scene-by-scene data:
- Audience Disengagement — Scenes StoryFit deemed unimportant to the overall show. These correlated well with moments of minimum detected intensity in focus. Notably, Killing Eve’s pilot has far fewer moments like these than Clarice.
- Argument — Scenes in which StoryFit registered an argument taking place correlated to where study participants showed an increased time spent in high-stress states, as measured by Kouo.
- Obstacles — Scenes in which StoryFit deemed an obstacle is present correlated with scenes that Kouo detected having high time spent in high focus.
These correlation coefficients are especially meaningful because the models ran on two different inputs: StoryFit analyzed the pilot script, and Kouo measured the emotional responses of test viewers watching the finished pilot episodes.
Despite this, we observed high correlation, a true testament to how StoryFit models reflect the end viewers’ emotional experiences when watching the show.
Takeaways
The results of the study are clear – StoryFit and Kouo helped identify that viewers found Clarice 50% less emotive than Killing Eve, which is notably reflected in the difference in ratings for the two shows.
On comparable scenes, StoryFit and Kouo’s models had meaningfully correlated matches on insights, showing that StoryFit can identify insights reflective of the audience’s future emotional reactions.
We believe these tools are essential in predicting audience responses in the early development phases of large budget, high-stakes shows and films.
While StoryFit can successfully indicate the audience’s reception of a script before it goes into production, Kouo gives direct insights into audience responses to the content produced, reducing the cost attached to changes in post-production and forecasting the success of the finished product.
All of these are vital for studios, streaming platforms and production companies alike.
Interested in working with us? Learn more about Kouo and StoryFit’s groundbreaking work or request a demo to get started today.