The filmmaker Charles-Stéphane Roy offers audiences an unparalleled experience, during which the soundtrack of his film is generated by the audience’s cerebral activity.
Hugo Samson
With the support of the partnership cinEXmedia, the “adaptive cinema” work V F C will be presented in the “Explore” section of the Festival du nouveau cinéma. Film-goers can take part in this unusual audiovisual experience at the Cinéma Moderne in Montreal on 12 October. While waiting for the event, I interviewed Charles-Stéphane Roy, the film’s producer, director and scriptwriter.
“Adaptive Cinema”
V F C opens on a neurosciences researcher suffering an uncontrollable emotional shock when she hears a piece of music delivered to her in a mysterious package. When her colleagues notice the effect this piece of music has had on her, they decide to listen to it as well, subjecting themselves, like her, to a great shock.
What is peculiar about the film is that its soundtrack varies according to the cerebral experience of every spectator at the moment they view it. The team that made V F C calls this practice “adaptive cinema.”
“We use an electroencephalogram to analyse cerebral data in real time,” Roy explains. “‘Adaptive’ audio has long been used in the field of video games, among others. Our challenge was to apply this technology to an experience in linear cinema. People see the same thing, but do not hear the same thing at the same time.”
The electroencephalogram transmits the audience’s cognitive data to a server, which filters it, analyses it and joins it with pre-established soundtracks. These soundtracks are then transmitted to viewers through bone conduction headphones, which play the film’s soundtrack and an individualised soundtrack at the same time.
The team behind the film decided not to include dialogue in it. “The idea was to convey information and certain emotions through music and the image rather than through dialogue, as is the case in traditional films,” Roy explains.
Asked about audience response, Roy replies that his film gives rise to “something similar to the reaction people have when they try virtual reality for the first time. It’s impossible to imagine the experience without having tried it. It puts you in a state you are not necessarily familiar with.”
Putting Information in the Audience’s Hands
The V F C experience extends beyond the movie theatre. A report is given to each viewer so that they can consult the data produced by their cognitive experience while watching the film.
“One of our goals was to provide a great variety of experiences. If people compare their profiles and it is just 50-50, we have missed our goal. We wanted people to be able to see that there are several ways of experiencing the film.”
Rather than revealing the information in the report directly, the production crew opted, moreover, for a more interactive process, in which viewers must first come up with hypotheses as to the results they think were obtained before they receive them.
“We found that interesting, because if you do a poll on paper, people rely on their memory, they rely on an impression and can write anything. Whereas in this case, they compare their impressions with very concrete data.”
Roy points out that the reports are in accessible language. “They’re made for the average person,” he states. “We explain the process we used. Then we give more concrete information, like the name of the character who most captured people’s attention, or the scene which most captured their attention.” The reports also show the percentage of time each viewer spent in a state of dynamic cerebral activity and, conversely, the time they spent in a calmer state.
Cinema and the Neurosciences
The production crew of V F C was made up of film professionals, neuroscience researchers, experts in interactivity and user experience designers. “These are not people in the habit of working together, so we had to find a common language so that everyone could understand each other,” Roy adds.
“Almost all theories of narrativity, until quite recently, could be explained by the way in which the brain reacts when analysing a film. We have learned that the brain always shifts from a state of observation to a state of reaction, and that it then falls back to a state of observation. That happens almost automatically. The way stories are told takes this into account, whether they take place over ten minutes or three hours.”
The technology developed by the filmmakers has opened the door to numerous possibilities. “We are in discussion with organisations and studios to try to implement the system we have created in other projects, either in cinema or in immersive media,” Roy relates. “We see real potential for love shows, for performances, for anything collective.”
