NEXT: Mood-Sensing Technology & Reactive Media
Both of my boys read “Choose your own adventure” books when they were younger. You know these books, where you read a few pages, and then you are given a set of choices at the end of the page: “If you want to go through the door, turn to page 27” or “If you want to turn and go the other way, turn to page 49,” and so on. The more modern version of this is Interactive Fiction (IF) which works along the same principles except that rather than turning pages you click onto links. Interactive Fiction had a following among a few techno-geeks, but I don’t think it has really caught on either with readers nor with video gamers (why read text when you can experience an adventure).
That’s why I was curious when I learned of “neurofiction.” Developed by Hannu Rajaniemi and Samuel Halliday, the system involves a reader wearing an electroencephalography headset, like the Emotiv headset from Epoc. Depending on the signals the brain sends while reading a piece of fiction, an algorithm will alter the story to fit the mood of the reader. The first such neurofiction experience was displayed at the Edinburgh Science Festival, and involved a version of the story of Snow White. Those whose brain patterns exhibited an “affinity for life imagery” would soon discover that the story bent toward such imagery, while those whose brainwave signaled a preference for death imagery would find themselves reading a different story. There are 48 possible narrative directions the story might take depending on how the algorithm interprets the reader’s brain waves. “It would be really cool,” says Rajaniemi, “to see if, in a science fictional context, we could find elements that would elicit a ‘sense of wonder,’ for example, and then try to write a story that optimizes for those.” Unlike Interactive Fiction, the experience is like reading a typical book, in that there are no links to follow or pages to turn to: the story unfolds in linear fashion, the alterations seamless and unobtrusive. The creators observe that “by opening themselves to be read, neurofiction readers become subconscious collaborators in the creation of a new narrative.”
The headlines in the technology press were inevitable: “The book that reads you!” or some such like this. Like Interactive Fiction, I suspect that neurofiction will be a niche genre. It is unlikely to replace traditional books and reading conventions. On the other hand, I suspect that the experience of video games will be impacted. A game that senses my moods might be altered to heighten certain emotions. Each player would experience — and feel — a different game from any other player.
And indeed such “reactive media” has not been confined to textual fiction. At SXSW in 2011 Sensum showed a film that relied on skin response sensors to gauge emotional responses. Alexis Kirke has similarly produced a film that uses “brainwaves, muscle tension, perspiration and heart rate in a selection of the audience to adjust the story in real time, choosing the most appropriate of the film’s four narratives to maximise intensity.”
Beyond their entertainment use, mood-sensing technology will prove powerful marketing tools. Advertising has long been built on the manipulation of our emotions and unconscious longings. The same tools that can read our brainwaves to craft a unique narrative might also be used to exploit our emotions in new ways. If our brains send off emotion cues, that “emotional data” will no doubt be harvested in a manner similar to other kinds of data we emit. In addition to crafting emotionally-specific narratives, advertisers will perform analytics on that emotional data to craft new forms of advertising, finding new ways to make recommendations and suggestions as Amazon or Netflix does. “If your unconscious says you like death imagery, then you’d also like…” As we interact with mood-sensing technologies, the unconscious itself will be mined for our emotional data.
Mood-sensing technologies would also accelerate the trend toward “digital bubbles.” If my media is chosen for me to be emotionally soothing, compatible and non-disruptive, then what will such technology do to non-fiction? My news feed might be tailored such that I read only news stories that align to my mood. The algorithms might filter out or refashion anything that does not conform to my emotional needs. Mood-sensing technology might interpret my emotional reactions to “immigration” or “Nancy Pelosi” and modify anything I may read or watch to conform to those attitudes. If my impression of Pelosi is favorable, I might only receive positive stories about her; if I loathe immigrants, those stories might be composed in such a way to heighten the negative feelings I might have. I need not be confronted by any news or information that does not already conform to my emotional and unconscious biases.
The next Columbus Futurists monthly forum will be Thursday March 23 at 6:30 PM at the Panera Bread community room (875 Bethel Rd.) Our topic for the evening will be “The Changing Picture of Jobs”.