A New Genre of Audio Story
“Given the positive impact of music on film, one might wonder whether similar results would be found when combining specially-composed music with a fiction text” – Alicia Strong
When the fiction podcast genre emerged in 2012, it enabled a revival of interest in the old radio shows. This evolved into modern audio dramas. Audiobook publishers and fiction podcasts also began using full-casts, music, and sound effects. They became the key components in the sound design of audio storytelling—and this media convergence changed my life.
A Happy Accident
In 2011, I became interested in digital storytelling and podcasting, so I decided to teach myself how to use Audacity sound editing software. However, I had no idea it would result in an amazing journey of discovery and self-discovery about audio stories, film music and sound effects that resulted in my stories2music project and a new genre of audio story.
After I installed the Audacity program on my computer, I decided to narrate one of my short stories, "The Boy Who Was Loved by the Wind" on one track as an experiment. Next, I decided to lay a track of background music just to see how different tracks worked together. I had no idea where to find background music, but I remembered a YouTube video with this hauntingly beautiful background music called "Arco Noir" by composer Richard Allen Harvey. I googled the piece and found several CDs by Richard Harvey on Amazon because he was a movie composer, so his short orchestral pieces for movies, TV, commercials, and other media—called production music—were compiled on various CDs. I listened to all of the music pieces and then purchased four of them. Note that these were random pieces of music that I thought might work well for the story.
I inserted the four pieces of music on separate tracks in Audacity and then listened to the narration over the music. I was utterly astonished. I expected to hear nice background music for the story, but the music fit the story so perfectly that it could have been written specifically for it. The music matched the words, scenes, and emotions exactly. For example, if a kite fluttered here and there in the story, the music sounded like a kite fluttering. When the Wind grew very, very angry, so did the music. When the Wind expressed her tender emotions for the boy, the music was tender and moving. Sometimes it matched specifically and precisely to a short phrase in the story. It was just mind-boggling.
This intrigued me, so I experimented with my other short stories and found that Richard Harvey’s music consistently matched the stories perfectly. It was uncanny how these random music pieces fit every story as if they were written specifically for them in the same way that film music is written specifically for movie scenes by the composers. Most fiction podcasts use snippets of music in key spots to evoke emotion, to set the atmosphere, or to identify a character theme, but they don’t use background music throughout the entire story. If music was used that way, it would have to function like film music and match the scenes, the emotions, the characters, and the plot throughout the story.
How the Music Changed Everything
I soon realized that the film music was no longer simply background music. The music was an integral part of the storytelling. The words and music were intertwined, interdependent, and symbiotic—they were separate but depended entirely on each other to tell the story. They set the scenes, created the atmosphere, and told the listener how to feel about the story—which is exactly what narrative film music does for films. Robert Elliot said, "Narrative music is like another actor in the movie . . . like the announcer for a movie." According to Alicia Strong,
"Once music is linked with a visual narrative, it takes on elements beyond that of simply musicality—it takes on a character of its own, becoming almost as another player in the story, one with its own perspective, voice, and interrelations with other characters. Given the positive impact of music on film, one might wonder whether similar results would be found when combining specially-composed music with a fiction text."
Although the production music was not specifically composed for my stories, it synchronized with them as if it were. That changed my understanding of the power of words and film music to tell audio stories.
Unfortunately, I seemed to be the only person who "heard" how the music matched the words of the stories. When I let family and friends hear the stories, they liked them, but they didn’t understand why I was so astonished by them. They didn’t hear the connection between the words and the music or, more precisely, where the music fit the words. They could comprehend the general feeling of the music, but I could distinctly hear every place the music matched the words. Maybe it was because they weren’t as musically educated as I was.
Media Grammar
As it turns out, the concept of "media grammar" may partially explain my ability--and others’ inability--to hear the music’s connection to the words. According to Pavlik and McIntosh, media grammar is "the underlying rules, structures and patterns by which a medium presents itself and is used and understood by the audience." Essentially, all media—especially the arts—communicate their messages in specific structures that people learn to recognize through exposure and education. According to Joshua Meyrowitz, people must "have some understanding of specific workings of individual media" to have media grammar literacy. Anahid Kassabian explains that "musical competence is based on decipherable codes learned through experience. As with language and visual images, we learn through exposure what a given tempo, series of notes, key, time signature, rhythm, volume, and orchestration are meant to signify." I had a music background, so I definitely had music grammar literacy, and I was raised on movie soundtracks, so I had literacy there as well.
Most people who have been exposed to a variety of film music probably have developed film music grammar literacy, so it is obvious to them when music sounds like a Western, a sci fi scene, a romance, a mystery, a spy thriller, a horror scene, or an adventure. Carolyn Fortuna stated, "the audience instinctively understands the feelings the filmmaker wants to evoke with a certain style of music. According to German film composer Robin Hoffmann,
"Practically anybody in the western world has probably been exposed to some degree to 'media music and its vocabulary. You don't need to be 'well educated' in film music to understand the basic narrative of film music, and even if you don't, you will not have a ‘wrong’ experience but a less deep one" (email message to author, October 27, 2022).
It wasn’t that the listeners of my stories didn’t hear what the music was doing; they didn’t need recognize specifically what it was doing in the story--in the same way that I did--to enjoy the listening experience. The fact that I could hear the details of the music and connect them to the words is a skill a producer or composer needs to have in order to create effective audio stories with music. The listener doesn't need these skills to enjoy the story.
"As media composers we usually try to target the 'general audience' and accept that there are musically under- or even over-educated people in the audience that we can't properly target. Again, the general audience (in the western world) has a more or less uniform vocabulary that they understand so our job is to trigger this in the right context, independently of visuals or not" (Robin Hoffmann, email message to author, October 27, 2022).
My ability to hear this connection enabled me to explore and develop the use of production music in my audio stories.
Film Production Music
The type of film music I used may also explain why it would sync with the stories so perfectly. Production music is non-exclusive stock music (or library music) that has been composed for use in movies, TV shows, commercials, video games, radio and TV ads, radio dramas, theatrical performances, podcasts, multimedia projects, training materials, and digital storytelling--basically any commercial, business, educational or personal use. It is available through various types of licensing, which can be expensive. It has all types of genres and is usually available in different versions from the original length such as 10- or 30-second spots that are used for commercials. It is different from normal orchestral music because its purpose is to help tell the story—whether that is to sell a product in a commercial or to evoke emotions in a TV or movie scene. The varying lengths make it easier to find music that matched the length of my story narration during development.
Flash Fiction
Another reason why my stories worked with the production music is because they were flash fiction stories. Flash fiction used to be called "short, short stories," but the genre has evolved to fit the reading preferences of the digital age: instant gratification. Typically, flash fiction is between 75 and 1,000 words. However, flash fiction is not just a story with fewer words. It has unique literary qualities. Richard Thomas compares it to the cooking term "reduction, which means to reduce a sauce, from something of a greater quantity, usually more liquid, down to a thicker, more intense flavor." Just like reducing a sauce, flash fiction takes all of the elements of a short story and reduces them down into dense, poetry-like prose. Stories have one main character, dense poetry-like prose, vivid scenes, middle-starting plots, character-revealing actions, and off-page endings. According to Robert Elliot, "Often narrative music will be linked together to make a single, long piece of music with many different moods, one mood after another," so this musical structure works well with flash fiction story structure.
Film Music Whisperer
In 2017, my process changed. I had run out of previously-written stories, so I had to write new stories. However, one day, I was listening to a production music piece that really grabbed me emotionally, and suddenly I saw the story in my imagination. The music was written to tell a story, and I “heard” the story. I listened to it over and over as I wrote the story down to make sure that I matched the words to the music. My ability to recognize how the music fit my previous stories now enabled me to decipher the music and its story. I guess I became a film music "whisperer."
This not only happened with my stories, but it happened with someone else’s story as well. In summer of 2021, I was listening to a film music piece by Simon Author Rhodes called "Romanza." I initially felt that the music sounded like flying in a plane, but I soon realized that it felt more like sailing in a boat. When I began to search for information about sailing, so I could write a story about it, I found Professor John D. Norton's story "Why We Sail" on his website. I decided to do a draft recording to see if the story would fit. With a bit of editing, it fit perfectly. I emailed Professor Norton to see if he would be willing to let me use his story for the audio story, and he agreed. He also agreed to do an author interview about the story. This resulted in the first "Guest Author" s2m story called "Why We Sail." It also launched the "Author Interviews" series for stories and poems that are written by other authors.
The Power of Sound Effects
Another aspect of my audio stories is the use of sound effects. Audiobook publishers were moving from narration-only audiobooks to audio dramas that used voice actors, music and Foley sound effects (like the old radio shows of the 1930s and 1940s), and there was a growing renaissance of radio dramas being produced as podcasts. Therefore, I began to experiment more with sound effects. Dr. Emma Rodero, Professor of Communications at Pompeu Fabra University, said:
"The main function of sound effects in a fictional story . . . is precisely to create an audio reconstruction of reality, imitating reality’s actual sounds so as to create in the listener’s mind a specific image of the phenomenon that it is intended to represent."
I learned that the correct choice of sound effects and the timing of where they occur in the story are crucial to creating sounds that enhance the stories but do not distract from them.
A Tiny Research Study
In fall 2017, I took the History of Multimedia class at Palomar College and did a small research survey for my final project: Media Grammar and stories2music. The purpose was to survey how film music and sound effects affected the listener's imagination. It did not have a large sample size, but there was evidence that music and sound effects changed the listeners' perception and emotional reaction to the story.
My study participants had to listen to three versions of my story "Aurora’s Secret." They listened to the story narration without music or sound effects, then with the narration and music only, and then with the narration, music, and sound effects.
When my participants listened to the story without music or sound effects, they described the scenes very specifically on the questionnaires. It was clear that the words did generate similar images in all three participants’ imaginations. The words did create emotions in some cases.
However, when the music was added, it changed their original imagery. They described the same scenes differently. The emotion was deeper; one participant nearly cried. The music told the participants what to feel. The music changed their imagination.
Then the sound effects also changed the original imaginative images. One participant said the mud falling on the casket sounded heavier than she had originally imagined, and she reasoned that the rain made the dirt muddy and heavier, something she did not imagine originally. One participant had a stronger sense that the grieving woman was leaving in the carriage because of the carriage sound effect. That same participant said that he somehow missed the idea of "fierce rain" in the first version, but the sound effects of the rain brought that to his attention, so his view of the scene changed.
A New Genre
Twelve years and 25 audio stories later, my stories2music project has evolved into a new genre of audio story—flash fiction stories with film music and sound effects that are written to music. These audio stories are a unique genre because they are original flash fiction audio stories. They are innovative because they show the power of film music to tell a story. They are captivating because the music and sound effects, like radio dramas, make the stories immersive. They provide a rich emotional and imaginative experience as well as instant gratification. The stories are bite-sized morsels of mystery, romance, adventure, and drama—all in 2 to 5 minutes. There is a momentary flash of action, of emotion, of experience, of wondering . . . and then it’s over—leaving the listener intrigued and wanting more.