Whale Songs

Whale Songs is a 2D hand-drawn sound puzzle game. Player and her whale friend travel through different fantasy worlds, changing volume and pitch to light up scenes and reveal hidden architectures, search for lost notes and learn to sing whale songs in response to her whale friend, and identify her location to rescue.

Inspiration

From a young age, I was deeply fascinated by the vast mysteries of the ocean. My love for the sea is reflected in my daily life—I have a small saltwater aquarium on my desk, home to a variety of colorful fish and uniquely shaped corals. This passion for the ocean inspired me to create a game themed around the marine world. Ultimately, I collaborated with a friend who shares a similar fascination with the ocean's wonders, and together we designed and developed this game, a fusion of our shared love for the sea.

Inspired by the news that a humpback whale got lost and stranded due to ocean noise, we wanna draw parallels to humans drowning in today's information overload.

Just as the whale is disoriented by the overwhelming noise, humans are often swamped with shallow buzzwords and emoji, making genuine communication harder. In this post-pandemic world, humans seek meaningful connections with friends and nature, like the relationship between the Metkayina and Tulkun in film Avatar2

Therefore, we hope to help reforge the bond between individuals and the world through the power of music, just as the lyrics of musical, 'Though sometimes you feel forgotten in the middle of nowhere, you will be found. 

Research

In the blue depths of the ocean, sound plays a vital role in the survival of whales. Whales rely on their highly developed sonar systems for long-distance communication, which is central to their social lives, as well as for locating food, navigating, maintaining social bonds, and migrating between breeding and habitat areas. However, continuous ocean noise pollution poses a serious threat to whales. This noise, stemming from ship activity, marine construction, and other sources, disrupts the whales' natural communication and sonar navigation, impacting their hunting efficiency, ability to avoid predators, and communication with mates. In extreme cases, noise pollution can even lead to whales being stranded, posing a risk to their lives. Therefore, protecting these marine giants from the impacts of noise pollution is an essential aspect.

Social media noise refers to the plethora of information and interactions on social media platforms. This constant and chaotic stream of information continuously diverts people's attention, undermining their ability to establish the meaningful connections and conversations they truly desire or need. The prevalence of obscure vocabulary and expressions in such environments further exacerbates misunderstandings about English and its true meanings. More critically, this situation promotes and perpetuates superficial thinking, impacting the quality of communication and depth of thought.

Player Companion

In the game, players take control of a young girl, accompanied by a whale as their in-game companion. This design, featuring a player companion, has been widely employed in the gaming industry and has been a core element in the creation of renowned works such as "The Last Of Us," "Titanfall 2," "God of War," and more. Player companions not only contribute to the narrative by adding vibrant colors to the story but also enhance the overall gaming experience.

They assist players in swiftly immersing themselves in the character and provide clear goals and motivation from the outset of the game. Moreover, player companions serve as an integral part of the game mechanics, enriching gameplay and elevating the gaming experience. Therefore, our game introduces a whale as the player's companion, seamlessly integrating it into puzzle-solving designs or game mechanics.

Storyboard & Flow

The game's narrative follows a storytelling approach akin to the "hero's journey," vividly portraying the touching story of the girl and the whale, including their initial encounter, mutual understanding, separation, and eventual reunion.

Game Play

The core gameplay of our game revolves around two dimensions of collected sound: volume and pitch design. In the forest levels, players are required to increase the volume of sound to a certain level to make flowers bloom and collect the musical notes hidden within them. After collecting the notes, players must mimic the calls of their whale companion to locate its position, involving judgments based on pitch and rhythm.

 

In the ruins levels, players also interact with platforms using sound. Specifically, the system assesses the volume of the player's sound to make certain platforms materialize and allows players to control the lateral movement of platforms by adjusting the volume. Additionally, the pitch of the sound determines the vertical movement of the platforms, offering a unique and engaging gameplay experience.

Work Flow

Since this is a collaborative project involving two people, after we jointly finalized the specific game design and established the requirements, I began the process with gray-box testing to validate the feasibility and playability of the game. During this process, I communicated with the art team to prioritize the requirements for art assets and gradually replaced placeholder assets with the final visuals, step by step, to achieve the ultimate game presentation.

Volume Level Calculation:

1. Retrieve Output Data:

   Use the `_audioSource.GetOutputData(_outputSamples, 0);` method to obtain the current output samples from the audio source, and store them in the `_outputSamples` array.

 

2. Calculate the Sum of Squares of Samples:

   Employ a `for` loop to iterate through the `_outputSamples` array, square each sample, and accumulate the results in a variable named `sum`.

 

3. Calculate the Root Mean Square (RMS) Value:

   Calculate the RMS value with the expression `_rmsVal = Mathf.Sqrt(sum / MicDecodeSettings.audioSamples);`. Firstly, divide the accumulated sum of squared values by the total number of samples, `MicDecodeSettings.audioSamples`, to obtain the mean squared value. Then, take the square root of this value to obtain the final RMS value, which is stored in the variable `_rmsVal`.

 

The RMS value serves as a measure of the energy in an audio signal and is commonly used to describe the loudness of a signal. This method provides a smooth and stable representation of the audio signal's intensity, which is crucial for sound processing and analysis.

Sound Comparison:

Once we have obtained the loudness and frequency of the captured sound, we can perform sound comparison to determine if the player's vocalization matches the standard sound. Initially, I used a method where I directly compared the frequency of the captured sound with the frequency of the standard sound within a certain tolerance range and then output the result. This approach worked fine during my initial personal testing, but problems arose when I had others test it. I had overlooked the fundamental differences in baseline sound frequencies due to variations in individual vocal tones, which are even more pronounced between males and females. Therefore, I had to improve my algorithm.

 

Later, I adopted an algorithm based on slope changes for evaluation. This involved discretely calculating the slopes between fixed time intervals, applying some smoothing, and then comparing these slopes with the expected frequency changes in the standard sound wave. Subsequent testing showed that this algorithm successfully addressed the issue by accounting for individual variations in vocal pitch.