What is a “木鱼” ?

“木鱼” (Muyu), also known as a wooden fish, is a traditional Chinese percussion instrument commonly used in East Asian Buddhist practices. The name literally translates to “wooden fish.” Here are some key aspects of the Muyu:

Muyu is often carved from wood and shaped like a fish. It has a wooden body and is hollow inside, with slits on the top to amplify the sound. The fish shape is significant; it symbolizes wakefulness, as fish are said never to close their eyes, which is a metaphor for Buddhist monks’ duty to be always vigilant.

Muyu holds a significant place in Buddhist rituals and practices, especially in Mahayana Buddhism, and is a respected symbol of the Buddhist teachings on mindfulness and vigilance.

Final results and lessons learned from our collaborative VR games

Issues I noticed:

  • a slight lag between the visual elements and the corresponding sound effects in our VR game.
  • Some interactions were confusing or unintuitive, resulting in a slight disruption to the overall flow of the game.
  • The balance of difficulty levels in the game. Some sections proved to be too challenging, leading to player frustration, while other areas were relatively easy and could lead to player boredom.
  • certain imperfections in the cohesiveness of the story elements.

Overall, the imperfections we encountered in creating our collaborative VR game did not diminish the significance of our achievement. These imperfections point the way to optimisation and provide us with the opportunity to continue to improve.

Oculus asset managment + efficiency & performance optimisation

  • Performance optimisation goals
    1. Profiling and optimising rendering performance
    2. Efficient use of system resources (CPU and GPU)
    3. Minimising latency and frame loss
  • Asset Management
    1. Texture compression and optimisation
    2. Level of detail (LOD)
    3. Asset caching and streaming – reduce load times
  • Debugging
    1. Analysis of frame timing, GPU utilisation and CPU spikes
    2. Memory management and detection of memory leaks
  • Workflow optimisation
    1. Collaboration tools and techniques for efficient team communication
    2. Version control best practices to ensure seamless iteration and code management
    3. Automation and scripting to increase productivity and reduce repetitive tasks

Challenges while collaboration

Challenge 1: Communication and language barriers
One of the initial difficulties we encountered was effective communication between team members. Sound art and VR have different backgrounds and terminology, sometimes conveying unclear messages to each other, and there were some communication effectiveness issues on both sides.

Challenge 2: Synchronising sound design and visual elements
Achieving seamless synchronisation between the sound design and visual elements was a major challenge during the collaboration. We needed to ensure that the sound effects were fully aligned with the events and visual cues in the game. This required close coordination with the VR team, as any discrepancies would break immersion and affect the overall experience.

Challenge 3: Technical limitations and compatibility issues
Using different software, tools and game engines presented technical challenges and compatibility issues. the VR team primarily used Unity for game development, while the sound art team used a Digital Audio Workstation (DAW) for sound design. Cross-platform imports are often not as smooth. This can present challenges in terms of implementing sound effects, optimising asset performance and enabling real-time audio rendering in a VR environment.

Challenge 4: Time management and co-ordination
Each team, with its own workflow, other concurrent tasks, different milestones and deadlines, requires careful coordination and effective time management.

Project Timeline

  1. Phase 1: Conceptualisation and pre-production
    • Brainstorming sessions to explore phobia themes and narrative ideas.
    • Define the scope and objectives of the project.
    • Establish roles and responsibilities.
  2. Phase 2: Design and development
    • Sound Art:
      • Create a library of phobia-related sound effects.
      • Experimentation with spatial audio techniques to enhance immersion.
      • Iterate and refine sound effects whenever possible based on feedback from the VR team.
    • VR Development:
      • Creating game environments using Unity and VR development tools.
      • Synchronisation of audio and visual elements.
      • Implementing interactive game mechanics and phobia related challenges.
    • 3D Asset Creation:
      • Modelling and texturing objects and environments related to phobias.
      • Optimising the performance and visual fidelity of 3D assets in the game engine.
      • Ensuring consistency between visual and audio elements.
  3. Phase 3: Integration and testing
    • Integrate sound effects, spatial audio and 3D assets into the game environment.
    • Fine-tune the audio-visual synchronisation and immersive player experience.
    • Conduct playtesting and gather feedback for further refinement.

The sound of Untitled Goose Game

https://www.fmod.com/blog/untitled-goose-game-interview

The role of dynamic audio systems and the use of FMOD Studio to implement interactive sound elements: using FMOD’s layering and parameter control features to create a real-time responsive audio experience that enhances the player’s engagement with the game world.

Field recordings: geese honking to rural ambient sounds, ambient sounds and more.

The goose uses a two-way interactive audio.

Multidimensional?

In the field of collaborative projects, the fusion of different disciplines often leads to unique and innovative results. As a student of sound art with skills in 3D asset creation, I had the opportunity to embark on a collaborative journey with VR students to explore the seamless integration of sound and vision in virtual reality. In this blog post, I will delve into my dual role as a sound art enthusiast and 3D asset creator, illuminating the interplay and rich collaborative process between these disciplines.

-Considering visual elements when designing sound effects:
Being involved in the creation of 3D assets has allowed me to approach sound design with a unique perspective. I consider the visual aspects of the virtual reality experience, such as the spatial arrangement of objects or the movement of characters, and produce sound effects that blend seamlessly with these visual elements. This holistic approach resulted in a more immersive and synchronised audio-visual experience for the user.

-Considering sound in the creation of 3D assets:
When creating 3D assets and models, I considered the possibility of sound interaction in these virtual spaces. I designed interactive elements that trigger specific audio cues to create visual representations of the objects that produce sound, and this integration allowed me to enhance my understanding of the impact of sound when creating 3D assets.