Percussion Simulator

Our project is related to music, video game technology and virtual reality because the game will allow the player to play percussion in virtual reality.

We have created a virtual reality game that allows the player to play different percussion instruments either accompanied by a variety of custom and existing music or not. People can produce music the way they want, without limit.

To use our game, we must start the game with Unreal engine, version 4.15.1, and then launched the game with virtual reality.

The purpose of our game is to play different percussion instruments without having to spend a lot of money to purchase them. This game, when refined, could be used in school environments to teach percussion to students or in a professional setting to allow for the writing of music and for playing music, for example.

Step 1: Tools, Material and Files

Tools needed for the game production:

-Computer

-VR HTC Vive headset and controlers

Files produced with software and then used in the game creation

3D meshes:

-cowbell.fbx

-cymbal.fbx

-tumpani.fbx

-triangle 2.fbx

-stick.fbx

Musescore music:

-test-1.mscz

-test-2.mscz

-test-3.mscz

Content taken from the internet

-We Are Number One But With The Sherlock Holmes Theme But In Musescore:

https://musescore.com/user/7679561/scores/3664216

-Basic Unreal Engine Assets

-Infinity Blade: Grass Lands assets available from the unreal marketplace:

https://www.unrealengine.com/marketplace/infinity-...

-Various instrument sound files from the Musescore software

We were inspired by the games Band Hero and Guitar Hero and of our past as musicians in the music concentration of the Saint-Anne college.

List of software used:

-Unreal Engine version 4.15.1 / 4.15.2

-MuseScore 2.0

-Blender

-Gimp 2

To create the partition in the software MuseScore 2, we went to the top right in File, and selected New. After completing the information according to our preferences, we selected the armor of the partition and the number of flats or sharps. Finally, the information corresponding to the body of the partition, the number of measurements and the indicator number of the partition was selected. We then selected different types of notes and silences to put them in the score to create our own original pieces of music to be used in the background of our game.

The first step of the game creation process was to create a basic level in Unreal engine 4. We started off by creating a new virtual reality project for desktop/console at maximum quality and with starter content. We then loaded the Motion Controller Map from the Virtual Reality BP content folder and deleted the boxes and walls so we could have a clean slate to work with.

Once we had a clean slate, we started to use assets to create an environment for our game. We textured the ground with a water plain from the Infinity Blade: Grass Lands asset pack. We then used a ruins asset to create an elevated platform in the middle of the water that would be the main player area. Finally we added some cliff sides and mountain assets, also from the Infinity Blade: Grass Lands asset pack, to hide the end of the water plain and make the environment feel more realistic.

We used a 3D modeling software called Blender to create the 3D mesh instruments. We created a timpani drum, a triangle, a cowbell, a crash cymbal and a general percussion stick to hit the instruments with. Each of the meshes was created by first adding a two dimensional picture of the object in the background of the modeling window to serve as a guide for the shape of each object to be created using basic shapes that where modified with the software's variety of editing tools. When each object was completed, we exported them as .fbx files so they could be compatible with Unreal Engine 4.

With the environment made, we added the instruments to the game. The instrument meshes, all custom made in Blender, were places on the ruin platform at a close proximity from the player starting point so that the player can start experimenting and trying the different instruments shortly after starting the game. The cowbell, triangle and cymbals as well as a stick were placed on a table prop from the starter content pack right next to a half-circle formation of 4 timpani drums (the standard amount in any harmony/orchestra) to make shifting from the smaller instruments to the timpani drums easy. The height of the timpani had to be adjusted by compressing the original mesh as they were too tall for the player to reach effectively. On the other hand the table with the other instruments may need to be heightened as it seems too low for players to reach the objects.

Once the basic scene of our game was done we started experimenting with the mechanics. We started by adding gravity and physics to our game world, which proved to be challenging since we were not yet familiar with all of the options of Unreal and we had trouble finding the correct settings. We then added collisions to every object by going into the details settings of each object and setting their complex collisions as simple. We then went into the blueprints of each mesh instrument (minus the timpani) and created a blueprint to make the objects grab-able. Finally, we added a Navigation Mesh to the level to enable the player to teleport next to the instruments, but we also used Nav. Mesh Boundaries to limit of the teleportation so the player would not teleport on top of the instruments. We tried to add a system or blueprint to enable sounds to be played when hitting an instrument, but we the system kept having bugs and imperfections.

During the process of making our game, we encountered multiple problems. We often had corrupted files or parts of the game that didn't save correctly and had to restart the assembly of the game world around 3 times. We had some minor problems with the 3D instrument models and the music, but fixed them quickly. It was hard to find a suitable environment at first. We originally planed on editing a library environment that w had found online, but when we tested it with the Vive, the player controls didn't work. After finally deciding to create our own environment, we once again encountered some problems with the VR hands in the game not responding to the players controllers. We eventually fixed the issue, only to find that objects would stay frozen in the air when let go of and that most collisions and the sound on collision didn't work at all.

The game is still a work in progress and we will continue to fix it's problems and expand upon it. Our next priority is to fix all the collision and physics problems and to add the background music as well as a way to freely change to switch between different background tracks.

After the first hand-in of our project, we started to by trying to fix the things that were broken. Instead of a blueprint tied to the instrument, we used trigger boxes and placed them over the timpani drums so that the sound would play when the stick enters the very specific area to make it look like the sound play when the timpani is hit. We also did used trigger boxes for the cowbell and triangle that we attached to a stand mesh, thus removing their grab-ability and making them stationary. The cymbals used a modified version of sound cues which involves creating a socket in the object details and placing it on the object. We created a round disk shaped socket that we placed on the bottom of each cymbal and made a blueprint to allow them to play a sound when the socket overlaps or collides with another object. We planned to also add a gong, but we were unable to get it finished in time.

Once we finished the basics of the game we decided to add textures to our instruments. We created the textures in Gimp 2 by combining different images from the internet. We then exported the edited picture and put it in the UV mapping screen in Blender. We marked seams on the instruments and UV unwrapped them so they can fit on top of the image and map the textures. We then exported the mesh with the mapping and the texture and replaced each instrument in Unreal with the final textured model. Some textures, like the cymbal one, broke during the transfer, but still looked like a cymbal texture.

Unfortunately, we didn't manage to finish our game for the deadline since our maps from the past 3 weeks were lost or deleted by the computer. Before the level was erased, we tested the collisions and everything worked well. We still have the all the assets to be able to recreate it in the future. We intended the final game to have better floor to fix some minor teleportation glitches, finish the background and add a button to change the background music.

In the future we plan to make more backups to make sure we don't lose any progress.