Developer's Blog #9 - Audio Crisis
by Otmar Nitsche, 2.09.2018
As you all know, sound is a primary key to creating an immersive experience. The Wing42 Lockheed Vega 5 is still lacking this level of immersion and it has always been one of my sore spots. By-passing my to-do list for the next version, I sat down and put some serious time into the development of our own sound engine. In this article I want to outline the various pitfalls of this round of development and also what you can expect in terms of sound.
First of let me clarify a few things regarding sound in the sim. FSX/P3D doesn't have a practical interface into its sound engine which could be utilized by add-on developers. You are essentially limited to a handful of sounds that you can define for the engine (inside & outside), the sound of the doors, gear, flaps and such. This however is utterly useless if you want to hear the mechanical clock ticking, or if you want to make the engine backfire. You can't listen to the spinning up of the various gyros or the feedback by your ground crew. If you want any of those really immersive sounds, you need to ignore the sim's sound engine and either find a suitable solution or program one yourself. And since this is Wing42, I chose the later.
The requirements for our own sound engine were the following:
- Ability to play a simple sound on demand.
- Ability to stream sound files.
- Ability to define channels for mixing and to "queue up" sounds.
- Volume control for the mixing channels.
- Ability to "loop" a sound with functions to start and stop.
- Ability to assign sound loops to levers and similar controls.
- Dynamic pitch control for those looped sounds.
One feature I always wanted the Vega to have, is a responsive crew and with the upcoming update the first trial of this talky crew will be started. Since the functionality of the ground crew interface and payload manager is already complete, I chose that interface as a testing ground and gave your rampie a voice - my voice. But one thing that shouldn't happen with voice is an overlap of sounds, e.g. if you activate two tasks quickly after another, the oral response from your rampie must come one after the other, hence the necessity of channels with queuable sounds. The sound engine will support virtually an unlimited number of such channels.
In addition, I felt the need to create mixing channels as well and the Vega will have three of those. These mixing channels can be used to set the volume for various types of sounds of the add-on.
Looping sounds is quite important for all those repetitive machine-noises. But I went a step further and implemented the ability to dynamically control pitch and volume for those sounds as well. Take for instance the three gyros of the Vega's instrumentation. Depending on the gyro's rotational speed, it will produce a different sound, the faster it turns, the higher the pitch of the gyro will be. So using the same loop-sound for a gyro, I can let it spool up and down by the means of a simple pitch & volume control.
Furthermore, I wanted dynamic sounds for all those noisy levers and lever-type controls as well. Take the windshield wipers as an example to let me explain how this dynamic sound is implemented. When you first click on the windscreen wipers, I start to playback a looped screeching sound, but I set the volume of this sound to 0. As you then drag the lever over the windscreen the software changes volume and pitch of this sound according to the speed at which you operate the control. As a result, you'll hear a higher pitched screech when you wipe fast, and a much lower pitched screech, at lower volume, when you wipe slowly. Since there are a lot of those click & drag type controls in the Wing42 Vega, this functionality of the sound engine was vital.
A nice plan you got there, would be shame if something happened to it...
So I started reading up on Microsoft's SDK ("Software Development Kit", in case you don't know the lingo) for XAudio2 and started to set up the required interfaces. Despite Microsoft's tendency to over-complicate their sample codes, it was all a very straight-forward process and soon enough I had the first working sounds in the aircraft. I continued coding for the next few days, implementing all the mixers, filters, and structures to fulfill the requirements I set above.
Unfortunately nobody told me about the slumbering nightmare that is the ghost of directX! Let me explain:
DirectX is a now depreciated framework that gives developers access to the computer's hardware. It was mainly designed for the video game industry as it allowed fast access to the graphics card for 2d and 3d rendering, controller inputs (such as joysticks) and sound cards. Sound could initially be produced using directSound for general sound effects and directMusic for the playback of music. In later updates those two interfaces were combined into xAudio and then xAudio2. Microsoft kept updating directX until 2010 when they started to merge the directX libraries with the core windows operating system thus eliminating the need for consumers to "update the directX runtime files".
Great! So now xAudio2 is part of the operating system and hence there shouldn't be a need to install directX any more. But here comes the problem, that I obliviously stumbled across: the latest version of xAudio2 installed on Windows 7 machines is version 2.7 and Microsoft won't apply further updates of xAudio2 to this operating system from 2009. This creates a bit of a problem for me, since I promised compatibility of the Lockheed Vega 5 down to Windows 7 (partly because my test-rig still runs on this OS). The issue is that starting from Windows 8, XAudio2 comes in version 2.8 and since my development computers runs Windows 10, all the software development kits are geared towards that version of the OS.
After reading endless pages on the matter I came to the conclusion that the only way to make the XAudio2 module backwards compatible with Windows 7, is to use the out-dated directX libraries from 2010 and those libraries come with further dependencies that need to be part of the next installer to ensure that the module works for everyone. As we speak, I haven't achieved my goal of full Windows 7 compatibility yet, since my test rig still refuses to load the DLL file. It looks as if there are some dependency issues with the Visual Studio runtime files, but I have a few weeks to sort this problem out.
This has been a long journey, and I wish I was a bit further ahead by now. But I also think that the battle is well worth fighting, since the result brings so much more immersion to the sim.
Recently I took a little video to demonstrate the capabilities of the Wing42 sound engine. I cranked up the volume a bit and some of the sounds you hear still need some refinement. But I think it gives you a good idea about what to expect in the next update: