In one of my earlier posts I discussed some of the issues (or limitations) with HTML5 audio support in iOS and Android mobile and tablet platforms. In this article I am going to try to take a look at some of the ways we can overcome those limitations and quirks.
Swap Out Audio Source
We discussed the problem of single audio stream limitation where you can play from multiple sources. This can be fixed by swapping out the current audio source for another when required:
What's the one thing every developer wants? More screens! Enhance your coding experience with an external monitor to increase screen real estate.
var audio = document.getElementById('music'); audio.play(); // Sometime Later audio.src = 'music_2.mp3'; audio.play();
Note: There will be a slight delay initially while the new audio file loads! Hence this ain’t a clean solution really, but a hack. There’s a more appropriate hack to fix this initial delay problem along with the limitation called Audio Sprites.
Audio Sprites
You may already know about CSS Sprites where you combine all your images into a single image and then use them with proper pre-defined background positions for various portions (backgrounds for html elements) on your webpage. Similarly you can combine all your audio files into a single sprite with a 1 second gap/pause after each part. You’ll need to make a note of the start and end times of each part (or just the start time and the length) in a JS object and then play them as required.
When the time that equals to the length of the track has elapsed, just pause the audio. It requires a simple logic, either use a setTimeout
or add a timeupdate
event handler.
// tracks object var tracks = { normal_jump: { from: 0, to: 0.4 }, super_jump: { from: 1, to: 2.2 }, space_belt: { from: 3, to: 6.6 }, enemy: { from: 7, to: 9 }, game_over: { from: 10, to: 12.1 } }; // name is the audio part name var name = 'normal_jump'; // following code can be inside a function var track = tracks[name]; audio_sprite.currentTime = track.from; audio_sprite.play(); setTimeout(function() { audio_sprite.pause(); }, (track.to * 1000) - (track.from * 1000)); // or use timeupdate var handler = function() { if (this.currentTime >= name.to) this.pause(); }; audio_sprite.addEventListener('timeupdate', handler, false);
Again, this ain’t clean and you still cannot play multiple sounds or multiple parts simultaneously which is a common requirement in games. For example, at a particular point in time you may need to produce the sounds for player movements, enemies shooting and the player grabbing a powerup, at the same time.
Autoplay Issue
There’s no fix to this issue, not even on iOS 6. You have to wait for user interaction (touch, click, etc. events) in order to start playing audio. Maybe you can present some options on the game splash screen where the user can select (with a click or tap) whether to start playing audio or not.
Trying out AppCache
Decided to give HTML5 AppCache a shot to try fix audio delay issues on ios and android.
On iOS 5 and 6 (chrome) and Jelly Bean (stock browser and chrome) there are delays initially when you play an audio file. But with appcache the issue is completely eradicated. So awesome! Although, when you load an audio for the first time in iOS Safari, the initial delay will be noticed. Hence, the problem is not entirely solved. Infact sometimes the audio doesn’t start playing at all, no matter how many times you hit the play button (probably because it’s not downloaded or something similar) which won’t lead to save anything in appcache. So until the audio is downloaded entirely, appcache won’t be triggered, which means subsequent requests will load resources over the wire (not from appcache).
Android has a weird issue in this case, when wifi is disconnected, play control will switch to pause control (on play) but no audio will be produced and sometimes there’s no progress bar.
Conclusion: AppCache is not a viable solution to this problem yet. It doesn’t help with games when you are playing multiple audio files via JS (due to reasons mentioned here). It does fix one problem though, which is audios do not delay anymore when you add the app to home in iOS Safari and launch from home screen which used to happen before for the first time as audio doesn’t downloads on page load.
Web Audio API
This is by far the neatest way but then has its cons. It’s neither supported in iOS 5 and below nor in the android stock and chrome browsers. This means we can atleast use it to fix audio support in iOS 6. I felt like writing an entire new post on it, so feel free to read here.
Conclusion
As far as HTML5 audio support goes, I’ve pretty much given up on iOS 5 and android browsers for my games. I’ve resorted to using tools like CocoonJS to convert my HTML5 games to native iOS and Android apps that not only fixes audio limitations but also allows me to put my app on Appstore and Google Play! Tools like Cocoon fixes audio limitations and boosts the performance of your games massively by utilizing the native capabilities of the device – hardware accelerated canvas and audio support.
Great blog post. I’ve been having similar issues – I had the sound for my game running perfectly using WebAudio in iOS6 only to find the performance of the actual game to be terrible once I Phonegapped it with the UIWebView. I gave up on iOS and settled on Android instead only to find that it didn’t support WebAudio, and ironically, while googling different sound solutions for Android I came across Cocoon. I think I’m going to have to go down this route too. It still shocks me that in 2013 we have incredible looking 3D games running natively on smartphones but basic javascript 2D apps are having problems like this.
CocoonJS is a good solution if you’ve built your game purely in Canvas, i.e., no DOM. If there are DOM elements then you might face issues during the compilation or even the testing process/phase. This happened with me and I had to give up on cocoon too. Obviously I’ve reported the bugs to them, but I doubt they’ve been fixed yet. All these happened around feb-mar this year.
I’m currently looking into the autoplay issue myself and I just came across an almost-solution that won’t help me but might help most ppl. The original post is here: http://blog.foolip.org/2014/02/10/media-playback-restrictions-in-blink/
Basically you would wait for ANY user interaction at all, which would give you permission to autoplay audio. Then you can proceed to play any other audio file you’d like without restriction. I haven’t tested it out so I wouldn’t know exactly how to pass along that permission, but I thought I’d drop it here to help any one looking for a potential solution.