Does a webaudio module exist?

Monkey Targets Forums/HTML5/Does a webaudio module exist?

MikeHart

(Posted 2013) [#1]

Hi folks,

I want to target mobile HTML5 as a platform and I wonder if someone has created a module that addds webaudio to Monkey with a fallback option for the regular Audio elements that Moneky uses by default. IOS6 supports webaudio now and so does the latest Chrome for Android.

Any info is welcome!

Michael

skid	(Posted 2013) [#2]

It's kind of a work in progress:

webaudio.monkey

' monkey webaudio interface
' Based on August 2012 draft interface www.w3.org/TR/webaudio/ 

#If TARGET<>"html5"
#Error "webaudio is only available with html5 target"
#End

Import dom
Import "webaudio.js"

Extern

Class WebGamePad
	Field axes#[]
	Field buttons#[]
	Field id$
	Field index
	Field timestamp
End

Function LoadWebAudio%(context:webkitAudioContext,url$)
Function PlayWebAudio%(context:webkitAudioContext,sound,looping?)
Function StopWebAudio(channel)

Function SetWebAudioRate(channel,rate#)
Function SetWebAudioGain(channel,gain#)

Function JoyCount()
Function GetJoy:WebGamePad(index)

Class AudioNode
End

Class webkitAudioContext

	Field destination:AudioDestinationNode
	Field sampleRate#
	Field currentTime#
	Field listener:AudioListener
	Field activeSourceCount
	
	Method createBuffer:AudioBuffer( numberOfChannels, length, sampleRate) 'raises(DOMException)
	
	Method createBuffer:AudioBuffer( buffer:ArrayBuffer, mixToMono?) 'raises(DOMException)
	
	Method decodeAudioData( audioData:ArrayBuffer, successCallback:AudioBufferCallback, errorCallback:AudioBufferCallback ) 'raises(DOMException)
	
	Method createBufferSource:AudioBufferSourceNode()
	
	Method createMediaElementSource:MediaElementAudioSourceNode( mediaElement:HTMLMediaElement ) 'raises(DOMException)
	
	Method createMediaStreamSource:MediaStreamAudioSourceNode( mediaStream:MediaStream ) 'raises(DOMException)
	
	Method createJavaScriptNode:JavaScriptAudioNode(bufferSize, numberOfInputChannels = 2, numberOfOutputChannels = 2) 'raises(DOMException)
	
	Method createAnalyser:RealtimeAnalyserNode()

	Method createGainNode:AudioGainNode()

	Method createDelayNode:DelayNode(maxDelayTime#)

	Method createBiquadFilter:BiquadFilterNode()

	Method createPanner:AudioPannerNode()

	Method createConvolver:ConvolverNode()
	
	Method createChannelSplitter:AudioChannelSplitter(numberOfOutputs = 6) 'raises(DOMException)
	
	Method createChannelMerger:AudioChannelMerger(numberOfInputs = 6) 'raises(DOMException)
	
	Method createDynamicsCompressor:DynamicsCompressorNode()
	
	Method createOscillator:Oscillator()
	
	Method createWaveTable:WaveTable(real#[],imag#[]) 'raises(DOMException)

End

Class AudioSourceNode Extends AudioNode
End

Class AudioDestinationNode Extends AudioNode

	Field maxNumberOfChannels
	Field numberOfChannels

End

Class AudioParam

	Field value#
	Field minValue#
	Field maxValue#
	Field defaultValue#

' Parameter automation

	Method setValueAtTime(value#, time#)
	Method linearRampToValueAtTime(value#, time#)
	Method exponentialRampToValueAtTime(value#, time#)

' Exponentially approach the target value with a rate having the given time constant. 

	Method setTargetValueAtTime(targetValue#, time#, timeConstant#)

' Sets an array of arbitrary parameter values starting at time for the given duration. 
' The number of values will be scaled to fit into the desired duration. 

	Method setValueCurveAtTime(values#[], time#, duration#)

' Cancels all scheduled parameter changes with times greater than or equal to startTime. 

	Method cancelScheduledValues(startTime#)

End


Class AudioGain Extends AudioParam

End

Class AudioGainNode Extends AudioNode

	Field gain:AudioGain

End

Class DelayNode Extends AudioNode

	Field delayTime:AudioParam

End

Class AudioBuffer

	Field sampleRate#
	Field length
' in seconds
	Field duration#
	Field numberOfChannels

	Method getChannelData#[](channel)

End


Class AudioBufferSourceNode Extends AudioSourceNode

'	Const UNSCHEDULED_STATE = 0
'	Const SCHEDULED_STATE = 1
'	Const PLAYING_STATE = 2
'	Const FINISHED_STATE = 3

	Field playbackState

' Playback this in-memory audio asset 
' Many sources can share the same buffer 

	Field buffer:AudioBuffer

	Field playbackRate:AudioParam
	Field loop?

	Method noteOn(when#)
	Method noteGrainOn(when#, grainOffset#, grainDuration#)
	Method noteOff(when#)

End


Class MediaElementAudioSourceNode Extends AudioSourceNode
End

Class JavaScriptAudioNode Extends AudioNode

	Field onaudioprocess:EventListener
	Field bufferSize

End

Class AudioProcessingEvent Extends dom.Event

	Field node:JavaScriptAudioNode
	Field playbackTime#
	Field inputBuffer:AudioBuffer
	Field outputBuffer:AudioBuffer 
	
End


Class AudioPannerNode Extends AudioNode

' Panning model  
'	Const EQUALPOWER = 0
'	Const HRTF = 1
'	Const SOUNDFIELD = 2

' Distance model 
'	Const LINEAR_DISTANCE = 0
'	Const INVERSE_DISTANCE = 1
'	Const EXPONENTIAL_DISTANCE = 2

' Default for stereo is HRTF 

	Field panningModel

' Uses a 3D cartesian coordinate system 

	Method setPosition(x#, y#, z#)
	Method setOrientation(x#, y#, z#)
	Method setVelocity(x#, y#, z#)

' Distance model and Fields  

	Field distanceModel
	Field refDistance#
	Field maxDistance#
	Field rolloffFactor#

' Directional sound cone  

	Field coneInnerAngle#
	Field coneOuterAngle#
	Field coneOuterGain#

' Dynamically calculated gain values  

	Field coneGain:AudioGain 
	Field distanceGain:AudioGain 

End

Class AudioListener
' same as OpenAL (default 1)  
	Field dopplerFactor#
' in meters / second (default 343.3)  
	Field speedOfSound#
' Uses a 3D cartesian coordinate system 
	Method setPosition(x#, y#, z#)
	Method setOrientation(x#, y#, z#, xUp#, yUp#, zUp#)
	Method setVelocity(x#, y#, z#)
End

Class ConvolverNode Extends AudioNode
	Field buffer:AudioBuffer
	Field normalize?
End

webaudio.js


var gamepadSupportAvailable = !!navigator.webkitGetGamepads || !!navigator.webkitGamepads

var _gamepads
var _pads

function JoyCount(){
	_gamepads=navigator.webkitGetGamepads()
	if(typeof _gamepads === 'undefined') return 0;
	var n=0
	_pads=new Array
	for(i=0;i<_gamepads.length;i++){
		pad=_gamepads[i]
		if (typeof pad === 'undefined') continue;
		_pads[n++]=pad
	}
	return n
}

function GetJoy(index){
	return _pads[index]
}

var soundBuffer=new Object()
var channelBuffer=new Object()

var soundCount=0
var channelCount=0

function LoadWebAudio(context,url) {
	var sound=++soundCount
	var request = new XMLHttpRequest()
	request.open('GET', url, true)
	request.responseType = 'arraybuffer'
	request.onload = function() {
		if(request.status==404){
			return // not found
		}
		context.decodeAudioData(request.response, function(buffer) {
			soundBuffer[sound]=buffer
		}, function onError(){
			soundBuffer[sound]=null
		});}
	request.send();	
	return sound;
}

function PlayWebAudio(context,id,loop){
	var buffer=soundBuffer[id]
	if (buffer==null) return 0
	var source = context.createBufferSource()
	source.buffer = buffer
	source.loop = loop
	source.connect(context.destination)
	source.playbackRate.value=0
	source.gain.value=0
	source.start(0)
	var channel=++channelCount
	channelBuffer[channel]=source
	return channel
}

function StopWebAudio(channel){
	var source = channelBuffer[channel]
	if(source==null) return
	source.stop(0)
	channelBuffer[channel]=null
}

function SetWebAudioRate(channel,rate){
	var source = channelBuffer[channel]
	if(source==null) return
	source.playbackRate.value=rate
}

function SetWebAudioGain(channel,gain){
	var source = channelBuffer[channel]
	if(source==null) return	
	source.gain.value=gain
}

MikeHart

(Posted 2013) [#3]

Thanks man, i will try that.

MikeHart

(Posted 2013) [#4]

Could not get it to play anything. But thanks for the suggestion.

skid	(Posted 2013) [#5]

That's a shame.

nSYnth is the project I'm using the above with at moment. Click Play, then use keyboard to play notes, spacebar to toggle the arpeggiator, mouse position to alter the white noise.

MikeHart

(Posted 2013) [#6]

I am glad that you got it working.

Shockblast

(Posted 2013) [#7]

You can also try using howler.js for the webaudio+fallback to regular audio. I modified the mojo audio module to play sounds and music through howler instead of the regular audio elements.

David Casual Box

(Posted 2013) [#8]

Hi there.

We're trying for days to play a sound with no success.
On Safari with an iPhone, the nsynth sample mentioned in this thread is no playing sounds, neither on Chrome.

We tried 4 differents HTML5 audio Javascript libs with no success. The main issue is that the mobile browser check if the sound is play "during a user event", which is not the case when you play a sound using a simple function call.

We tried to play the sound in a real HTML button (in the onClick), then simulate a click on the button with Monkey, just calling a js function (using Fireevent) with no success. It seems the call stack is not trustable for iOS, and this f... b... is not playing the sound...

Any help or advice will help us a lot.

Shockblast

(Posted 2013) [#9]

@David

Like I said, I modified the mojo audio module to load and play sounds through howler.js

To unlock WebAudio on IOS6-7 you need to first play a sound through an user interaction. I added an extra function, enableWebAudio(), into howler.js to play a silent sound. Download the modded Howler here:

http://www.shockblastapps.com/files/howler.js

So in your Monkey folder, go to your HTML5 target folder and open html5game.js.
Then go to the 'canvas.ontouchstart' function and call Howler.enableWebAudio(); from there.

This should unlock WebAudio on IOS. Don't forget to import the howler.js file into the MonkeyGame.html. Also you now have to load and play your sounds through howler.js because only the AudioContext in there has been unlocked.

I hope this helps!

EDIT: Fixed the file so that it works with FF 25.
- "noteOn" method calls have been replaced with "start" method calls. This is because the "noteOn" method is deprecated.

David Casual Box

(Posted 2013) [#10]

We managed to get sound working on iOS thanks to you Shockblast! Thank you so much, we're quite proud to show this working to our customer!

programmer

(Posted 2013) [#11]

Firefox 25 adds Web Audio API support - http://betanews.com/2013/10/29/firefox-adds-web-audio-api-support-minor-tweaks/

Shockblast

(Posted 2013) [#12]

Yeah I noticed that FF 25 now supports Web Audio. The howler.js file that I was using did not work with FF 25 but that has now been fixed. You can redownload the file in my previous post and it should all be alright.