webaudio.monkey

Monkey Targets Forums/HTML5/webaudio.monkey

skid	(Posted 2012) [#1]

In an effort to induce Sunday night RSI, I have begun translation of the IDL for webaudio which is a draft HTML5 standard for audio.

Note, the object currently known as webkitAudioContext may need to be changed for browsers other than Chrome.



' monkey webaudio interface
' Based on August 2012 draft interface www.w3.org/TR/webaudio/ 

#If TARGET<>"html5"
#Error "webaudio is only available with html5 target"
#End

Function Main()
	Local context:=New webkitAudioContext()
	Print "samplerate="+context.sampleRate
	Print "currentTime="+context.currentTime
End

Extern

Class AudioNode
End


Class webkitAudioContext

	Field destination:AudioDestinationNode
	Field sampleRate#
	Field currentTime#
	Field listener:AudioListener
	Field activeSourceCount
	
	Method createBuffer:AudioBuffer( numberOfChannels, length, sampleRate) 'raises(DOMException)
	
	Method createBuffer:AudioBuffer( buffer:ArrayBuffer, mixToMono?) 'raises(DOMException)
	
	Method decodeAudioData( audioData:ArrayBuffer, successCallback:AudioBufferCallback, errorCallback:AudioBufferCallback ) 'raises(DOMException)
	
	Method createBufferSource:AudioBufferSourceNode()
	
	Method createMediaElementSource:MediaElementAudioSourceNode( mediaElement:HTMLMediaElement ) 'raises(DOMException)
	
	Method createMediaStreamSource:MediaStreamAudioSourceNode( mediaStream:MediaStream ) 'raises(DOMException)
	
	Method createJavaScriptNode:JavaScriptAudioNode(bufferSize, numberOfInputChannels = 2, numberOfOutputChannels = 2) 'raises(DOMException)
	
	Method createAnalyser:RealtimeAnalyserNode()

	Method createGainNode:AudioGainNode()

	Method createDelayNode:DelayNode(maxDelayTime#)

	Method createBiquadFilter:BiquadFilterNode()

	Method createPanner:AudioPannerNode()

	Method createConvolver:ConvolverNode()
	
	Method createChannelSplitter:AudioChannelSplitter(numberOfOutputs = 6) 'raises(DOMException)
	
	Method createChannelMerger:AudioChannelMerger(numberOfInputs = 6) 'raises(DOMException)
	
	Method createDynamicsCompressor:DynamicsCompressorNode()
	
	Method createOscillator:Oscillator()
	
	Method createWaveTable:WaveTable(real#[],imag#[]) 'raises(DOMException)

End

Class AudioSourceNode Extends AudioNode
End

Class AudioDestinationNode Extends AudioNode

	Field maxNumberOfChannels
	Field numberOfChannels

End

Class AudioParam

	Field value#
	Field minValue#
	Field maxValue#
	Field defaultValue#

' Parameter automation

	Method setValueAtTime(value#, time#)
	Method linearRampToValueAtTime(value#, time#)
	Method exponentialRampToValueAtTime(value#, time#)

' Exponentially approach the target value with a rate having the given time constant. 

	Method setTargetValueAtTime(targetValue#, time#, timeConstant#)

' Sets an array of arbitrary parameter values starting at time for the given duration. 
' The number of values will be scaled to fit into the desired duration. 

	Method setValueCurveAtTime(values#[], time#, duration#)

' Cancels all scheduled parameter changes with times greater than or equal to startTime. 

	Method cancelScheduledValues(startTime#)

End


Class AudioGain Extends AudioParam

End

Class AudioGainNode Extends AudioNode

	Field gain:AudioGain

End

Class DelayNode Extends AudioNode

	Field delayTime:AudioParam

End

Class AudioBuffer

	Field sampleRate#
	Field length
' in seconds
	Field duration#
	Field numberOfChannels

	Method getChannelData#[](channel)

End


Class AudioBufferSourceNode Extends AudioSourceNode

'	Const UNSCHEDULED_STATE = 0
'	Const SCHEDULED_STATE = 1
'	Const PLAYING_STATE = 2
'	Const FINISHED_STATE = 3

	Field playbackState

' Playback this in-memory audio asset 
' Many sources can share the same buffer 

	Field buffer:AudioBuffer

	Field playbackRate:AudioParam
	Field loop?

	Method noteOn(when#)
	Method noteGrainOn(when#, grainOffset#, grainDuration#)
	Method noteOff(when#)

End


Class MediaElementAudioSourceNode Extends AudioSourceNode
End

Class JavaScriptAudioNode Extends AudioNode

	Field onaudioprocess:EventListener
	Field bufferSize

End

Class AudioProcessingEvent Extends Event

	Field node:JavaScriptAudioNode
	Field playbackTime#
	Field inputBuffer:AudioBuffer
	Field outputBuffer:AudioBuffer 
	
End


Class AudioPannerNode Extends AudioNode

' Panning model  
'	Const EQUALPOWER = 0
'	Const HRTF = 1
'	Const SOUNDFIELD = 2

' Distance model 
'	Const LINEAR_DISTANCE = 0
'	Const INVERSE_DISTANCE = 1
'	Const EXPONENTIAL_DISTANCE = 2

' Default for stereo is HRTF 

	Field panningModel

' Uses a 3D cartesian coordinate system 

	Method setPosition(x#, y#, z#)
	Method setOrientation(x#, y#, z#)
	Method setVelocity(x#, y#, z#)

' Distance model and Fields  

	Field distanceModel
	Field refDistance#
	Field maxDistance#
	Field rolloffFactor#

' Directional sound cone  

	Field coneInnerAngle#
	Field coneOuterAngle#
	Field coneOuterGain#

' Dynamically calculated gain values  

	Field coneGain:AudioGain 
	Field distanceGain:AudioGain 

End

Class AudioListener
' same as OpenAL (default 1)  
	Field dopplerFactor#
' in meters / second (default 343.3)  
	Field speedOfSound#
' Uses a 3D cartesian coordinate system 
	Method setPosition(x#, y#, z#)
	Method setOrientation(x#, y#, z#, xUp#, yUp#, zUp#)
	Method setVelocity(x#, y#, z#)
End

Class ConvolverNode Extends AudioNode
	Field buffer:AudioBuffer
	Field normalize?
End

impixi

(Posted 2012) [#2]

Cool. I attempted this a while ago and hit a few barriers relating to the loading of sounds via asynchronous XMLHttpRequests, specifically local file permission issues. But I had coded a native Javascript-based loader and wrapped that. Maybe Monkey can handle this itself now with the new streaming features?

I see you've not yet implemented an Oscillator class/interface. I actually found that quite useful for testing purposes, since I couldn't load any sounds from files!

skid	(Posted 2012) [#3]

I have xna audio and microphone channels streaming in / out of Monkey app so am hoping to replicate the interface on other targets.

I might do flash implementation first before going any further with above as the lack of microphone stream on web is a bit of deal breaker.

This magic xylophone is pretty cool.

impixi

(Posted 2012) [#4]

I just dug up my old wrapper code - it's based on the 2010 draft, and that lacked an Oscillator interface. So my memory is crap in that regard! I was using a custom Sinewave generator class I found on the net for testing purposes.

Yeah, there are some nice demos for the web audio stuff. But, as you mention, there are some limitations, though it is quite powerful in other areas.