Listen with openAL

BlitzMax Forums/BlitzMax Programming/Listen with openAL

Kistjes

(Posted 2008) [#1]

In topic record with openAL we've addressed the option of recording sound with Bmax/OpenAL.
I want to have my BlitzMax program to listen if there is coming a signal from the microphone and if there is start recording as fast as possible.
The recording should last untill there is no more sound for (let's say) 700ms or after 10 seconds.

How can I do that? I hope you have some bright ideas!

jhocking

(Posted 2008) [#2]

I would really love this too. As I posted in that thread I have a project where I'm recording sounds in Blitz. Currently it starts recording when the person presses a button, but having it record automatically would be much better.

Incidentally, there was another thread I posted in shortly after that one where REDi provided code to record without the need for an addon framework:
http://www.blitzmax.com/Community/posts.php?topic=74048#831031

REDi	(Posted 2008) [#3]

My thoughts on this...

You'd need to have openal recording all the time, then check each captured chunk of audio for activity, if you find some then store that chunk to a list, if you dont find activity and there is chunks in the list already then build the sample and clear the list.

Hope that makes sense mate, bit baked ATM lol.

If you have any trouble give me a shout, always happy to help.

Kistjes

(Posted 2008) [#4]

@ REDi:
At the moment I have no idea how to execute most of the steps you suggest. Specially the steps:
- check for activity (?)
- build sample from list (?)

I'll need to dig in the OpenAL commands to find out. Your sample code in the treat jhocking mentioned is a nice starting point, though.
Btw. I wonder how much processor time is consumed by this approach.

@jhocking: thanks for the tip and if you come up with something... let me know.

edit: I think I can solve this! Right now I have other obligations :(
I hope to find some time over the weekend to look at this.

REDi	(Posted 2008) [#5]

- check for activity (?)

cycle through the samples in the audio data and check if sample[i]<>0 (or maybe abs(sample[i])>5 or something to eliminate low level static), you could also get away with a step of 16 or so to reduce cpu usage.

build sample from list (?)

code for that is already in that TOpenALSampler type I posted (in the Stop method).

Good luck mate :)

Kistjes

(Posted 2008) [#6]

About the "check for activity"step...
I would like to follow your advice to cycle through the samples but I don't know how to do that.
I presume in the folowing method the variable samplesbank contains the audio data. Is that correct?

Method CaptureAudioChunk()
	Local recordedsamplelength:Int
	alcGetIntegerv(CaptureDevice,ALC_CAPTURE_SAMPLES,4,Varptr recordedsamplelength) ; OpenALError
	If recordedsamplelength
		Local samplesbank:TBank = CreateBank(recordedsamplelength*BytesPerSample[format])
		alcCaptureSamples(CaptureDevice,BankBuf(samplesbank),recordedsamplelength) ; OpenALError
		ListAddLast(ChunkList,samplesbank)
		Size:+BankSize(samplesbank)
	EndIf
EndMethod

samplesbank is a TBank type. But how do I read the data in this data bank?
Something like:

For Local i:Int = EachIn samplesbank
	'do something like If Abs(i) > 5 then ...
Next

But the local i must be an Object so this will not work.

Kistjes

(Posted 2008) [#7]

I came up with this modification:

For Local i% = 0 To BankSize(samplesbank)-BytesPerSample[format] 'Step 16
	Local sample% = PeekInt(samplesbank, i)
	Print sample
Next

I don't get a compiling error but I have no idea what the values mean. I think I have to do something with the audio format, right?

Kistjes

(Posted 2008) [#8]

Ok. I have made a demo with 8 bits mono recordings.
Eventually I will use high quality mono sound recording.
@REDi: many thanks for you sample code!
Now I have all the puzzle pieces to complete my project.

SuperStrict

EnableOpenALAudio()
SetAudioDriver("OpenAL")
Graphics 500,30,0

Const cIDLE% = 0
Const cLISTEN% = 1
Const cRECORDING% = 2
Const cPLAYBACK_IDLE% = 3
Const cPLAYBACK% = 4


' create the openal sampler
Local Sampler:TOpenALSampler = TOpenALSampler.Create(44100,SF_MONO8) 'SF_STEREO16LE)
Local AudioSample:TAudioSample
Local Sound:TSound 
Local Channel:TChannel
Global mode:Int = cIDLE

Repeat
	Cls
	Select mode
		Case cIDLE ' idle, wait to start recording
			DrawText("Idle... Press SPACE to start listing",10,10)
			If KeyHit(KEY_SPACE)
				AudioSample = Null
				Sampler.Start
				Mode = cLISTEN
			EndIf
		Case cLISTEN 'listen
			DrawText("Listing...",10,10)
		Case cRECORDING	' record
			DrawText("Recording..."+Sampler.Size+" bytes",10,10)
		Case cPLAYBACK_IDLE	' idle, wait to playback recorded sound
			If Audiosample=Null Then AudioSample = Sampler.Stop()
			DrawText("Recorded "+AudioSample.length+" samples - Press SPACE to start playback",10,10)
			If KeyHit(KEY_SPACE)
				Sound = LoadSound(AudioSample)
				Channel = PlaySound(sound)
				Mode = cPLAYBACK
			EndIf
		Case cPLAYBACK	' playing
			DrawText("Playing... Press SPACE to stop playback",10,10)
			If ChannelPlaying(Channel)=False Or KeyHit(KEY_SPACE)
				StopChannel(Channel)
				Mode=cIDLE
			EndIf
	EndSelect
	Flip
Until AppTerminate() Or KeyHit(KEY_ESCAPE)

' ---------------------------------------------------------------------------------
' the sampler!

Type TOpenALSampler

	Field ChunkList:TList = New TList
	Field recordedsamplelength:Int
	Field DefaultSpecifier:String 
	Field CaptureDevice:Int
	Field oalFormat:Int
	Field Format:Int
	Field Hertz:Int
	Field Timer:TTimer
	Field Size:Int
	Field silence:Int
	
	' create a recorder
	Function Create:TOpenALSampler(Hertz:Int=44100,Format:Int=SF_STEREO16LE)
		Local This:TOpenALSampler = New TOpenALSampler 
		This.DefaultSpecifier = String.FromCString(alcGetString(Null, ALC_CAPTURE_DEFAULT_DEVICE_SPECIFIER)) ; This.OpenALError
		This.Hertz = Hertz
		This.Format = Format
		Print BytesPerSample[format]
'		DebugStop()
		Select Format
			Case SF_MONO8		; This.oalFormat = AL_FORMAT_MONO8
			Case SF_MONO16LE 	; This.oalFormat = AL_FORMAT_MONO16
			Case SF_MONO16BE 	; This.oalFormat = AL_FORMAT_MONO16
			Case SF_STEREO8		; This.oalFormat = AL_FORMAT_STEREO8
			Case SF_STEREO16LE 	; This.oalFormat = AL_FORMAT_STEREO16
			Case SF_STEREO16BE 	; This.oalFormat = AL_FORMAT_STEREO16
		EndSelect
		This.CaptureDevice = alcCaptureOpenDevice(This.DefaultSpecifier,Hertz,This.oalFormat,44100*16) ; This.OpenALError
		Return This
	EndFunction
	
	' start recording
	Method Start()
		If Timer Return
		Timer = CreateTimer(60)
		AddHook EmitEventHook,eventhook,Self
		alcCaptureStart(CaptureDevice)	; OpenALError
	EndMethod

	' stop recording and return a TAudioSample
	Method Stop:TAudioSample()
		StopTimer Timer ; Timer = Null
		RemoveHook EmitEventHook,eventhook,Self
		alcCaptureStop(CaptureDevice) ; OpenALError
		CaptureAudioChunk	' flush remaining audio
		Local samplesbank:TBank
		Local AudioSample:TAudioSample = CreateAudioSample(size/BytesPerSample[format],hertz,format)
		Local bptr:Byte Ptr = AudioSample.samples
		For samplesbank=EachIn ChunkList
			MemCopy(bptr,BankBuf(samplesbank),BankSize(samplesbank))
			bptr:+BankSize(samplesbank)
		Next
		ClearList(ChunkList)
		Size=0
		Return AudioSample
	EndMethod

	Function eventhook:Object(id:Int,data:Object,context:Object)
		Local Sampler:TOpenALSampler = TOpenALSampler(context)
		If Sampler
			Local event:TEvent = TEvent(data)
			If Event.ID=EVENT_TIMERTICK And Event.Source=Sampler.Timer
				Sampler.CaptureAudioChunk
			EndIf
		EndIf
		Return data
	EndFunction

	Method CaptureAudioChunk()
		Local recordedsamplelength:Int
		alcGetIntegerv(CaptureDevice,ALC_CAPTURE_SAMPLES,4,Varptr recordedsamplelength) ; OpenALError
		If recordedsamplelength
			Local samplesbank:TBank = CreateBank(recordedsamplelength*BytesPerSample[format])
			alcCaptureSamples(CaptureDevice,BankBuf(samplesbank),recordedsamplelength) ; OpenALError
			If mode = cLISTEN Then
				If isSound(samplesbank) Then 	'a ha! There is someone.
					mode = cRECORDING
					ListAddLast(ChunkList,samplesbank)
					Size:+BankSize(samplesbank) 
				End If
			Else
				If isSound(samplesbank) Then		'there is still noise
					ListAddLast(ChunkList,samplesbank)
					Size:+BankSize(samplesbank) 
					silence = 0						'reset silence counter
				Else
					silence:+1
					If silence < 30 Then 			'less than half a second
						ListAddLast(ChunkList,samplesbank)
						Size:+BankSize(samplesbank) 
					Else	'stop recording
						mode = cPLAYBACK_IDLE
						'I can remove the last 30 silent samplebanks from the list here to reduce sampleSize. I choose to remove 20.
						'size:-BankSize(samplesbank) * 20
						'don't know how to remove the last objects from ChunckList...
					End If
				End If
			End If

		EndIf
	EndMethod

	Method OpenALError()
		Local result:Int = alcGetError(CaptureDevice)
		If result<>0 RuntimeError "Sampler OpenAL Error Code = "+result
	EndMethod
	
	Method isSound%(samplesbank:TBank)
		For Local i% = 0 To BankSize(samplesbank)-BytesPerSample[format] Step 8
			If Abs(PeekByte(samplesbank, i)-128) > 15 Then Return True
		Next
		Return False
	End Method
	
EndType

REDi	(Posted 2008) [#9]

Nice one Kistjes, its coming on a treat :)

This might be of some help...

Function SignedByte:Int(unsignedbyte:Byte)
	Local signed:Int = unsignedbyte
	If signed>127 Then signed:-256
	Return signed
EndFunction

jhocking

(Posted 2008) [#10]

oo I'll have to check this out later, could be just what I need!

REDi	(Posted 2008) [#11]

BTW Kistjes, there is another error in your isSound method... but I wont subtract your fun by giving you the answer :P

Kistjes

(Posted 2008) [#12]

Ok, I'll try your SignedByte function. But after that I'll focus on the SF_MONO16LE format.

Apart from that I don't see the error in the isSound method. It runs through the sampledata (the Step 8 speeds this process a little bit up) and as soon as it finds an audiolevel above a certain threshold (in this method 15) it returns true otherwise false.

REDi	(Posted 2008) [#13]

Cool mate, you'll realize when you do SF_MONO16LE anyway ;)

Kistjes

(Posted 2008) [#14]

Here is my MONO16LE format isSound method:

'mono16le format
Method isSound%(samplesbank:TBank)
	Local sound%
	For Local i% = 0 To BankSize(samplesbank)-BytesPerSample[format] Step 8
		sound = PeekByte(samplesbank, i) | (PeekByte(samplesbank, i+1) Shl 8)
		If Abs(sound - 32768) < 32000 Then Return True
	Next
	Return False
End Method

Tell me REDi, what's wrong with it?
It works fine here! ;-p

REDi	(Posted 2008) [#15]

hmm nearly!

That (byte[i]|byte[i+1] shl 8)-32768 is giving you an incorrect value, you'll need to PeekShort for 16bit audio and then convert it to a signed short (like I did for bytes)

Anyway, did you miss my hint on my original post about the error ;)

Why Minus BytesPerSample from BankSize??? :P

REDi	(Posted 2008) [#16]

Function SignedShort:Int(unsignedshort:Short)
	Local signed:Int = unsignedshort
	If signed=>32768 Then signed:-65536
	Return signed
EndFunction

Kistjes

(Posted 2008) [#17]

That (byte[i]|byte[i+1] shl 8)-32768 is giving you an incorrect value, you'll need to PeekShort for 16bit audio and then convert it to a signed short (like I did for bytes)

Ok thats a good tip. But it seems to work, though.

Ah, you call that an error :p If I leave out the minus part I get an index out of range error because I do a peekbyte(samplesbank, i+1)
With your suggestion of using PeekShort there is no need of doing that anymore, I suppose.

One more question. Now I have my correct recorded sample, how can I save that to an ogg-file?
I can only find a LoadAudioSample() function.

REDi	(Posted 2008) [#18]

Just knocked this up for ya (not tested). I used a short pointer instead of peekshort...

Method isSound16Bit%(samplesbank:TBank)
	Local sptr:Short Ptr = Short Ptr(BankBuf(samplesbank))
	For Local i% = 0 Until BankSize(samplesbank)/BytesPerSample[format] Step 8
		If Abs(SignedShort(sptr[i]))>15 Then Return True
	Next
	Return False
End Method

Notice the divide ;)

As for saving to ogg vorbis There is a SaveOgg command ready to use.

Kistjes

(Posted 2008) [#19]

Ok. My new isSound method for MONO16LE format:

'mono16le format
Method isSound%(samplesbank:TBank)
	Local sound%
	For Local i% = 0 To BankSize(samplesbank) Step 8
		sound = SignedShort(PeekShort(samplesbank, i))
		If Abs(sound) > 500 Then Return True
	Next
	Return False
End Method

Now I only have to figure out how to do the SaveAudioSample()

REDi	(Posted 2008) [#20]

Cool mate, That'll do the job, Nice one! :D

REDi	(Posted 2008) [#21]

Oops my mistake, I thought the oggsaver module was standard but its actualy one of skidracers modules, so if you dont have it already you'll need to synchronize modules and tick the AXE checkbox to get it.

*EDIT*
Just noticed in the last code you posted, you may need to replace that "To" with an "Until" to stop you from reading out of bounds memory.

Kistjes

(Posted 2008) [#22]

I found out I already have the oggsaver module installed. So a simple Import axe.oggsaver line at the top of my demo did the trick.

Many thanks for helping me out.

REDi	(Posted 2008) [#23]

My pleasure Kistjes, good luck with your project mate.

jhocking

(Posted 2008) [#24]

Just tested your code (and made the modifications posted to do 16 bit sound) and it works great. Awesome work man, thanks for sharing!