Real-time audio input

BlitzMax Forums/BlitzMax Programming/Real-time audio input

ryanmsmith

(Posted 2006) [#1]

Is there a way in BliztMax to process audio input in real-time? For example, to display an oscilloscope or other "visualizations" of audio input?

Chris C

(Posted 2006) [#2]

s.armstrong did a portaudio mod, cant remember syncmod command of the top of my head and I'm not at home, but that will do what you want.

I think its axe.portaudio

ryanmsmith

(Posted 2006) [#3]

Eek. I've looked through it and haven't got a clue how to pull in sound data and manipulate it.

-Ryan

Chris C

(Posted 2006) [#4]

he he

Its worth having a good go at its a rather nice lib for realtime stuff...

can you give an example of what you are interested in doing and on what OS?

ryanmsmith

(Posted 2006) [#5]

I want to build a real-time audio analyzer/visualizer that allows me to choose specific frequency ranges to analyze and display. I ultimately want this to be used on stage with my band as a projected display that responds to each of our instruments. It would be like G-Force (http://www.soundspectrum.com/g-force/), but whatever control I need over it.

It would be on XP, for now.

Chris C

(Posted 2006) [#6]

I'm not sure how to analyse a specific frequency range, not somthing I've looked into, if I get chance I'll but together a simple microphone oscilloscope.

Chris C

(Posted 2006) [#7]

hacked this together quick, bit rough and ready but it gives you the idea

dont forget to syncmods -u xxxx -p xxxx axe.portaudio !

Import axe.portaudio


Global astream:Byte Ptr

''''''''''''''''''''''''' stuff missing from port audio
Extern
	Function Pa_GetDefaultOutputDeviceID:Int()
EndExtern

Const paInt8	:Int=32
Const paUInt8	:Int=64
Const paInt16	:Int=2
'''''''''''''''''''''''''' end of missing stuff

Graphics 640,480

Global buf:Float[256]

error( Pa_Initialize() ,"init")

' set the mix buffer running....
playstreamPA(astream,myCallback)	


While Not KeyDown(key_escape)


		Cls
		For Local n:Int=0 To 255

			DrawLine 320,n+100,320+buf[n],n+100
		Next

		Flip


Wend


Pa_CloseStream(astream)

End



Function myCallback:Int( inputBuffer:Float Ptr, outputBuffer:Short Ptr,..
                           framesPerBuffer:Int,outTime:Double, userData:Byte Ptr )
	For Local n:Int=0 To 255
		buf[n]=inputBuffer[0]*128
		inputbuffer:+1
	Next
EndFunction





Function error(e:Int,s:String)
	If e=0 Then Return
	Pa_Terminate()
	Print "error "+e+" in "+s
	Print "message "+String.FromCString(Pa_GetErrorText(e))
	End
EndFunction	


Function playstreamPA(stream:Byte Ptr Var,callback:Byte Ptr)

	error(.. 
	Pa_OpenStream( Varptr stream,Pa_GetDefaultOutputDeviceID(),2,paFloat32,Null,..
						paNoDevice,0,paInt8,Null,44100,256,0,paClipOff,..
			              			Callback,Null )..
	,"open stream")	
	
	error( Pa_StartStream( stream ),"start stream" )
	Print "playing"



EndFunction

ryanmsmith

(Posted 2006) [#8]

Thanks! I was getting close but kept crashing. This will give me a good start!

Chris C

(Posted 2006) [#9]

be careful what you do in the callback, it should not call other functions and should return as quickly as possible

let us know how you get on - drop us a mail by all means

ryanmsmith

(Posted 2006) [#10]

I'm getting this error now:

"Unhandled Exception:GC clrMemBit: membit not set!"

It seems that this occurs quicker the more I have going on the screen. With just plain old bars, it may not crash unless I really crank up the frames per buffer (say, above 256). When I throw in my little particle "engine" with creation, decay and drawing functions, it will crash after maybe 10 seconds. Obviously, I need to limit the number of particles in existance at one time, but is there a fix for this?

Thanks for the help so far!

Chris C

(Posted 2006) [#11]

you're not doing *any* drawing or calculation in the callback are you?

hard to say without seeing any code...

ryanmsmith

(Posted 2006) [#12]

No, I've left the callback function just as you've written it, except for changing the multiplication factor for Buf[n] and using a variable for the for..to limit instead of 255.

All of my calculations, drawing, etc are contained in the main loop. Here's the code, so far:

Import axe.portaudio                    
Include "particles.bmx"              
'Include "explosions.bmx"
'Include "screenflash.bmx"

Global astream:Byte Ptr

''''''''''''''''''''''''' stuff missing from port audio
Extern
	Function Pa_GetDefaultOutputDeviceID:Int()
EndExtern

Const paInt8	:Int=32
Const paUInt8	:Int=64
Const paInt16	:Int=2
'''''''''''''''''''''''''' end of missing stuff

Graphics 1024, 768 
AutoMidHandle True
Global fpb:Int = 8
Global Buf:Float[fpb]
Global gscl:Float = 1
Global imgPart = LoadImage("part64.png")
Global rot:Int = 0

error( Pa_Initialize() ,"init")

' set the mix buffer running....
playstreamPA(astream,myCallback)	

Global avgBuf:Double        

'---Set the seed of the RNG
SeedRnd MilliSecs()          

'---create a timer
CreateTimer 30
gotime = 0
While WaitEvent()                

	Select EventID()                    
	
		Case EVENT_TIMERTICK 
	        avgBuf = 0
			Cls
			For Local n:Int=0 To fpb -1
	            SetColor(0,127,0)          
	            SetRotation 0
	            SetScale 1,1
	            SetAlpha 0.7
	            SetBlend ALPHABLEND                                         
				SetColor(255,0,0)  
				If gotime = 0 Then 
					TParticle.Create(512+Abs(Buf[n])*Cos(n*45+rot), 384+Abs(Buf[n])*Sin(n*45+rot), Rand(2,8)*4, 255,127,127, 0.9, 8, imgPart, Rand(0,3))     
				                                                                                                                                                
					'TParticle.Create(512+(Buf[n]), 52+728*n/fpb, Rand(1,16)*4, 255,127,127, 0.9, 4, imgPart, Rand(0,3))     
				EndIf
				avgBuf:+Abs(Buf[n])
			Next
			avgBuf = avgBuf/fpb
			TParticle.Create(0,0,avgBuf, 63,63, 255, 0.5, 4, imgPart, 1)   
			TParticle.Create(0,768,avgBuf, 63,63, 255, 0.5, 4, imgPart, 1)    
			TParticle.Create(1024,0,avgBuf, 63,63, 255, 0.5, 4, imgPart, 1)    
			TParticle.Create(1024,768,avgBuf, 63,63, 255, 0.5, 4, imgPart, 1)     
			'DrawText String(avgBuf), 0, 100
			'SetColor 0,0,avgBuf
			'SetClsColor 0,0,avgBuf
			TParticle.DrawAll
			Flip   
			TParticle.UpdateAll 
			rot :+ 2    
			gotime:+1
			If gotime = 2 gotime = 0          
			
		Case EVENT_KEYDOWN
			If EventData() = KEY_ESCAPE   
					End	
			End If      
			 
	End Select


Wend


Pa_CloseStream(astream)

End



Function myCallback:Int( inputBuffer:Float Ptr, outputBuffer:Short Ptr,..
                           framesPerBuffer:Int,outTime:Double, userData:Byte Ptr )
	For Local n:Int=0 To fpb-1
		Buf[n]=inputBuffer[0]*1024
		inputbuffer:+1  
	Next
EndFunction

                                                                                                                        


Function error(e:Int,s:String)
	If e=0 Then Return
	Pa_Terminate()
	Print "error "+e+" in "+s
	Print "message "+String.FromCString(Pa_GetErrorText(e))
	End
EndFunction	


Function playstreamPA(Stream:Byte Ptr Var,callback:Byte Ptr)

	error(.. 
	Pa_OpenStream( VarPtr Stream,Pa_GetDefaultOutputDeviceID(),1,paFloat32,Null,..
						paNoDevice,0,paInt8,Null,44100,fpb,0,0,..
			              			Callback,Null )..
	,"open stream")	
	
	error( Pa_StartStream( Stream ),"start stream" )
	Print "playing"



EndFunction

Chris C

(Posted 2006) [#13]

hard to tell because i dont have particles.bmx but frames per buffer is WAY to small I'd say 256 is the *smallest* it wants to be...

splinux

(Posted 2006) [#14]

Chris C: with the code you posted it's drawn the audio spectrum of the input from the microphone?

splinux

(Posted 2006) [#15]

or buf[] simply contains 256 samples distribuited in time?

Chris C

(Posted 2006) [#16]

the last 256 samples collected...

44100hz/256 is approx 3.5 times a frame... however you can't tell 2/3 of the scope is missing each frame beacuse of the speed and persistance of vision

splinux

(Posted 2006) [#17]

So, after have read the buffer the first time, when i read it again, if it was updated, i find all new values?

PS because of the audio input, do anyone of you know how to do FFTs?

taxlerendiosk

(Posted 2006) [#18]

This "Unhandled Exception:GC clrMemBit: membit not set!" is something I came across when I finally got FMOD streams working. As I understand it, basically, BlitzMax's garbage collector means that all streaming audio is impossible because the GC won't allow a separate thread/callback function thing going on for long before throwing a fit and killing your app. It's the same reason why requests for an official thread library are always shot down.

I'd love to be proven wrong.

Chris C

(Posted 2006) [#19]

@denzilquixode what you discribe is a symptom of spending too long in an iterupt callback, streaming audio simultainously works just fine, providing you do it properly...

@SpLinux

when i read it again, if it was updated, i find all new values?

err arent you answering your own question here, if its been updated would you expect to find old values?

taxlerendiosk

(Posted 2006) [#20]

Sorry, how do I know what the maximum length I can spend is?

splinux

(Posted 2006) [#21]

@ Chris C: ok.

i'm trying to program a speech recognition engine for Linux.
this program it's very useful, bacause now i can get live data.

but i have some questions:
1)so every time it's updated buf[] contains an audio sample @ 44000hz long 256 samples, right?
2)i would be very happy if someone would like to program this engine with me, help is welcome.
3)i've asked if someone know hot to code ffts with BMax, so i wrote this question again.

Chris C

(Posted 2006) [#22]

@denzilquixode sorry I dont understand, if you mean max buffer size, I guess as large as you like, but to be honest I dont know, experimentation will soon tell you...

@splinux
1 yes, but it might have been updated several times b4 you get to it in you main code, you might want to implement a ring of buffers, or make the buffer large enough so that it contains more samples than 1 vertical blank
2 email me
3 I've not done this, but I'm sure it cant be too hard

taxlerendiosk

(Posted 2006) [#23]

You said "spending too long in an iterupt callback" is the problem. How do I know what "too long" is and keep below it?

splinux

(Posted 2006) [#24]

i'll contact you via mail.

Chris C

(Posted 2006) [#25]

basically do as little as possible in the callback, if you notice inconsistant behaviour from the debugger or the program quits unexpectidly then you are probably spending to long in the callback -or- you have a dangling pointer

*	(Posted 2006) [#26]

Can someone package the module and source and send it over to ed@... it would be greatly appreciated :). My current machine doesnt have net access, this machine doesnt have max on it.

Nigel Brown

(Posted 2006) [#27]

Fond the problem I had, you must not build with debug enabled or it will crash!