Realtime Synthesis with openAL
BlitzMax Forums/BlitzMax Programming/Realtime Synthesis with openAL
| ||
| YES, maybe I should make this a tutorial, but few of you may have been itching to figure this out and got as frustrated as I used to be about it. But I did plenty of C++ coding in the meantime, learned a fair bit about realtime synthesis and got comfortable enough to have another look at it in Bmax again. That, and the fact that Tomas Petterson (drpetter) has been here for a few days and we've made a bunch of nonsense together. Anyway, here it goes... OpenAL streaming to use for realtime synthesis in BlitzMax... I'll make it a tutorial, if you want me to. But it is fairly simple. A small setup section up there is from a type that <argh, who was that again?> posted here, but I was just too lazy to rewrite those lines. HERE I SHALL POST THE LATEST CODE - ALWAYS: Last edited 2011 Last edited 2011 |
| ||
| That's funny! anyway, you're having an array out of bounds with the "buffers" array. It should be created with buffers[3] in order to allow a "2" as its maximum index. sounds a lot like a theremin. I like it! |
| ||
| AH woops, darned, forgot that I changed it and got sloppy, haha. Thanks! The point of it is really that it's possible. By all the posts here that I ever found it appeared not to be. I'm therefore quite overjoyed, hehe. It's the beginning of some exciting exploration, I'm sure. I figure that some of you may pick up on it and take it to higher places on your own. I can't believe how easy the setup can be, though. But it gets trickier from here, that's for sure! Last edited 2011 |
| ||
| does it supposed to be crackling and intermittently stopping? I am thinking it's the OpenAL functionality on my mac. Last edited 2011 |
| ||
| That is excellent. I bet it could be adapted to work with old school emulators that use the old waveform patterns and then adjust the pitch to get the correct sound. A fair while ago I was trying to get something similar working to work with a z80 emulator with pacman roms that I'd written, but resorted to other methods. This is more like what I wanted to achieve :P I may pursue it in the future.. you never know :P Thanks for sharing :D Last edited 2011 |
| ||
| I had all manner of problems with openal on my mac. went back to freeaudio. |
| ||
| does free audio support the buffer queuing in the same or similar manner? That would be interesting. I also realized a few silly things in my code up there, I'll fix it real quick... Thanks, guys! :) *sorry about the mac issues with openAL. You could try to change the buffersize, see if that reduces the crackling with higher latency (4000 for example) |
| ||
| This is really cool, thanks for sharing it! Say "hi" to drpetter, too -- he's done some great stuff and I really recommend a trawl around his site for anyone that hasn't been. In fact, his sound synthesis article made me want to play with this stuff ages ago, and now I can! |
| ||
| I had all manner of problems with openal on my mac. went back to freeaudio. That doesn't sound good [no pun intended... ok just a little]. I mean, isn't OpenAL the #1 audio API for Mac? Last edited 2011 |
| ||
| Hehe...just added some keys to play along over almost the whole keyboard. It's still crude and lousy code, but I could share it with you, if you like. I can't believe I'm doing this right now, though. It's really amazingly pleasing, hahaha. Thanks, James, I'm happy you feel it, too! :o) And, yes, Tomas rocks! 8) Pahaha, Kryzon, I hope that's not the case. Whereby...hmmm...I do like to accumulate reasons for why I'm happy on the PC. ;) |
| ||
| just added some keys to play along over almost the whole keyboard. It's still crude and lousy code, but I could share it with you, if you like. Over here! I like how short the original code is, too, though haven't tried to 'read' it yet... |
| ||
| I would love a tutorial on this kind of stuff please? |
| ||
| Hahahah! Very cool, gives a great 'vintage sci-fi' feeling :) I kind of understand the 'joy' of doing this - when I first played a sound sample on MSX, by resetting all registers and just changing the volume up and down... I couldn't believe it worked, until I figure it all out :) You could include square waves in there...just for fun :) Last edited 2011 |
| ||
| Ehehe, ok, here's the little vibraphone combination (lazy me, only sines, haha) Alright, I will see that I write a tutorial later today. I think it's good for myself, too, haha. :} And, please, don't judge to harshly on my code, but let me know, if you find something stupid or less favorable and share your optimizations. That would be nice. Thanks in advance! Last edited 2011 Last edited 2011 |
| ||
The party starts, if you add more SIN()-Functions to the formula:buffer_A[i] = vol*(Sin(a)+Sin(a/1.02)+Sin(a*2))*$7fff In nature a "sound" is a combination of a dozend of sinus-components (overtones). Each overtone has the a*2, a*3, a*4, .... value of the base freqency, but it is less loud: Vol*Sin(a) + vol/2*Sin(a*2) + vol/4*Sin(a*3) + ..... Also try two nearly equal frenquenies: Vol*Sin(a) + vol/2*Sin(a*1.02) this sounds like a chorus effect. But the best is "clipping", which you can listen to with the code above. The signal is too big (> (255*255)) an starts again from Zero. This brings a lot of additional overtones to the signal here is a "engine of a car": buffer_A[i] = vol*(Sin(a/2)+Sin(a/3))*$7fff |
| ||
| This is great stuff, and midimaster's tweaks are interesting too -- I like how you can see the point at which the waveform starts clipping and the 'new' effect coming in. (Besides the musical and sound effects possibilities, one thing I'd really like to recreate using this stuff is drpetter's engine sound from this video -- if you click More Info he describes how it's made. One of the best synthesised engines I've ever heard.) I buried this in a link above, but this article is really well worth a read for anyone else interested in this stuff: Basic sound theory and synthesis |
| ||
| Ehehe, nice, yeah. There's no end to the possibilities. But it does get much more complicated, once you go into anti-aliased waveform generation and, of course, happy filtering. Whereby that's not so tough. Just springs, really. Anyway, I've added a tiny pattern recorder and player, but the timing is horrible. I know, it's the oldest of all problems and tons of suggestions around, but I can't ever find the right solution. Does anyone have a solid timing idea for pattern playback? Thanks in advance. |
| ||
| First of - hi all, this is my first post as a noob (7 weeks into blitzmax) there are two things you need for timing 1. use events as your framework 2. use timer events to get your timing for the framework here is the base code: While Not AppQuit WaitEvent() Select EventID() Case EVENT_APPTERMINATE AppQuit = True Case EVENT_TIMERTICK EventTimer() End Select Wend this is very different from normal blitzbasic standard loops as it plays nice with the os :) to set a timer use something like: Global SongTimer:TTimer Global SongBpm:Int = 130 If SongTimer StopTimer(SongTimer) SongTimer = CreateTimer( SongBpm/15.0 ) the main issue you will have is that your program will now wait and not do any background processing. here there are two options: 1. move your audio processing into a thread so it will be independent 2. recode the audio subsystem yep I know that both are not good answers but sorry on those! You will need to ask yourself do i need to do it realtime, or can I pre-process it and play realtime which is a different thing I've got a 32 track sequencer running realtime with no timing issues in blitzmax plus additive synthesis systems based on those from the ppg wave term and fairlight cmi. |
| ||
| there is no need of "record" what you hear. You just can save the buffer directly to a file. Here is a audio recorder listing based on OpenAl. Perhaps you can find some additional information: http://www.blitzbasic.com/Community/posts.php?topic=90830 and here is a code to save buffer directly to WAV-files. (sorry this part of the tutorial is only avaiable in german): http://www.blitzforum.de/forum/viewtopic.php?p=358688#358688 Here is a (english) software of mine, based on theorie of stapling sinus waves to a more complex sound: http://www.midimaster.de/download/hoerensehen_demo.exe  |
| ||
| Thanks JAddams, I'll continue with that. I had already started to implement hooks. When I'm done I'll post the code. I hope you'll spot any stupidities of mine. Chances are quite good, haha... :P Midimaster, thanks, but that's not what this is about. When I say "record" I mean actually sequencing and not recording wave data. I've already done that 4 years ago, I think, with my offline synth. Now I want a complete realtime sound and music engine I can use with my games or simply to make some tracks. Anyway, thanks, looks good what you got going there! :) Alright, here's a bit of a chaotic code, that tries to use the event system favorably for the timing. It starts to work, though. I have no clue whatsoever about Threads, yet, and need to learn how to deal with them. So far it doesn't look all that easy at first glance. Don't know how to set up data sharing or any of this, no clue what I need or if I even need it. Anyone of you, who could take this code and turn the audio processing into it's own thread, or what ever would work best, it would be awesome! How this works... Hit [F1] to start step recording the sequence. hit [F2] to playback the sequence. When recording, your keyboard acts as piano, while [spacebar] or cursor right skips a step. [delete] deletes the last entered note and moves back on its step so you can replace it. [shift]+[delete] clears the whole sequence (while in record mode!) While in playback mode you can always doodle to it with your mouse on the theremin section (and beyond, LOL) and with your keyboard, of course. Last edited 2011 |
| ||
| Taron, here's some real nice code for you. it's the frequency tables of all notes :) just copy/paste. any probe let me know :) 'this is the sample frequency data in octaves (12 notes per octave, 9 octaves) #SampleNoteData DefData 16.35, 17.32, 18.35, 19.45, 20.60, 21.83, 23.12, 24.50, 25.96, 27.50, 29.14, 30.87 DefData 32.70, 34.65, 36.71, 38.89, 41.20, 43.65, 46.25, 49.00, 51.91, 55.00, 58.27, 61.74 DefData 65.41, 69.30, 73.42, 77.78, 82.41, 87.31, 92.50, 98.00, 103.8, 110.0, 116.5, 123.5 DefData 130.8, 138.6, 146.8, 155.6, 164.8, 174.6, 185.0, 196.0, 207.7, 220.0, 233.1, 246.9 DefData 261.6, 277.2, 293.7, 311.1, 329.6, 349.2, 370.0, 392.0, 415.3, 440.0, 466.2, 493.9 DefData 523.3, 554.4, 587.3, 622.3, 659.3, 698.5, 740.0, 784.0, 830.6, 880.0, 932.3, 987.8 DefData 1047, 1109, 1175, 1245, 1319, 1397, 1480, 1568, 1661, 1760, 1865, 1976 DefData 2093, 2217, 2349, 2489, 2637, 2794, 2960, 3136, 3322, 3520, 3729, 3951 DefData 4186, 4435, 4699, 4978, 5274, 5588, 5920, 6272, 6645, 7040, 7459, 7902 'initialise the data with a call to SetupNotes() 'rate=SampleNotes[octave,note] 'the notes go up to 17 so you can use an octave and 5 more notes on top Global SampleNotes:Float[9,17] 'rate=SamplePureNotes[flat note number 0 to 108] Global SamplePureNotes:Float[109] function SetupNotes() Local k:Int Local j:Int Local freq:Float Local count:Int = 0 RestoreData SampleNoteData For k = 0 To 8 For j = 0 To 11 ReadData freq SampleNotes[k,j] = freq/220 If j < 5 SampleNotes[k,j+12] = freq/110 SamplePureNotes[count] = freq/220 count = count + 1 Next Next end function |
| ||
| MidiMaster do you have any code that does FFT on a sample? I'm looking for something that will give me at least 48 frequencies ?? |
| ||
| Taron, instead of the many if commands - that will eat cpu time to check consider doing the following: in your event loop add the following: Case EVENT_KEYDOWN EventKeyDown(EventData()) then do all you keyboard processing here: function EventKeyDown(EventKey:Int) Select EventKey Case KEY_ESCAPE 'do some code Case KEY_1, KEY_2, KEY_3, KEY_4, KEY_5, KEY_6, KEY_7, KEY_8, KEY_9, KEY_0 'do some code Case KEY_NUM1 'do some code end select end function the reason to use select/case instead of if, is case uses an internal jump table which is very efficient in speed terms :) lots of if's will need all to be checked :( |
| ||
| Ah yes, that's a clever call. Thanks! And thanks for the table, too. I don't think I'll touch this thing until magically some good timing solution arises. It's a bit too frustrating to me right now. I hope, who ever figures it out finds inspiration enough to share as I'm sharing my findings. OH, and JAddams, feel free to alter the code and post your version! ;) Last edited 2011 |
| ||
| WAAAHHHHHH...me SOOOOOO stupid! <lol, shame> And again, drpetter came to the rescue. I don't know why my brains where so freaking weak today, but there's a much more elegant solution to this sort of timing... LOL This doesn't care about the timing goofs of the system and just generates the sounds with their internal tick timing properly. Wow... I really underestimated my degree of stupidity. Anyway, BIG THANKS to Tomas for his brilliance in simplicity and my humble apologies for sharing such silly frustration... Last edited 2011 Last edited 2011 |
| ||
| This clumsy doodle may or may not amuse you -- created with the first keyboard version. (Recorded sequence then mouse afterwards.) http://www.sendspace.com/file/qkd4vy (The unfortunate clicking actually came from Audacity recording the WindowsXP "What-U-Hear" device.) |
| ||
| wow. very sad with the clipping from audacity. I'll write a little wav output or so... or why does it all have to be me? LOL That's VERY cool, though! THANKS! The theremin is great fun as bass, too. Uh, I feel so relaxed now, haha, thank you, J! |
| ||
| Way off topic but.... I once had a go in 4wd Dax Rush kit car with a full race cosworth engine plus goodies. The power figures were something stupid like 660bhp per tonne and it weighed in at 800kg and could go from 0 to 100mph and back to 0 in 9 secs!!! Truly amazing. Anyway - It sounded identical to that engine noise in the vid and OMG!!! It would rev just as fast as the engine in that vid too. I took another friend out in it, and as he wasn't expecting the acceleration, it took his breath to the point his couldn't breathe during that initial squirt of power when going from 2,3,4 and into 5 gears :D It was an amazing machine and I've never heard anything sound and rev like it until now. That savage scream at top revs will live with me forever :D Last edited 2011 |
| ||
| Sounds like quite a privilege there. Fun for sure! :) Here's a little update... It's now tuned properly again, including the theremin. You can record chords now, too, which is great fun, actually. I don't think I've seen that before, but that doesn't matter. Also I've reduced the buffers to 2 now, which makes it a bit more responsive to me. If it's crackly for you, change either the BUFFERSIZE or add a third one again. Just make sure you enter it everywhere it needs to be! [F4] record mode [F5] playback [HOME] 1/(x+1) note length [END] 1/(x-1) note length [PAGE UP/DOWN] octaves (whereby that's a bit shotty) [MOUSE EDITING of SEQUENCE] possible now. Still not perfect, but well. updated! Last edited 2011 Last edited 2011 |
| ||
| Thats really cool. Reminds me of This and This A neat feature would be to keep the note length related to time of the keys pressed :p Last edited 2011 |
| ||
| hehe, yeah, there are a number of possible choices for input. Currently it's just like a vibraphone, which is why there is no notelength at present. I did, however, hook in shift+key to skip one step, which can come in very handy. I'm currently adding mouse editing, too. It needs a few more things on the pattern type design and interaction, but it's beautifully open to that, too. Nothing is too convoluted, yet. That'll happen soon, though, hahaha. :P THANKS, Dave! |
| ||
| Taron, a quickie for you... Are you on pc, linux or mac? Mac here |
| ||
| pc (look at my tagline!) ;) |
| ||
| You are on a good way.... Here some more ideas: Use a sawtooth signal to get a more "piano" like sound. Use a rectangle signal to get woodwind instruments. Of course the Envelope has to be changed back to linear volume Use a staple of sinus signals to get an organ. linear volume! ORGAN: f=frequence g=f*1.01 r=( sin(f)+sin(f*2)+ sin(f*3)+ sin(f*4)+ sin(f*5)+ sin(f*6)+ sin(f*7)) r=r+( sin(g)+sin(g*2)+ sin(g*3)+ sin(g*4)+ sin(g*5)+ sin(g*6)+ sin(g*7) r=r/14 Do you know what ADSR means? Attack - Decay - Sustain - Release |
| ||
| Lovely thread! +++ |
| ||
| Thanks, Ian! I'm having great fun with it for sure. :) Thanks, Midi. Once I'm done with the major mechanics, I'll add the synths. Those will have a great deal more than just waveforms, hehe. We shall see! :} Right now I'm working out the pattern editor. My brain's a little fried right now, but it's coming together. It'll get really exciting, once I'll start dealing with parameter curve implementations and the likes. |
| ||
| This may be of interest here -- the source code to the old Amiga version of Audio Evolution, a decent multitrack recorder/audio editor, has just been released: http://aminet.net/package/mus/edit/AudioEvolution3_src The main point of interest in the above .lha file (opens in WinRAR or 7zip) is "dsp.c", which has loads of audio processing functions in plain C code: filtering, pitch-shifting/time-stretching, FFT, compression, ring modulation, reverb, flange, etc. (The Sample structure is in "AudioEvolution.h".) As the author states above, "You're free to study and use parts of the code in your own projects." |
| ||
| NICE...that might be very pleasant to check out. Compression is most interesting to me, if it's any good. I've added a whole bunch of things to the pattern editor, but I feel quite retarded, when I'm coding these days, haha. It ain't pretty, that's for sure. If there was any indication of anyone here caring to participate actively, I would share my shameful mess, but currently it feels rather a bit like I'm flashing everyone with an ever more dysfunctional erection of code (like erection of a house, right, no? Maybe that word isn't so versatile after all?!) :} Anyway, I'm ashamed. But it does work quite beautifully. It has zoom and pan, copy and move, perfect BPM and note lengths (+triplets) and a few more catastrophic implementations. <lol> Thanks for the link, J, that's awesome! How long is it ago that I heard or saw LHA, hahaha. Wonderful! |
| ||
| Posted an updated version in the first post. Please, feel free to make known to me how lousy my coding is! Also, I'd be super grateful, if you can offer suggestions to correct some of my stupidities. [LMB] = left mouse button [MMB] = middle mouse button [RMB] = right mouse button New features: - pattern editing dramatically improved: ... shift+[LMB] toggles pattern step selection ... [LMB] on selected steps and drag to move and transpose notes ... control+[LMB] on selected steps to copy, move and transpose notes ... [cursor up] transpose selection one note upward ... shift+[cursor up] transpose selection up by an octave ... [cursor down] transpose selection one note downward ... shift+[cursor down] transpose selection down by an octave ... [MMB] pan across pattern (if larger than screen) ... alt+[MMB] zoom pattern - went back to three buffers, which seems to allow for faster feedback (potentially) - display of notes on a pattern step, if in record mode Probably a whole bunch of other little things and a great deal of chaos, but it does kinda work. It may have a few little bugs with the selection dealings, but shouldn't be too disruptive. The coding is terrible, though, I'm sure. Last edited 2011 |
| ||
| I don't know why you're so paranoid about your code -- looks fine to me! Anyway, the latest version is really cool -- nice delay effect too! I don't think you should be put off posting updates just because nobody else contributes, either. I'm certainly interested in seeing any progress you make, and it appears that others are too. I don't have any experience of this stuff -- just very interested in it -- and I still need to try and play with the original version (for simplicity's sake), so I don't think I have anything useful to contribute at this point! Anyway, even if you were to never post a single new update, this stuff is invaluable! |
| ||
| Nah, just a bit frustrated with myself, really. I have some "dumb" days, somehow. BUT: Thank you very much, James, it's much, much appreciated! I will most likely continue to post some updates. I normally try to figure out as much as I can myself, but sometimes it's a bit rough and I'd be super happy about any support. For example: I'm already thinking about preparing implementation of a player routine of it in a game. For that I'm sure it would be best to hook that into its own thread or process. I'm a bit shaky on that so far and if you know anything about that or have some suggestion, that would be quite cool. I guess it's about making it run fairly independently, but sending certain instructions into the thread such as queuing a song or playing a certain sound. Well...there will be better days ahead again, I'm sure. You've been already helping the moral aspects a good deal! :) Big Thanks! |
| ||
| yep, the code looks great. tried to max it out with notes and it sorta went choppy with 30+ notes being played at once ;) |
| ||
| Yes, It's a great read, a bit beyond me but it's a great pleasure to see this evolve. I'm learning quite a bit from your example, thank you! |
| ||
| Haha, now you're scaring me even more!!! Holy cow, I don't think my coding is a good example, really. Or at least I'm afraid it is not. I've seen such wonderful Tlist kung-fu in those threads and all the really clever stuff that my code is miles away from. Not to mention that I'm still a bit uncertain about some of the general logic, which may change to something better at some point, I hope, hehe. But it does work, that's true. I've got an update, too, which I will post later. Some little extra thingies. Maybe I even get saving in there. I will try to use it in a little game and see how it performs, too. This could be really cool, I think. At least it SHOULD be (supposed to be). :o} THANKS GUYS, really, very encouraging! If you have any questions, don't hold back! Might give me a chance to revisit some of my nonsense up there at some point, too, hahaha... :oP |
| ||
| I had a quick look to see how it could be threaded, but it's a bit too complex for me, I think! Still looking though... I suppose it's the actual manipulation of the audio data you want to move to another thread, but for what it's worth, it seems the actual playback happens in a separate thread, spawned automatically by OpenAL, according to this. Running Task Manager seems to confirm this -- it lists 4 threads for the program, and just under 50% CPU usage; I have 4 cores, so this means two cores are being fully used: the main app busy-loops (that would be showing 25% CPU usage on its own, ie. one core), so I guess the other 25% is OpenAL mixing and playing. (If I comment out the process () call, I get 20% CPU usage, confirming that the app alone is using almost a full core. (Also, if I comment out all the device setup/OpenAL calls prior to the main loop, it shows only 3 threads, which seems to be normal for a BlitzMax app.) BTW I noticed I could add a delay of 5-10 millisecs to the main loop, with no ill effects on the sound here, taking CPU usage right down to 2-10% in general, though I don't know how it would affect other systems. Finally, I don't know if you're using the full SDK, but it seems OpenAL actually provides a bunch of effects, so you could possibly avoid a lot of coding in this area; there's an example here. Last edited 2011 |
| ||
| Thanks, James, that's awesome! Really, thanks for checking all this out! I've seen the EFX extension stuff, but I guess I'll have to add some C to gain access to it. I'll see that I have a look at it for sure. I'll do more tomorrow and check into all that. Thanks again! |
| ||
| Just posted a little update, adding a few waveforms to it (saw and square) and some more goodies for editing and performance. Code doesn't get less crazy, but well... Page up/down controls octaves +shift controls instrument type (just waveforms currently) HOME / END controls note duration +shift controls BPM Don't know what all else I've changed, but you'll find it out, I'm sure. :) |
| ||
| Damnit, won't be be near my BMX pc, till tonight, looking forward to it! |
| ||
| Excellent, it works well, esp. considering there is no assembly language here, which I assumed would be a pre-requisite fir this type of application. Even if you stop now Taron, this has proven to be a considerable insight into the obscure and black art of sound synthesis. |
| ||
| HAHAHA, I still tend to view it that way, too, haha, the black art of sound synthesis. ;D I'm currently experimenting with making more buffers at smaller sizes, which seems to improve the feedback a bit. Not sure, if assembler would help in that regard anyway, unless one was to bypass openAL and write an audio-wrapper oneself. No idea, though. It's likely to improve the wave creation part, but bmax is surprisingly sufficient so far. Thanks and you're very welcome. I figured it was time to solve that great audio riddle. :) |
| ||
| Taron, do you have any experience with openal EFX stuff. I'm trying to get any info on how the filters/delays etc can be used in blitz? |
| ||
| Looks like you have to dig through their sdk and implement it yourself. I had a quick look, but didn't care to try hooking it up. On a sidenote, I had no idea this was as good as it is before I tried it on a silly little laptop, where the latency appears to be next to nill, it's so quick. On my dumbass mac pro it felt hopeless with the big delay after each keystroke. Now I'm both pleased and freaked out! Have you all had such a great experience with this one, that the sound comes almost immediately with the keystroke and the mouse action on the theremin section? Somebody could've told me, hahaha... or so. Anyway, just thought I'd share my joy. I also started a new one, which already has a more sophisticated pattern editor and a the beginning of a neat GUI as well as a whole different kind of synthesis. Really promising, but I can't continue right now. :o[ Last edited 2011 |
   |