FEATURE REQUEST: Streaming audio
BlitzMax Forums/BlitzMax Programming/FEATURE REQUEST: Streaming audio
| ||
| As above. I'm using REDi's Maxmod for this at the moment but we desperately need a cross-platform solution, and for me, Blitzmax really needs to have this by default as its a very important feature for 99% of games. A 4mb OGG takes up 40mb of resources if you use LoadSound and you can't justify expenditure like that to play a bit of music. |
| ||
| I completely agree. I'm almost tempted to give it a shot, but time is of the essence right now :X I needed a simple windows/mac music playing solution yesterday. Literally :) |
| ||
| Agreed But don't think it will happen. Streamed music -> something like threading mechanisms in language -> so far always seen as "not needed in BM" out of Marks view (sadly). |
| ||
| Streaming doesn't really need threads. What you're doing is just decompressing a small chunk of a file each frame - no different to loading in part of a file each frame and displaying it. All you then need is to be able to read the file format. |
| ||
| Agreed. A simple callback from the FreeAudio mixer for each sound would be ideal (like the FillBuffer method in MaxMod TMusic) Threads wouldn't be needed, it can work the same way that FreeAudio works now AFAIK. This would be the prefect solution for me, allowing me to have maxmod just as a module playing module, eerrr.. mod mod :) |
| ||
| Really need it, but we can only keep our fingers crossed... AngelDaniel is right, I've seem some way to implement streaming with this method in some Japanese games, I hope BRL can look at MaxMod and bind it in... |
| ||
| Callbacks actually are not the way to go as well. The GC is not threading safe. It does not make a difference if it is use for threads or by something outside that tries to get around that with callbacks. It will just crash if it takes too long so unless you can gaurantee that the user PC (CPU and Harddisk mainly) is fast enough, a callback is a potential crash bug that you can not resolve. Angel: I know that this can be done. It has been done with Blitz3D like this for quite some time. But actually, Streaming media stuff is just something that works much smoother with true threads instead of faking the whole stuff through time slicing in software. I don't know the OGG format well enough to say if it is able to work like this but non the less: streaming from netstream isn't possible with slicing, as it needs a continous "get - decompress - play", slicing will lead to some very unpleasant results if the game is taking a little longer for something as the whole net part will be blocked thanks to not having threads. (net stuff does not need much of CPU but it needs to do its stuff continously behind the scene. Some "funny" people try to do that through timers but as they are only guaranteed to break at Nr 16, they are not that usefull if something must happen at a given time) The slicing only works to stream from harddisk, not from a network stream and as in BM TStream is TStream, it will need to work for both. |
| ||
| To be honest MaxMod works great. And if it was cross-platform then I probably wouldn't have felt the need to start this thread. I would have started a different one demanding MaxMod gets a more descriptive name (i.e. StreamingAudio.mod), and that BRL put it on their mod server ;) |
| ||
| So I'm curious. What would it take to make maxmod cross platform? |
| ||
| portaudio is cross platform and works now? |
| ||
| As far as i know the SID module streams. |
| ||
| I agree that streaming is needed, but not just for Ogg, Wavs need it to for the same reason (so the entire data is not all loaded into memory). Oggs are nice and small but I find the decomress time way too long. Maybe this would be reduced by streaming? (I'm not sure how Oggs are compressed - can you decompress part of an Ogg?) I found it best to convert the oggs to wavs at startup since wavs load quickly, but you still have to have the whole thing in memory. I put some code in the archives for ogg -> wav conversion. |
| ||
| I agree that streaming is needed, but not just for Ogg I don't think that was even suggested? We were talking about audio in general. |
| ||
| I also think BlitzMAX needs streaming audio ASAP, but find the belief that people are willing to take a massive performance hit in their game, by not using threads somewhat disturbing. |
| ||
| As far as i know the SID module streams. SID music is tiny since its actually machine code, it wont need to stream. Unless I completely misunderstood what you were saying? :) |
| ||
| The sid module converts it to wav or something before playing, the original release decompressed it all to ram before playing but the updated version streams the decompression. |
| ||
| Dreamora, a simply fix for that is to have a buffer which you make to be bigger than is needed for 1 frame so you have the audio streaming in faster than it needs to play, then if you get slowdown you just read ahead in the buffer - just like how cd players handle shock resistance. All you then need to be able to do is decompress the audio in chunks as needed. What if you had raw audio that was compressed with your own compression method, such as zlib, so you can implement whatever system you like? |
| ||
| ... but, as FlameDuck says, it is all a performance hit. |
| ||
| AD: I have a Dual core system, any current gen PC is dual core. If you explain me why I should waste hundreds of lines to fake threading in a "modern language" although I have true multithreading, I perhaps start seeing any sense for it. But for now, its just a joke. Not only BM does not support threading, the GC is not even thread safe so you could use callbacks and threaded libraries because it will bomb this damned thing. But I'm sure as this topic is all time returning, Mark will accept at some point that BM users did not pay to switch to BM to stay on 1998 tech, even if he insists that anything can be done by faking threading through slicing.(which for most things might be true but it is still much harder and tends to create more problems than true threads, especially as current gen computers are fast enough for several threads side a side even on single core machines) Until then, I will simply use the same languages to do this kind of stuff I'm using now. PureBasic, C++, C# even thought I would prefer working with BM. But it is just not possible at the moment. No interest in time slicing AI stuff and the like. The topic is complex enough without adding another layer of complexity with a large bug potential that is not needed nor desired. |
| ||
| What I mean is, FreeAudio doesnt use threading now, and it is still possible to do it that way, ofcourse threading would be better, but i cant see it happening for quite a while yet. FreeAudio mixes and streams its own audio to the sound card already, all it needs is a way for the mixing bit for a TChannel to be done "outside" FreeAudio (for now anyway). As for it taking to long to process the decompression of audio, I could (with my DJ app) stream 20+ mp3/ogg/mod's with mid cpu usage on my crappy machine, with a buffer size of .25 of a second, so streaming for games should be minimal (one track at a time). *EDIT LOL* |
| ||
| I still don't really see that it has to be threaded. I am sure if I had a dual core machine I'd love for Max to be a multiprocessing system. But that doesn't mean that a streaming sound system HAS to be thread oriented. Like I said, all you need to be able to do is read the audio file in big enough sections and far enough ahead to compensate for any temporary delays, and to understand the file format/compression so that you can turn it into playable sound. Whether you use ogg or mp3 or zlib or whatever for the compression doesn't matter so much. If the decompress is fast enough and it gives reasonable file sizes I don't see why you couldn't just maintain a buffer and stream it yourself as part of your game loop. Same thing with video. If you knew how to decompress it you could stream it. |
| ||
| The alternative for it being threaded is timesliced. This means you're doing quite a bit of extra work per cycle. If, as you suggest, you stream in big chunks then you've either got to stream it all or get the same performance hit. Imagine a fast-paced shooter with lots of objects and then having to stream audio on the same process... if I understand the problem correctly. |
| ||
| make a dedicated exe that monitors stdin for new url's and streams audio from its current url, use delay and preferabley the event system to make sure this stream play server uses the minimum cpu time possible (use framework and you should get it in at under 100kb, 200 max!) with your main app use the freeprocess module to spawn the stream play server and and send it url's down the Tprocess pipe using threads should NOT be done glibly they are at times an UNMITIGATED NIGHTMARE it is *very* easy to cause *very* difficult to find intermittent bugs with threads, they are NOT fun! for just about *any* use you might THINK you need threads you can make a simple server type program and use that instead This approach is both simpler in the long run and a lot less prone to bugs and likely to behave a lot more robustly |
| ||
| Angel: You are right, the sound would not need to be threaded. But some other parts would need to get not out of sync due to the streaming sound. Chris: Mainly you are right, using exes is a way to work around that currently and thats the way I've been doing it. The problem comes in when you want to do AI. AI needs realtime reflection of the whole world. This is a serious problem, no mather if done over pipe or through UDP (I prefer GNet as it allows perfect ghosting). As well this approach has the problem that: 1. Firewall apps will step inbetween if networking is used instead of pipeing 2. Security apps will most likely try to prevent my app from starting the virtual CPUs (thats how I called this apps) and interact with them. At least Zone Alarm and Norton will do this for sure. If the user does not have manual response method, the app will just fail ... Threads on the other hand aren't that hard to use in this case. The AI operates on very distinct data, the only thing it does is query information from the world on its environment, which is easy to implement threadsafe through a requester system. Thats what I currently have in PB. Don't know if I will continue my experiments in PB or if I will extend it in a way where I can use it as DLL in BM until I can natively implement it to BM. (C++ would be a way as well but I would like to implement it bugfree, something C++ hasn't been able to allow me so far without massive headache in the end ^^) |
| ||
| Dreamora: But some other parts would need to get not out of sync due to the streaming sound. What I was thinking is along the same lines as your average video player - start by buffering the beginning of the audio until a buffer is filled up, then start your game, and each frame read in several small chunks of audio, enough for a whole frame (delta time?) and some extra so that it keeps ahead of the when it needs to be played (doesn't matter what the framerate is because you're basing loading needs on audio playback timing). Because you load lots of small chunks rather than big chunks, you can `smooth out` how many chunks are loaded per frame so as to have a smooth amount of cpu usage per frame. You'd have to figure out some kind of `kilobytes versus time` number which is how many bytes you need to read and decompress per frame on average, figure how much time that takes ideally and then the time left is your time to process the game. That said I think it would be nice if the o/s was loading data from disk at the same time as the game running just so the game isn't doing a lot of extra waiting for the file to be accessed. Or... don't go with recorded audio files, go with a module and sound samples played back at runtime. |
| ||
| the way maxmod streams its sound is to have a looping buffer of a set size (say 0.25 of a seconds worth of audio), then all you need to do is stream the same amount of data that gets played per frame. so it basically plays a bit then fills the buffer with new audio behide the play cursor. so with a sample rate of 44100 at 60fps = 735 samples per frame average (thats 2940 Bytes at 16bit), and the higher the frame rate the less you'll stream. |
| ||
| I've agreed about a zillion times in other threads so I'll agree here too. However I'll add that I've made two pro games without this by having 1Mb ogg files that load in (minor delay) and consume say 40MB and then play. This seems OK if you structure your game accordingly. |
| ||
| Silly question, but why would the OS need 40MB of resources to play a 1MB audio file? I knew Windows was bloated and inefficient, but 'Good Golly, Miss Molly!'. This is ridiculous! Cheap ram and storage systems have made programmers lazy.... Russell |
| ||
| No not the least. But OS can only play WAV so for playback you have to decode it ... and OGG is as MP3 a very good "minimalize" format ... good for harddisk bad if your music player does not support streaming and decodes the whole track before start playing ... which is exactly what BM does thanks to its damned locking behavior on any IO command (and TCP). No mather what you do, as long as you do it, BM is on halt. |
| ||
| I'll add that I've made two pro games without this by having 1Mb ogg files that load in (minor delay) and consume say 40MB and then play. This seems OK if you structure your game accordingly. Grey, has this caused any problems for you? I mean with users complaining, or having problems running your games? I'm thinking I might just throw in the towel, and use playsound after all ( ugh :X ) What are you getting away with as far as song duration goes? I tested a 2 minute song yesterday, with an empty program, and it consumed nearly 50 megs ( ugh x2! ) |
| ||
| It's also a bit more difficult with music intensive games (RPGs, RTS) which might have a number of themes. You can't *just* play "The Start To Fantasy" you know. |
| ||
| "Wheres da schtart?... wheres da schtart of fantaseeee..whoa.oh.." |
| ||
| Russell: Windows unpacks the ogg which is highly compressed into RAM and it takes up more space, simple. Nothing to do with bloated coding. Orca: Well I've sold thousands of games with that method over the last 3 months (i.e. no streaming) so no it's not a problem. If a user's PC can't load in and play an ogg file then it probably won't cope with the rest of the game either. I only load in one ogg at at time .e.g title screem, game music, level complete music. |
| ||
| Orca: Well I've sold thousands of games with that method over the last 3 months (i.e. no streaming) so no it's not a problem. If a user's PC can't load in and play an ogg file then it probably won't cope with the rest of the game either. I only load in one ogg at at time .e.g title screem, game music, level complete music. I'm genuinely surprised that you're satisfied with that solution.~40mb for a four-minute soundtrack is an awful waste. |
| ||
| I agree that when you load the ogg it has to decompress it all to ram, so your resulting memory use is going to be way higher than the file size - thats the whole point of the compression, from a non-streaming standpoint. And what's 40MB free ram nowadays when people are throwing gigabytes into their machine thinking it will help windows to not be so slow and bloaty. That said, a stream would still be nice. |
| ||
| Personally, I think a cross-platform sound library would be nice http://www.openal.org/ |
| ||
| BRL won't implement audio streaming in the near future at least... Mark seems to hit the bottle neck of Max3D, and skid is busy at MaxGUI, thgs are progressing extremely slow. I didn't find a Mac solution yet, and PureBasic seems not in any better situation either... |
| ||
| BRL won't implement audio streaming in the near future at least... Where did you get this information from? Mark seems to hit the bottle neck of Max3D, and skid is busy at MaxGUI, thgs are progressing extremely slow. |
| ||
| Mark did state in a post about 2-3 months ago there was no intention for BRL to work on Bmax audio. I tried searching for it but got an Internal Error for what I think was the post. |
| ||
| a) What are the options available for getting streaming audio working by ourselves? b) Why aren't we working on it to make it happen? |
| ||
| Well, there's Maxmod, which is doing the job perfectly for me. The only drawback is that it isn't cross-platform and we haven't exactly established why this is, or what would need to be done. |
| ||
| I dont have a mac, or even linux atm LOL, I've tried various sound libs for maxmod... openal sucks balls on linux, stability problems with portaudio ect. directsound works great for me atm, but the thing about oop is its brilliant when it comes to swaping things about like this, so i'll probably have another go with portaudio at some point (as Nigel says it works now). But I'd still prefer to see streaming added to FreeAudio as standard. at the end of the day this is just a hobby project for me, its had very little interest over the last year and a half. |
| ||
| what is this 40 mb .ogg file anyway. a 5 minute song compresses to like 3-5 mb. maybe just break this enormous file into smaller chunks for real-time use. I use audacity to work with my ogg files. |
| ||
| what is this 40 mb .ogg file anyway. a 5 minute song compresses to like 3-5 mb. Yep, and when you load it into Blitzmax it takes up the original 40mb again. Try it. maybe just break this enormous file into smaller chunks for real-time use. What did you think 'streaming audio' meant? ;) But I'd still prefer to see it added to FreeAudio as standard. Blitz3D had "PlayMusic" which was more than adequate. How hard can this be?? |
| ||
| Blitz3D had FMOD built in |
| ||
| > Blitz3D had "PlayMusic" which was more than > adequate. How hard can this be?? I agree, it sucks having to load in your audio in its entirety. If you use more than 2~3 music files in your game you are greeted with up to an entire minute worth of swap file deallocation once your game ends. This has been a sucky thing of Blitzmax right from the start, I can't see this issue ever being solved to be honest. |
| ||
| Blitz3D had FMOD built in Yep, and FMOD is cross-platform so the point is, it *is* possible to do it.It baffles me why they didn't just use FMOD for BlitzMax and stick an extra few $ on top of the reg fee to cover the licence. |
| ||
| You can stream ogg's and wav's ( and mp3's if you want to spring for a license ) with IrrKlang. Seems to work just fine although you probably don't want to set it running in it's own thread, as that seemed to have some issues. |
| ||
| FreeAudio is cross-platform already, it just needs to let us have access to the mixing buffer. IMO Skidracers done a great job with FreeAudio so far, It'd be a shame to waste it! ;) |
| ||
| Actually, this is not the first time for requesting music streaming, I've suggest this since around 1.18, but BRL didn't implement it yet, so that makes ppl find alternative solutions like OpenAL/PortAudio/FMod... Currently I use uFmod and convert mods to XM, that enables the minimal requirement of "Playing Long Music", however OGG streaming is good if posible. BRL isn't ignorant to this, just because ther're more important things to do. |
| ||
| I've suggest this since around 1.18 Wow, since all the way back then? ;-) |
| ||
| And till now nothing happens, shows the ignorance of BRL, and shoulb never in the future. |
| ||
| It baffles me why they didn't just use FMOD for BlitzMax and stick an extra few $ on top of the reg fee to cover the licence. My guess would be that Brett ( the FMod guy ) is not very receptive to indie developers any more. I tried to deal with him and found him extremely patronizing and unhelpful. I've heard similar things from other people. Perhaps BRL are just nicer than me and don't want to come out and say that about the guy. If he wants $5k from indie developers releasing one game, I can't imagine what he's going to ask for BRL to let everyone release their games with it. It may even be that statically linking FMod and letting everyone sell games which are using that statically-linked code is just not available at any price any more. |
| ||
| I think the problem with FMod is more that BM is a source thingy. For the old blitz there was no problem, it was prebuilt but with BM comming with source, I would assume it would have given some problems. Or perhaps he just did not give a license for this kind of project and wants the users to get license for themself which naturally is the better deal for him. |
| ||
| It baffles me why they didn't just use FMOD for BlitzMax and stick an extra few $ on top of the reg fee to cover the licence. Because an FMOD liscense costs a bit more than "an extra few $". In fact it's $15.000 on top (triple platform commercial site liscense), or more than 150 times the price of BlitzMAX. It may even be that statically linking FMod and letting everyone sell games which are using that statically-linked code is just not available at any price any more. Bingo. |
   |