Synced actions with audio?
Monkey Forums/Monkey Programming/Synced actions with audio?
| ||
Does anyone know for sure if Monkey has enough by default to reliably sync actions with how long audio has been playing on a timeline? We have done our own tests and they are inconclusive. Like with any rhythm based game the beats you hit line up with a song. |
| ||
Monkey can't offer exact equivalence across targets for audio latency (or other low-level functions). Some platforms might be fine, others not. |
| ||
Does anyone know for sure if Monkey has enough by default to reliably sync actions with how long audio has been playing on a timeline? I doubt it will matter how long the music has been playing. The question is on which targets does MusicState return an honest value so you can timestamp when exactly the music began playing out the speakers (after network, decompression lead times and system latency are taken into account). |
| ||
in my DUETT project ( http://www.monkeycoder.co.nz/Apps/app.php?id=182 ) I recognized (and you can hear it) that the HTML5 target has big problems in starting sounds at a certain point. Sometimes it sounds exact sometimes not! We recognized differences of 100msec. I seems not to be a fixed latency, but a hickup, that sometimes happens. The Android target is much better. For long audios I can conclude, that a already running sound never broke on any target. So you only have the problem of max 100msec offset at the start moment. |
| ||
@Midimaster: The Android target is much better. Huh? Android is terrible. The getCurrentPosition() method on MediaPlayer can't be trusted at all (I've seen differences up to 600ms), and the latency with playing sound samples is atrocious. http://code.google.com/p/android/issues/detail?id=3434 On the other hand, I've found the HTML5 audio to be quite good. Check out the Diddy storyboard example. I can't vouch for its latency since I haven't tested that, but it has to be more reliable than Android. |
| ||
Seems to be the death of a rhythm based game which relies on correct timing in relation to the music then. :( |
| ||
@Soap: Seems to be the death of a rhythm based game which relies on correct timing in relation to the music then. :( Which is why there aren't many of them, and partly why I gave up on my osu! port. |
| ||
thats exactly what we trying to do: rhythm based games. But I still believe in browser developers. They decided to improve the audio drivers. On my Android target (a small, slow Samsung with 2.3) I can say, that it is good enough. And if CPU speed rises, those problems getting smaller over device generations. |
| ||
Yes, I think maybe later this year even it may be more viable. Things change fast. |
| ||
Maybe you could make a rhythm-based game using sounds instead of music. What I mean is that the music track would be broken down into short sounds which are assumed to fire at the time they are played. |
| ||
That would work for tracker based music, but there would still be problems if you wanted to do anything complex. The plan b we are taking is to divorce the patterns from the music completely. And if that doesn't work out to being fun we have a plan c too. Maybe later this year if things are different we will risk having them in sync - or at least closer to being more viable. Chrome auto updates and that's the only HTML5 browser we care about, so when it's fixed it will be fixed for everyone, and Android devices break over time and need to be replaced so eventually everyone will be on newer Android versions which also have the problem fixed. |
| ||
@gerry I'm only talking about music we diveded into sounds and try to fire them in the right moment. Exactly here we have the problems of different latency from sound to sound. So a beat of four sounds fired at... 000msec-250msec-500msec-750msec-1000msec ...sometimes reaches the ear at... 010msec-333msec-510msec-888msec-1010msec do you see the always changing latency? +10msec +83msec +10msec +133msec +10msec What we can recognize is, if you fire 5 sounds at the same moment, they all have the same latency and reach the ear together. What about chrome browsers. Does somebody have experience with sound latency on it? I always test on Firefox. |
| ||
We were testing on Chrome and there was up to 100ms latency with playing audio on localhost, and Chrome lied about the audio actually being played - or at least the Monkey language doesn't account for possible latency and says the audio is playing instantly when it is not. I should look into that. It may be that Monkey assumes it is playing instantly but not that the browser actually is playing instantly, and modifications could be made to make tracking more accurate. |