BMX wrapper for Lame_enc.dll?

BlitzMax Forums/BlitzMax Programming/BMX wrapper for Lame_enc.dll?

Grisu(Posted 2015) [#1]
Hi guys,

I'd like to use Lame (http://lame.sourceforge.net/) in my app to convert wav-files to mp3.

Has anyone written a simple wrapper for it yet?

Thanks
Grisu


skidracer(Posted 2015) [#2]
In theory you should be able to use CreateProcess "ffmpeg" with ffmpeg and lame binary files bundled with your own app. Talking to lame codec directly seems like an awful lot of work for little gain.


skidracer(Posted 2015) [#3]
Sox is another light weight command line tool that will recognise lame dll.


Grisu(Posted 2015) [#4]
Thanks for the help.

I'm probably better of creating a batch file and the users can compress the wav files themself.

Also, I'm not sure if using this stuff would cause license fees.


xlsior(Posted 2015) [#5]
Also, I'm not sure if using this stuff would cause license fees.


More than likely it would -- for comparison, even Audacity (one of the most popular free audio editing programs) doesn't bundle lame_enc.dll with the program itself but allows the end user to drop it on their system to add support for MP3 export:

http://wiki.audacityteam.org/wiki/Lame_Installation

Because of patent considerations, Audacity cannot ship with a built-in encoder to export MP3 files, but is able to use the third-party LAME encoder. This page explains how to download and install the correct LAME encoder for your system, and outlines the legal issues surrounding LAME encoding patents.


Apparently MP3 patents won't expire until 2017 in the USA.


Grisu(Posted 2015) [#6]
I could add a batch file / command line creator for lame.
So users that have the dll installed on their system, can use it.


Brucey(Posted 2015) [#7]
I could add a batch file / command line creator for lame.
So users that have the dll installed on their system, can use it.

If you aren't going to provide anything directly in your program, it seems a little pointless generating a batch file to do something. Certainly not something you would expect an "average" user to have much confidence (competence?) using.

If someone really wants to make an mp3 for the wav data you've ripped off the radio stream, I'm sure they will already have some software on their machine to do it? (or at least something more useful than a batch file)
Usefully, you could perhaps just open the folder for them in explorer, where you've just dumped the .wav file. Then they can choose to compress it in whatever way they might usually do so.

Just a thought, or few :-)


skidracer(Posted 2015) [#8]
I always get a dirty feeling having to download lame.

Using an ogg encoder would not leave the user with that feeling and is trivial from BlitzMax.


Grisu(Posted 2015) [#9]
The "folder switch" looks like a good option.

For the ogg encoder (http://www.vorbis.com/faq/):

Is there an encoding function in the code archives?
I only found the "brl.oggloader.mod" and an "ogg to wav converter" on the forums.


skidracer(Posted 2015) [#10]
Oops, sorry to not link.

If I had, I would have found the relevant module abandoned on googlecode out of public reach.

Back momentarily...


relevant files are still available here:

http://max-edit.googlecode.com/svn/trunk/mod/axe.mod/oggsaver.mod/

Google is currently exporting the entire kaboodle to github where I will test soon to make sure it works.


col(Posted 2015) [#11]
If you really do want to venture into mp3 land then the Blade mp3 dll wraps the lameenc library to give a simple interface to encode wav to mp3.


Grisu(Posted 2015) [#12]
@Skid:
I needed to make some changes to the module code in order to make work.
Therefore I replaced the header with the ogg files present in the pub.mod (Brucey Edition).

Btw: There are some updates available on the official site (put as comment below).

Import "../../pub.mod/oggvorbis.mod/libogg-1.3.1/include/*.h" ' Latest version: 1.3.2
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.4/include/*.h" ' Latest version: 1.3.5 
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.4/lib/*.h"     ' Latest version: 1.3.5
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.4/lib/vorbisenc.c" ' Latest version: 1.3.5


Modules rebuilt fine and the exampe works too.

But the given functions don't seem to work, if I throw a WAV at them? I'm only getting "blank" data file.

	Local filename:String = RECORDINGS_DIR+station_name
	Local ogg_sample:TAudioSample

	filename.ToUTF8String() ' Setting the basic filename of the last recording.

    Print "Loading wav..." 
    ogg_sample=LoadAudioSample:TAudioSample(filename+".wav" ) ' Loading original wav-file

    Print "OGGing sample..."
    Print SaveOGG(ogg_sample, filename+".ogg") ' Encoding...


@col:
I love adventures, but porting the blade code over to bmx is beyond my coding skills.

The FLAC format might be an additonal free alternative. But again, there's no bmx-module for it.


col(Posted 2015) [#13]
@Grisu
Oops. I forgot about this thread, my apologies.


I love adventures, but porting the blade code over to bmx is beyond my coding skills.



I love them too :-)

Here's a simple wrapper, if you can call it that. Just stick the BladeMp3Enc.dll file in the same folder as the .exe produced by the following code and you should be able to choose a wav file to encode to an mp3 in the same folder as the wav file. I did start to put the encoding function in a thread but then I got sleepy... and I'm sure that you wouldn't want me have all of the fun by doing ALL of the work now eh ;O)

Do read the source though as I've put some hints in there for you...
Also it may be a good idea for you to read the BladeMp3EncDll.h file for some acceptable parameter values so that your users can change the encoding bitrate etc



Grisu(Posted 2015) [#14]
Hi Dave,
thanks a lot for the simple code! ;)
Now I have enough bits to tinker with for the next months.

I'm currently adding ogg support to my recoder (see other thread). Mp3 may follow later. The licensing unclairity bugs me though. Thus I probably won't be able to include the dll in my releases.


Grisu(Posted 2015) [#15]
Hi Dave,
I tried your example code from above. It compiles fine but produces 768 Byte large empty MP3 files?!

No error message. It just states: Encoding of ... is finished.

P.S.: I'm using the latest BladeEnc.dll, i.e. 1.3.2.


col(Posted 2015) [#16]
Hi Grisu,

That's the same version I used :-)
How big is the .wav file? Could it be over 2Gb? the size of a signed 32 bit integer?


Grisu(Posted 2015) [#17]
For testing I used .wav files that were created via the FMOD_OUTPUTTYPE_WAVWRITER.


Const FMOD_OUTPUTTYPE_WAVWRITER:Int
Description All - Writes output to fmodoutput.wav by default.
Information Use the 'extradriverdata' parameter in System::init, by simply passing the filename as a string, to set the wav filename.



My guess is that this might cause the issue. Though I can't change any settings for this via FMOD.

All test files were <500 MB. You could record a station with PRP and see for yourself.

Sorry to bother you. :(


col(Posted 2015) [#18]
Hey Grisu,

My guess is that this might cause the issue.


Naaah... it was bug in the header reader.
Here's a fixed version - there's only one extra line, line 96.




Grisu(Posted 2015) [#19]
Awesome! Works fine now and I can set a custom bitrate.

I like you progress bar a lot. Do you know how I could encode the ogg files in chunks as well to create a progress bar? From the oggsaver.mod the external command doesn't seem to allow this.


Encode_Ogg(outputstream:Object,callback:Byte Ptr,freq,channels,samples:Float Ptr,length,compression:Float)



This way I could offer the same functionality / gui for mp3 and ogg saving.


Grisu(Posted 2015) [#20]
Did more digging this morning.

The Lame website states: BladeEnc One of the first free encoders for Linux. Development has stopped at version 0.94.2. Quality is not satisfactory.

As I tried, you can also use the lame_enc.dll directly with the construct.

Global hBladeDll = loadlibrarya("lame_enc.dll")

Works nicely. And should be faster / better than the blade.dll.


col(Posted 2015) [#21]
Thats cool!

It looks as though they've incorporated the BladeEnc api into the LameEnc.dll. That's a pretty smart thing to do as I think BladeEnc was used in a fairly large amount of software in days gone by.

The progress update is a bit of an illusion really. Ideally you'd have the encoder running in a separate thread. That way the user can cancel halfway through encoding should they choose to. As it is the main app thread is doing the encoding with no way to exit - you could check for user input I suppose in the encoding loop and exit early I guess.

I'll have a play with the vorbis encoder later to get some encoding feedback going. It looks as though only a minor tweak would be needed.


col(Posted 2015) [#22]
I made a small modification to the axe.oggsaver module ( hope you don't mind skidracer?!? ) that adds an optional extra parameter to allow a callback function to be called during encoding. This modification is also 100% backwards compatible will any code that already uses the original axe.oggsaver module and should suit as a drop in replacement.

Also here's the same app as above with the option to choose ogg or mp3 output. The app now expects the lame_enc.dll as you noted above and of course the modified axe.oggsaver module...

SampleApplication.bmx:


oggencoder.c [axe.mod/oggsaver.mod/oggencoder.c]


oggsaver.bmx [axe.mod/oggsaver.mod/oggsaver.bmx]



Grisu(Posted 2015) [#23]
@Skid:
If you update your module, could please also include the latest oggvorbis codecs?

Import "../../pub.mod/oggvorbis.mod/libogg-1.3.2/include/*.h" ' Latest version: 1.3.2
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.5/include/*.h" ' Latest version: 1.3.5 
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.5/lib/*.h"     ' Latest version: 1.3.5
Import "../../pub.mod/oggvorbis.mod/libvorbis-1.3.5/lib/vorbisenc.c" ' Latest version: 1.3.5


@col:
Thanks a lot for your help Dave.

I'm still coding on my app's framework, will report back asap. I also made your "old" example "SuperStrict". Will change your new one as welll.

Stefan


Grisu(Posted 2015) [#24]
After spending my weekend on this. I got it up and running. I posted some images and a test build in the PRP thread.

Below is an updated example code:

1. It checks if one of the dlls is present at startup. If both are present the app uses Blade > Lame.
2. You can close the Window while encoding.
3. Sets custom bitrates via a new drop down.
4. SuperStrict code.
5. Additional status messages.

Example:


The issue that I have is that the different bitrates are only supported by the blade_enc.dll, if you use the lame_enc.dll, it'll always compresses at 320 kb/s. - Do you know a workaround for this? Must have something to do with the <h1> construct. But I'm not sure how to set this up.


col(Posted 2015) [#25]
Nice one!

To get the compression parameter working as you're expecting with the lame_enc.dll I think it's best to now use the BE_CONFIG_LAME struct.

Here's a modified version that uses that more fine grained struct.
Also, I hope you don't mind but I got rid of the 2 large Select statements and put the values into arrays - personally I think it keeps the code easier to read on the eyes, and it is purely my personal flavour on keeping code tidy. You can change it back if you prefer Selects :D

edit:- Tested with lame_enc.dll 3.99.5, and BladeMp3Enc.dll 1.3.2




Grisu(Posted 2015) [#26]
Strange, from my testings the files created are now always at 32 kb/s rate?

I checked the results via the VLC player:


Both codecs don't work anymore for me. As usual used PRP recordings as example files.


col(Posted 2015) [#27]
Omg, sorry about that!

Try changing line 218 to
config.nPreset = 9000 or = 10000
and all should be good!


Grisu(Posted 2015) [#28]
From my tests it looks ok now!

Now to threadding mp3 chunks... *rides into the dawn*


*(Posted 2015) [#29]
Looking good :D


col(Posted 2015) [#30]
oh ok theeeennn :D

Encoding of mp3 is now done in a different thread. The 'Encode now' button toggles between 'Encode now and 'Cancel encoding' depending on if its encoding or not. Hmm... maybe threaded Ogg later too?

Remember to build the axe.oggsaver module in threaded mode too otherwise you'll get errors during compilation.

It seems ok for a little example application :^)

EDIT:- It turns out that the app would crash when threaded during encoding in release build. It happens that upon closer inspection of the dll that the functions are being exported as standard c calling convention functions and not using the stdcall convention. This will cause the stack to be corrupted. Strange how this only showed up when threaded in release build. Oh well, fixed again :D




Grisu(Posted 2015) [#31]
I can't compile the code in threaded or unthreaded mode: "Identifier ProgressCallback not found" or "TSemaphore not found"?

I did rebuild all modules in threaded mode beforehand.


col(Posted 2015) [#32]
Hey,

Try again with the same code as I've just made a change. I can copy/paste the code into the standard editor, switch on threaded build, gui build, with or without debug and it runs ok.

TSemaphore... that's strange as it's a BMax multithread Type that's recognised when threaded build is switched on - it's a part of the standard Max installation.


Grisu(Posted 2015) [#33]
Compiles now.

Can I put the ogg encoding into the same thread construct as well or is it not thread-safe?

Will need to find some time over the weekend to implement and test it.


col(Posted 2015) [#34]
Hey Grisu,

I was a little bored last night so I had a play with this little app and finished it off this morning.

Keeping in mind that you mentioned FLAC encoding in a previous post above I changed the complete structure so that now, with correct planning, any future encoders can extend from a base encoder and the app can use it without too much fuss - its just doing things a little more towards the OO way. The base encoder is still heavily tied to this application but its just a start in the OO direction.

There's another little tweak required to the axe.oggsaver module to allow cancelling in the middle of encoding. The tweak should still be 100% compatible with existing sources etc.

The app will require your robust testing ;-)
Have Fun!!

Application.bmx


oggsaver.bmx [axe.mod/oggsaver.mod/oggsaver.bmx]


oggencoder.c [axe.mod/oggsaver.mod/oggencoder.c]



Grisu(Posted 2015) [#35]
Will do my best. :o) Though I caught a cold and don't feel well.

Anyway, when I try to encode ogg with your latest stuff, the app crashes right at the start after I pressed encode.

Debug says MAV:
	Function ProgressCallback:Int(AmountDone:Float,EncoderObj:Object)
		UpdateProgBar(enc_progbar,AmountDone)
		Local Encoder:TEncoder = TEncoder(EncoderObj) <- MAV here!!!!
		Return Encoder.GetCancel()
	EndFunction


[Edit] Might have to do with my module. Seems to work now again. Testing goes on. The evening is still young. :)

[Edit2] More fun, the oo-structure completely truns my code upside down. This is Sparta!


col(Posted 2015) [#36]
The evening is still young. :)

Indeed :-)

Just to plant a seed of thought ;-) .... have you thought about encoding the live stream directly instead of saving to .wav first?


Grisu(Posted 2015) [#37]
That's easy to answer:
- When I started my little app it was never intended to record streams.
- Recording is done internally via FModEx.
- Recording is done over the sound card, so it's compatible over all platforms.
- Wav-Recoding has a lesser CPU-footprint.
- Avoiding license issues.
- And I don't know how. :)

Sooo... after 2 hours I converted the code over and got rid of lots of bloated stuff.

The basic RAM usage went up from ~10 MB to ~20 MB due to threadded compiling which is dislike.

But more importantly the ogg (and mp3 saver?) have memory leaks. The RAM usage goes up to 1 GB and isn't cleared after encoding. I also fear that big wav-files may crash my app due to that.


col(Posted 2015) [#38]
I like the fact that your player is very small and lightweight. Its really cool.
I'm not sure why the ram usage would double under threaded mode just yet.

And yes, I just monitored the memory foot print during encoding too ( 150Mb wav file )....

The mp3 encoder doesn't appear to leak here, the memory usage went up about 1 mb during encoding, stayed rock solid while encoding and actually dropped ( using 3Mb less than it was using before it started ) when it was done. I encoded the same file 5 times during that test. The mp3 encoder is already set up for streaming, also encoding while streaming doesn't load up the cpu any where near as much either.

The Ogg encoder however is a different story and it is indeed leaking like a sieve under a waterfall. I'd think that the contribution to the high memory usage during encoding is because of loading the wav file completely into memory ( as a TAudioSample ) instead of streaming it in. But that doesn't explain why the memory isn't being freed after encoding. Streaming into the ogg encoder in the same way as the mp3 encoder is doing will resolve the memory leak and the high memory usage, I'm sure.

*rolls sleeves up*

the oo-structure completely truns my code upside down

Sooo... after 2 hours I converted the code over and got rid of lots of bloated stuff

any changes to this little app... shall I leave it oo ? or change it back to procedural? make it a module ? or are you going to drop the idea of the encoding?

Hold on... Doesn't FModEx already do encoding while streaming?


Grisu(Posted 2015) [#39]
Hold on... Doesn't FModEx already do encoding while streaming?


Not writing as MP3. The lib offers some Windows Media format, WIIU and XBox360 as output. Not sure what these formats do or look like.

I just rechecked your first module changes above. They aready have the memory leak for ogg.

Hard call. With 16 GB Ram I don't care much about that 10 MB extra. But we need to fix this leak first.


col(Posted 2015) [#40]
Ahh ok, we won't worry about using FModEx then :p

I'll write a new ogg module using ogg 1.3.2 and vorbis 1.3.5 to allow us to encode in the same way as we're encoding to mp3.


Grisu(Posted 2015) [#41]
The newer version works 1.3.x works fine for me.

But I find strange is that the loading of the WAV-file doesn't increase the memory usage for mp3-encoding during my tests. These files get large quite easily.


col(Posted 2015) [#42]
Good isn't it :)
That's because its streaming the file into a small buffer ( uncompressed data ), encoding that small buffer into another smaller buffer ( mp3 data ) then writing to disk. At one any one time there's only a small amount of uncompressed wav data and a small amount of mp3 data in memory. As opposed to the oggsaver requiring the whole uncompressed data to be in memory before encoding ( as a TAudioSample ). To be fair the axe.oggsaver isn't designed to be used for the purpose we're currently using it, it's designed for smaller audio samples that fit comfortably in memory. No worries though as there's a streaming ogg saver on its way that will use very little memory too ;-). A thumbs up I'm sure :)


Grisu(Posted 2015) [#43]
Sounds all too easy. ;)

I was thinking of ways to reduce the memory usage. So vars should only be added until the user presses "encode". So I can keep memory usage low until really needed.


col(Posted 2015) [#44]
Just a little update to let you know that I'll probably be able to get back to this over the weekend. I've not touched it all week, so I'm eager to get it done asap :-)


Grisu(Posted 2015) [#45]
Thanks again for your assistance and feedback Dave.

Would be great to push out a new release over the weekend (aka sunday eve). :)


col(Posted 2015) [#46]
Hiya,

I managed to find a couple of hours this evening...

After your comment about the OO style I changed the code back to the original procedural style. I hope I haven't goofed up your code base again - you did make a backup of the original yes? ;-)

There's a little issue in that I've put the extracted 1.3.2 ogg and the 1.3.5 vorbis in the new module folder, instead of putting them in the pub.mod folder which would otherwise have the risk of being overwritten during official updates. However this then creates a clash of vorbis* functions which then means importing the module manually while not importing the pub.mod version of the ogg decoder. I'm not sure on the best way to resolve this at the moment - any thoughts? I noticed that you use the Framework command so maybe it's not a problem for you?

To get this little app to compile the module file structure should look like this:
/mod/srs.mod/oggencoder.mod/libogg-1.3.2/*all ogg files*
/mod/srs.mod/oggencoder.mod/libvorbis-1.3.5/*all vorbis files*
/mod/srs.mod/oggencoder.mod/oggencoder.bmx
/mod/srs.mod/oggencoder.mod/oggencoder.c


Application


srs.oggencoder [/mod/srs.mod/oggencoder.mod/oggencoder.bmx]


oggencoder.c [/mod/srs.mod/oggencoder.mod/oggencoder.c



Grisu(Posted 2015) [#47]
Hi,

I think it's not a bad idea to split the module from the original one. And adding a Framework should be normal for many users.

I managed to convert your latest code into my main branch. Testings so far went well. No memory issue at all. Even when Playing + Recording + Encoding different test stations at the same time, the app remained stable. Not sure how well it will run on lower end machines though. ;)

Thanks again for all your help!

[Edit] New release with a fancy name is up. :o)



Stefan


col(Posted 2015) [#48]
Excellent release :-)

Works faultlessly here too and on my works pc.
It's a good simple example and practical use of threading which would otherwise make the application laggy.


Grisu(Posted 2015) [#49]
Thanks for the feedback Dave!

It's just a small app, but I hope it displayed some efforts, progress and time put into it. - It also might be the last release as bmx is now Abandonware.


col(Posted 2015) [#50]
It also might be the last release as bmx is now Abandonware.


Surely not? 'C' hasn't been updated for a loooong time and there's still plenty of applications that are written using it, including complete OSs. There's nothing wrong with BlitzMax at all as it is, who cares that it's gone open source? Your version still works yes? :^)


Grisu(Posted 2016) [#51]
Sorry to bring up this necro-thread. :(

The code above doesn't seem to work with WAV files larger than 2 GB. Doesn't matter, if I use ogg or mp3-encoding. I don't get an error message. It just runs through 100% (within a second) and the created file is broken.

Is there a way to fix that?


TomToad(Posted 2016) [#52]
2 GB? A full audio CD is only about 650 MB. Maybe break down your file into individual "tracks", then encode.

I'd imagine the 2GB limit has to do with using an Int pointer somewhere in the .dll instead of Long. That was the reason why FAT32 was limited to only 2GB files.


Grisu(Posted 2016) [#53]
My app writes a raw WAV file to the hard drive. Some of my users record longer sessions. You reach the 2 GB wall after around 3 hours or so. For breaking down the file into tracks I would have to analyse the data which is even more complicated. In addition most tracks are faded into each other so cutting them is nearly useless.


Derron(Posted 2016) [#54]
2 GB ...isn't this a 32bit thing?

So if integers are used, you run into a barrier then.

Maybe "the ones" with knowlegde are able to shed some light.


bye
Ron


col(Posted 2016) [#55]
Hey all!

I'll get a test installation up and running tomorrow to see about the large file size encoding.

I'd tend to agree with everyone else regarding the 32bit limit and I think it'll be just the signed 32int integer maximum value being reached. My *initial* thoughts are that changing some Int variables to Long will give you 4GB support and better file size handling should allow up to I think 16TB.


col(Posted 2016) [#56]
Yep, its the 32bit signed integer limit thats the problem. Also a .wav file larger than 4gb has its header size parameter set to -1 (32bit).

What makes things even more interesting is that the brl filesystem module is 32bit only too :D I know Brucey has a filesystemex module that depends on his boost module which obviously depends on the boost library - maybe a bit heavy weight just to know the file size?


TomToad(Posted 2016) [#57]
instead of saving as .wav, try saving as raw PCM data. Seems LAME-Enc will accept raw data. Also could try compressing the data as it streams to the encoder, so no need for raw data to be saved in the first place.


col(Posted 2016) [#58]
Hiya TomToad,

.wav is a simple container for raw pcm already. The only difference being that the wav header describes the data, and a raw pcm file contains just the data only.

Also could try compressing the data as it streams to the encoder

The encoder is the compressor :p

I think encoding on the fly is the way to go, and coupled with large file support you could store days of music in one file :-)
I did briefly mention it up at post #36.


Grisu(Posted 2016) [#59]
In my naive world I thought that the current code already reads out chunks of data. So I didn't think of bigger files being a problem. Until I got multiple user requests that wanted to use larger WAV files.

It would be elegant, if FMOD would write the streams directly as MP3 to disk. But such a feature doesn't exist.

I wouldn't mind, if the chunks encoded would be larger. Because tiny ones are slowing down the process. I already changed the code so that it only updates the progress bar in 1%-streps.


col(Posted 2016) [#60]
But such a feature doesn't exist.

Not yet, but we can make it exist... in your app at least for the time being.

I'm been playing with fmod and I can get it to encode the live stream without the need of going to a .wav file first.

I assume you have some existing BlitzMax code to interface with the dll otherwise you wouldn't have it working already, I don't fancy writing the interface code if you already have it done ;-)

Are you using Bruceys module?


Grisu(Posted 2016) [#61]
I do. Using it for years now. Updated it to the latest Fmod 4 release that is.

Would it help, if I send you my apps source code?


Brucey(Posted 2016) [#62]
I know Brucey has a filesystemex module that depends on his boost module

NG has large file support as standard... :-)
(I've used TStream to process files in excess of 50GB)

But Grisu's code needs some work to become completely compatible - given the bad Windows API integration he is using (Ints <> pointers, etc).


col(Posted 2016) [#63]
NG has large file support as standard... :-)

Yeah I know ;-)

Would it help, if I send you my apps source code?

Nah, It's ok, I tested with Bruceys Fmod4 module too.

I've reworked our little simple app above for the updates.

There are a couple of small tweaks needed in Bruceys module. The callback functions should be stdcall so Lines 2169, 2178 and 2187 of fmod.bmx from https://github.com/maxmods/bah.mod/tree/master/fmod.mod need the "Win32" declaration on the end of the parameters after the last parenthesis.

It was nasty to get this working with callbacks but don't we love a challenge eh.
There's an issue with the debug stack when running in debug mode when using FMOD callbacks. It shows the classic signs of the function calling convention being wrong somewhere. I *think* they are setup correctly now but it won't run in debug mode at all, so you have to run it in threaded release builds only. Maybe the fault will show up or it may just be because of the callback and the 'Max GC doesn't enjoy it in coming in from the 'backdoor' :D

I've hardcoded a net radio station ip so you'll need to be connected to 'net to try the app. I've had it encoding for 2 hours straight to ogg followed immediately by another 2 hours to mp3 and it's working lovely here - between 1.0 and 1.5% cpu usage while encoding live streams :-)

See how you get on with testing...




Grisu(Posted 2016) [#64]
Hey, thank you for looking into this!

Done some testing over the week. Ripped out the old code, made the new one "super strict" and also replaced the whole "fmod overhead".



A new test build of my player is available: http://www.mediafire.com/download/ii4r8r474sbdl4u/prp_win32_test.zip

So far it works nicely. Though I'm happy to get some more user feedback / from different hardware.

The debug mode crash is the only thing that bothers me a bit. I get a random crash after some minutes of ogg recording. But I can't track it when I can't use the debug mode.


col(Posted 2016) [#65]
I get a random crash after some minutes of ogg recording

I left it playing all night without encoding and it crashed here too. So far I've managed to get it working in debug mode by using the old debugger but it still bugs out after about 10 mins.

There's plenty of hints as to where its going wrong, the callback is the main culprit and all hell breaks loose when the code is multple stack frames deep into the mt debugger. Something changes and it looks as thought the GC might not holding a reference to an instance somewhere. I may look at another way to get at the samples, maybe even a different api.

More investigations...


Brucey(Posted 2016) [#66]
The legacy GC doesn't like it if you callback into Max from a separate thread - like, for example if you are doing it from FMOD.

Either :
1) Don't do any GC-related work from the callback
2) Don't use the legacy GC ;-)


Grisu(Posted 2016) [#67]
I'm still using the old GC as I lack the skills and time to make a full 64 Bit transfer.

After about 10 mins it crashes for me when running a release build (with ogg encoding) here.


Derron(Posted 2016) [#68]
NG does also compile to 32 bit targets ...

So you would need to transfer your "int" to "byte ptr" (handlers) and of course your code must be at least strict.
Dunno if all your used modules are compatible to NG ...



bye
Ron


col(Posted 2016) [#69]
I have the little sample app running in NG in x64 on windows :-)
It took about 2 mins and only half a dozen modifications.

It has the exact same fault, so either I'm using Fmod wrong or there's something else somewhere in Fmod. I notice that the sample length in the callback starts out as 13824 in length and then changes to a length of 4608. I believe the first packets are a 'pre-buffering' of the incoming music, filling the internal buffers, so that when you want it to play then it will play instantly. As the music is streamed at a more regular pace then the data length drops down to 4608. This is quite normal for this kind of thing.

When in debug mode the crash happens exactly at the point that the buffer size changes.

Any ideas?


col(Posted 2016) [#70]
In release mode the NG version fails after a few minutes with an error 'collecting from wrong thread'. It seems FMod is just proving to be a pain in the a$$ with garbage collectors.


Derron(Posted 2016) [#71]
That "Collecting from unknown thread" is a message from the GC... (ng... blitz.mod/bdwgc/pthread_stop_world.c) so you might be right, that GC and your code (maybe FMOD) do not play well together.

In this Thread Skidracer/Nitro/Simonarmstrong suggested to code your callback in C and to have a look at brl.system on how to communicate with your app.


bye
Ron


col(Posted 2016) [#72]
Thanks Derron,
funny that Fmod is mentioned in that thread too.

Thanks for the pointer to brl.system.


Grisu(Posted 2016) [#73]
If everything else fails; I can also live the the extra manual encoding option. So you don't need to mess around with FMod directly.


col(Posted 2016) [#74]
Bumping...

Getting back on to this over the next couple of days...


Grisu(Posted 2016) [#75]
Thanks again for helping me out with this.


col(Posted 2016) [#76]
Hiya,

A little progress over the past evenings and today. Mp3 encoding is done and seems to be working well.
If you want to do some Mp3 testing the current code without Ogg encoding is on github at https://github.com/SRSSoftware/FModEncoder

What are you thoughts on using the latest libmp3lame.dll?
My 2c...
I changed to use the latest libmp3lame dll as it only seemed right being as lame will keep things up to date if needed. The blade dll, which uses a version of lame, could possible fall behind if lame gets updated. It's LGPL so linking via the dll allows you to keep your source closed.


Grisu(Posted 2016) [#77]
I'm away till the end of next week so I can't check it out now.

Sure, drop the Blade support. It will make the code more future-proof and it will be easier to update/fix.

I'm going to give the users 3 options:
A. Wav encoding
B. Ogg encoding (default)
C. MP3 encoding (optional/external)

BTW. Does the 2 GB filesize limit still apply to your new code?


col(Posted 2016) [#78]
I'm the example... yes the 2gb limit applies. I'll get it to support large files. Saying that I encoded in ogg at 320kbs for 16 hrs and it was just under 2gb file size. The cpu was at 1%, using 7mb while encoding in debug mode.


Grisu(Posted 2016) [#79]
Should be "long" enough for most cases. Will the app just quit recording when the limit is reached? I don't want it to cash.


col(Posted 2016) [#80]
Will the app just quit recording when the limit is reached?

I'd imagine that with large file support you'd be able to record for months before the limit is reached :-) but I'll put some checks in to stop the recording.


Grisu(Posted 2016) [#81]
My provider usually disconnects after 48h, so I won't be able to check that. But I'll create a test build for the users that wanted this kind of feature.

P.S. Do you think you can finalise the code until next monday?


col(Posted 2016) [#82]
Do you think you can finalise the code until next monday?

I would think so yes, a release candidate at least.


Grisu(Posted 2016) [#83]
That would be great as I'm going to be back in monday and spend the week on PRP. Please tell me when you have updated the code by then.


col(Posted 2016) [#84]
Hey Grisu,
ogg encoding is now in the repo


Grisu(Posted 2016) [#85]
Thanks for the update, will be able to check it out on monday.


Grisu(Posted 2016) [#86]
I got you latest code and recompiled all modules (normal, debug, threaded).

When trying to run your example app I get a MAV error for "NetRadio.Init()". :(


col(Posted 2016) [#87]
Ahh you probably need a different version of MingW. I used this one from Tom Toads guide, 'no 2 the old way'

Don't forget to rebuild all modules so that they are all built with the same mingw.


Grisu(Posted 2016) [#88]
I'm using GCC 5.1.0: https://sourceforge.net/projects/tdm-gcc/files/TDM-GCC%20Installer/tdm-gcc-5.1.0-3.exe/download

It has a better runtime and compiling speed. I really don't want to downgrade to 4.6.x.


col(Posted 2016) [#89]
I'll look into compiling it with that version.


Grisu(Posted 2016) [#90]
Hi, I'm now using the lasted build from the thread you posted before: http://www.blitzbasic.com/Community/post.php?topic=105745&post=1293859

With this build it, I get a proper error message:
[ 46%] Compiling:fmodencoder.cpp
[ 46%] Compiling:fmodencodercodec_mp3.cpp
[ 47%] Compiling:fmodencodercodec_ogg.cpp
[ 47%] Compiling:fmodencodercodec_wav.cpp
[ 47%] Compiling:bitwise.c
[ 47%] Compiling:framing.c
[ 47%] Compiling:analysis.c
[ 47%] Compiling:bitrate.c
[ 47%] Compiling:block.c
[ 47%] Compiling:codebook.c
[ 48%] Compiling:envelope.c
[ 48%] Compiling:floor0.c
[ 48%] Compiling:floor1.c
[ 48%] Compiling:info.c
[ 48%] Compiling:lpc.c
[ 48%] Compiling:lsp.c
[ 48%] Compiling:mapping0.c
[ 48%] Compiling:mdct.c
[ 49%] Compiling:psy.c
[ 49%] Compiling:registry.c
[ 49%] Compiling:res0.c
[ 49%] Compiling:sharedbook.c
[ 49%] Compiling:smallft.c
[ 49%] Compiling:synthesis.c
[ 49%] Compiling:vorbisenc.c
[ 49%] Compiling:vorbisfile.c
[ 50%] Compiling:window.c
[ 50%] Compiling:bitwise.c
[ 50%] Compiling:framing.c
C:/BlitzMax/mod/srs.mod/fmodencoder.mod/fmodencoder.cpp:2:26: error: token "=" is not valid in preprocessor expressions
 #define _FILE_OFFSET_BITS=64
                          ^
C:/BlitzMax/mod/srs.mod/fmodencoder.mod/fmodencoder.cpp:2:26: error: token "=" is not valid in preprocessor expressions
 #define _FILE_OFFSET_BITS=64
                          ^
C:/BlitzMax/mod/srs.mod/fmodencoder.mod/fmodencoder.cpp:2:26: error: token "=" is not valid in preprocessor expressions
 #define _FILE_OFFSET_BITS=64
                          ^
[ 50%] Compiling:analysis.c
[ 50%] Compiling:bitrate.c
[ 50%] Compiling:block.c
[ 50%] Compiling:codebook.c
[ 50%] Compiling:envelope.c
[ 51%] Compiling:floor0.c
[ 51%] Compiling:floor1.c
[ 51%] Compiling:info.c
[ 51%] Compiling:lpc.c
[ 51%] Compiling:lsp.c
[ 51%] Compiling:mapping0.c
[ 51%] Compiling:mdct.c
[ 51%] Compiling:psy.c
Build Error: failed to compile (1) C:/BlitzMax/mod/srs.mod/fmodencoder.mod/fmodencoder.cpp


Does this help to get to core of this?

Grisu


col(Posted 2016) [#91]
Using the exact same version of MingW in your link, it turns out to be the calling convention for interfaces in cpp, lol.

I've updated it on the repo.

edit: large file support isn't finalised just yet, that '#define' shouldnt have been in there :p
edit2: are you testing with the legacy bmax 150 ( on account of your copy/paste )? none of this has been tested in ng at the mo. Let me know which one you want to prioritise for...


Grisu(Posted 2016) [#92]
I only switched to get you some error message. Because the legacy compiler crashes without any error message at all.

Switched back to my old setup (legacy BMX + core modules updates) + GCC 5.x.

It seems to work now. Encoding is in progress. ;)

Now I have to get rid off all the PRP code / gui and put the new stuff in. Will keep you updated.


Grisu(Posted 2016) [#93]
Do these functions need to be in C code?

NetRadio.Init()
NetRadio.SetRadioStation("http://relay.181.fm:8030")

I'd love to set the values, such as the volume fmod flag myself. For instance the recording channel will mostly run on volume=0.

Also, is the "libmp3lame.dll" set for a reason. Most dlls you find on the net are called "lame_enc.dll"? Or is there a difference between them?


col(Posted 2016) [#94]
Can you not supply the dll with your app?

The fmod functionality is all kept in the c files to keep the structure of the code under control. I guess you can write some getters to get the fmod system, sound and channel instances and then call the FMOD** functions directly. A tidier solution is always to wrap the low level stuff away from the regular 'max code. If you make a list a of the functions that you use then I'll put add them in, feel free to email me if you want to discuss some of your closed source stuff.

[edited due to predictive text getting it wrong :D]


col(Posted 2016) [#95]
I found some time earlier to include wav file output.


Grisu(Posted 2016) [#96]
Yeah noticed that, though wav file output was already supported. You can use the "normal" fmod commands for that.

Did you get my mail? :)


col(Posted 2016) [#97]
wav file output was already supported

oh well it makes this little app more complete :D

Did you get my mail?

Yep :-)