LIBVLC + video frame capture userlib
Blitz3D Forums/Blitz3D Userlibs/LIBVLC + video frame capture userlib
| ||
| LAST UPDATED: 23 MAR 2016 Created this for copying LIBVLC rendered video frames to textures or images in Blitz3d. Putting it here so as to not pollute the codearchive where it originated (Use VLC DLL to play video). This userlib has the same requirements as the one in the code archive, libvlc.dll, libvlccore.dll and Plugins folder. See https://www.videolan.org/developers/vlc/doc/doxygen/html/group__libvlc.html for more info on the VLC api. Note that this is for VLC 2.2.2, though it may work with higher versions, unless VLC 3.0.0+ removes deprecated functions. The packages below contains all that is needed to play videos except the vlc dlls and plugins. Download Blitz3D: libvlc_bb.7z (includes source) Download BlitzMax: grb.libvlc.mod.7z (includes source) [NOTE: Pure VLC api, nothing is added] Old deprecated userlib: hwndbuffer.7z (includes source) The below is only the Blitz3D files, the DLL and its source is in the archive above. libvlc_capture_test.bb libvlc_bb.decls libvlc.bb |
| ||
| Fixed a memory leak, and a bug that would exit the application when free_hidden_window() was called. |
| ||
| Can we get a total count of frames as well as the volume of the video so we can set the volume & get a time count of the video? Also, loadurl should be working, but it's not. :( How to scale the texture to the same size as the object so there's no black screen on the object? The Video stops at the end of the video on the last frame then freezes, and doesn't go back to black :( Thanks alot, grable! =) ~GF |
| ||
| The LIBVLC api has tons of functions, the ones used in the code archive are just enough to display video in a window. Its all here https://www.videolan.org/developers/vlc/doc/doxygen/html/group__libvlc.html Heres a few of them: libvlc_video_get_width%( player%) libvlc_video_get_height%( player%) libvlc_audio_toggle_mute%( player%) libvlc_audio_get_volume%( player%) libvlc_audio_set_volume( player%, volume%) libvlc_media_player_is_playing%( player%) libvlc_media_player_get_position#( player%) libvlc_media_player_set_position( player%, pos#) libvlc_media_parse( media%) I havent found one that gives you the total number of frames or the current frame though, but with libvlc_media_player_get_position and libvlc_media_player_set_position it might not be needed. EDIT: About vlc_load_url(), it plays this url fine: http://www.quirksmode.org/html5/videos/big_buck_bunny.mp4 What are you giving it? You mean the black borders at top and bottom? This can probably be mitigated by setting the hidden window size and texture size to the size of the video, but i havent tested this. Calling libvlc_media_parse will get the needed stats (including size) from the video but it seems to only work for local files, alternatively you can play the video a few frames. I should probably add another function that can resizes the hidden window for this... The reason it stops at the last frame is because it is the last one decoded by libvlc. And blitz3d happily displays it, as it doesnt know that the video has stopped playing. There are some functions in the libvlc api to get events like that, but i havent looked at them yet. Or you can just call libvlc_media_player_is_playing. |
| ||
| Added resize_hidden_window for resizing the window after creation time. Now whether LIBVLC notices this is another matter, it might be needed to stop the player and or reset its hwnd. |
| ||
| Thanks @grable! :) The only problem with the URL code, is that it stops running after a while of watching it, most likely due to a non-constant buffer of the data received from the link. :( Thanks again! ~GF |
| ||
| No problem GF :) I looked into if there was any buffer options, and there is one: vlc_option( "--file-caching", 10000) The second argument is in milliseconds. |
| ||
| Using a separate window and copying from it was a bit hackish, and i wasnt too happy with it. So i rewrote it into using libvlc callbacks, which should be less intrusive and faster to boot :) Updated top post. |
| ||
| Cool, grable! :) Can you add a function to get the length of the video so I can repeat my video? Thanks! ~GF |
| ||
There are functions for position using floats (0.0 - 1.0).libvlc_media_player_get_position#( player%) libvlc_media_player_set_position( player%, pos#)There is also time based versions of them in milliseconds, but they rely on 64-bit numbers so im not sure if they are usable im b3d. libvlc_media_player_get_length%( player%) libvlc_media_player_get_time%( player%) libvlc_media_player_set_time( player%, time%)I see no point in adding them since they are unusable without wrappers, and then you only get 32-bits of the data, not a good thing. There is no repeat in libvlc though, not without creating a playlist and repeating that. And im not sure i want to implement playlists yet ;) But the code below handles it nicely anyway. ; in the main loop before capturing a frame If repeatmode Then If Not vlc_is_playing(vlc) Then vlc_stop(vlc) vlc_play(vlc) EndIf EndIf |
| ||
Added string versions of the libvlc time functions.; gets total length of loaded media in +H:MM:SS format (examples: 0:00:01 123:12:34) libvlc_media_player_get_length$( player%):"libvlc_media_player_get_length@4" ; same as above, but gets the current playing time libvlc_media_player_get_time$( player%):"libvlc_media_player_get_time@4" ; sets the current playing time, effectivly seeking (if supported) libvlc_media_player_set_time%( player%, time$):"libvlc_media_player_set_time@8" |
| ||
| THANK you, Grable! :) You just saved me with this code: PLEASE continue to add functions to this! This is a GREAT project & I HIGHLY encourage it! I wanna be able to stream my media live with a webcam! :) ~GF |
| ||
| Hehe, glad you found a way :) Im not using this myself though, so i wont promise anything. But il probably add things whenever im bored, as the OCD in me doesnt like incomplete things ;) On that note, i added the functions for streaming (VLM as libvlc names them), but i found that it can be a tad unstable. Though this is probably Blitz3Ds fault, your mileage may vary. Also wrapped Track functions, but it was somewhat disappointing as none of the video files i tried it on had any usable meta-data (only bits and pieces). Also added most of the Media Player and Media api functions, so that should be fairly complete, though some of them still need wrapping... There are also some new tests that deal with VLM and Tracks. |
| ||
Why is it returning "no instance" when I did a: DebugLog FileType ( VIDEOFILE$ ) and it returned 1? ~GF |
| ||
| Which test does that happen in? If any of them say "no vlc instance" its because it failed to create a VLC instance, it didnt even get to loading the file. Which shouldnt happen really... Unless perhaps any options passed to libvlc_new() freaks it out or something. EDIT: There was a bug with building options for libvlc_new(), where it always got a count of 0. Which should just have resulted in no options being read by it, but you never know. Updated archive. |
| ||
| Fixed some bugs with Tracks, leading to invalid data. Now its not so disappointing anymore, as all the data is actually correct this time ;) |
| ||
| This is great grable. Is the Blitmax version updated along with this? |
| ||
| @wmaass, as far as I know, this thread is for Blitz3D only. ~GF |
| ||
| Yeah, BlitzMax doesnt need any extra support to handle the various libvlc api functions. But i took and "ported" the ones i have already over to bmx, though its pure api only, and doesnt have all the functions. Note that i only machine translated this and checked that it compiled. So some types may be wrong ;) EDIT: Removed bmx version, download grb.libvlc.7z instead. Its a complete and precompiled module. |
| ||
| Fixed some small bugs, and added functions to get Track Descriptions... Too bad i cant get them to work, as they always return NULL :( I must be missing something, as the videos i have tried do have Audio, Video, Subtitle tracks and named Chapters... Or maybe they have been deprecated in favor of the Track Metadata (even though the "docs" doesnt say so) |
| ||
| Cool, Grable! :) Let me know when you figure it out! Thanks! ~GF |
| ||
| It was a typo in the sample :( Thanks for not letting me know b3d! And i figured that out after reworking the entire thing.. But it wasnt a complete waste of time, as now Blitz3D only loads libvlc_bb.dll. And that in turn statically links to libvlc.dll. Its now acting as forwarder, making so i dont have to manually import each function i use. NOTE: There is no longer a libvlc.decls, everything is in libvlc_bb.decls. |
| ||
| grable, thanks buddy. |
| ||
| No problem wmaass :) Btw, you should probably download grb.libvlc.7z instead. The api is generated from the header files, so its a complete and precompiled module. |
| ||
| Generated all the functions from the header files for the userlib as well. So now all functions and constants are defined. Both libs are updated. BlitzMax version is complete, it doesnt need anything more. The Blitz3D version might need wrapping of some more functions, but since i use very few of them i see no reason to wrap them all. Let me know if there is any functions you need and that needs wrapping and il see what i can do :) |
| ||
| An easy way to test the streaming example would be lovely. :) Preferably, port-forwardable webRTC. Thanks grable! ~GF |
| ||
| Im not sure what you mean.. You want to stream output as webRTC or forward an already streaming webRTC through VLC? In any case, as far as i can tell VLC doesnt output webRTC. And i didnt find any info on webRTC+VLC either, but i dont know anything about webRTC anyway so i wouldnt be any help. What i did when testing was simply: Delay 1000 ExecFile "vlc rtsp://localhost:5544/"Streaming is somewhat unstable too, so its not very fun to work with :/ It seems to be built around listening to events and doing stuff in their handlers, because it uses threads and few of the functions block. Ending up in having to wait arbitrary amounts of time hoping it has done its thing before one can continue... I did make another sample though, which streams and plays at the same time. But it can be a bit fickle ;) EDIT: Also added a YouTube example. |
| ||
| Yea. I wanna stream WebRTC. ~GF |
| ||
| Are there any "how to" examples (for BMX version)? Am I correct that frames from the media can be used as textures? |
| ||
| Yes, @LT. =) @Grable :: Would be lovely if you could add WebRTC webcam streaming! =) Keep up the EXCELLENT work, @grable! ~GF |
| ||
Yes to which question? I can't see any BlitzMax examples. As a quick test, I tried the following:Framework grb.LibVLC Global vlc:TVLCInstance = libvlc_new( 0, Null )But it says: undefined reference to `libvlc_new' Also, is there a way to load the media and use textures without the VLC window? I'm looking at this as a potential replacement for TheoraPlayer. EDIT: Just saw the other thread with another version of BlitzMax VLC. Looks like it has a good example. |
| ||
| The latter question. Yes to being able to use live media / video on a Texture! =) Hope this helps! =) ~GF |
| ||
| @LT The BlitzMax version has one sample "libvlc_trackdesc_test.bmx" which gets a frame as texture and some track information. Dunno why you get undefined reference error though, ALL functions in libvlc has been exposed. For more examples you could check out the B3D version, as it has more of them. I just didnt bother to reimplement them in bmx. Just know that B3D has some extra functions not in libvlc since its kinda limited in what one can do with it. @GF I would not know where to start to be honest, WebRTC is a confusing and overly complicated tech. And with my limited knowledge it seems to require more than libvlc can do... |
| ||
| @grable I wasn't able to get the version here to work, but the one originally posted by BlitzSupport works fine. No reason I can't update that to include all the functions. |
| ||
| @GF I would know where to start to be honest, WebRTC is a confusing and overly complicated tech. And with my limited knowledge it seems to require more than libvlc can do... I meant HTML5 WebRTC. my bad ~GF |
| ||
| @LT Yeah, my bad. It was missing a file ;) Should be fixed now. @GF I figured that, still dunno how to use it though. As far as i can see Its RTSP with some other tech bolted on, but i havent the foggiest how it all works :p |
| ||
| I'd be happy with a server that can STREAM to a webpage coded in Blitz. =) ~GF |
| ||
| Thanks grable, this is what I have been looking for! problem solved. :D |
| ||
| @GF It would require an intermediate web server to re-transmit an existing RTSP/SDP stream to WebRTC, as VLC only does FLV(flash) streaming via HTTP. Its a little out of my field though, and frankly WebRTC is so new (and unfinished) that its hard to find good info on it. Heres a link to something similar, though not using VLC for the streaming it will show how involved it can be ;) http://www.codeproject.com/Articles/800910/Broadcasting-of-a-Video-Stream-from-an-IP-camera-U Personally id just wait until VLC supported WebRTC out of the box, which will probably happen once it stabilizes. |
| ||
| It was missing a file Grabbed new version - still getting the error. :( |
| ||
| Have you tried recompiling it? There may be some differences between the archives that GCC 5.1 (which i use) creates and whatever bintools you have installed or what came with blitzmax. The missing file was libvlc.dll.a which is a copy of libvlc.dll to fool blitzmax into linking to a dll statically. Its getting late, so il have to look into it tomorrow in detail. |
| ||
| ?? libvlc.dll.a wasn't missing. It was in the version I was testing. And yes, I recompiled it. It's not really an issue for me since I can just use the version from the other thread. Still, this version should work... Also, since you mention linking a dll statically, doesn't that mean that any programs that use it would be forced into an LGPL? That's the license LibVLC uses, after all. |
| ||
| Also, since you mention linking a dll statically, doesn't that mean that any programs that use it would be forced into an LGPL? That's the license LibVLC uses, after all. No, it still loads the dll at runtime. It does this by adding it to the import section, meaning the OS loader takes care of it instead of having to LoadLibrary+GetProcAddress yourself.Its what i had to resort to since mingw64-dlltool doesnt produce proper i386 import libraries. But i downloaded mingw32-binutils and ran dlltool again so now libvlc.dll.a is a proper import library, which should work. (crosses fingers) Redownload and see if it works now. |
| ||
| Sorry, still not working. :( I'm testing on Win32, if that helps. |
| ||
| That is weird, im not doing anything out of the ordinary. In fact this is my preferred way of linking to DLLs, by making import libs. All my other modules do the same, and ive never had any trouble with it :( And to be clear, the error you get is "undefined function libvlc_xxx" ? Im gonna download and install vanilla BlitzMax 1.50 and see what happens.. |
| ||
| Just did that, and it worked flawlessly, didnt even have to rebuild the module! So i dont know if its something with your setup or something else. In any case, as far as i can see there is nothing wrong with the module itself. But im gonna try the same on another machine to be 100% sure. |
| ||
| Yes, that's the error I'm getting. I'm using vanilla and Brucey's bmk - perhaps that matters? EDIT: May I ask what the benefit of this version is over the one in the other thread? Is it the addition of the other functions? |
| ||
| It shouldnt matter which bmk you use, even i have modified mine. I just tried on a clean machine with clean bmx and it all worked, even downloaded the module to be sure we are using the same one. So i am at a loss as to what causes that error. Maybe you could try the same? Find a clean machine, install bmx 1.50 on it and copy the mod over and the dlls+plugins and running the supplied test. If not i understand, its not fun when shit doesnt work ;) |
| ||
| May I ask what the benefit of this version is over the one in the other thread? Is it the addition of the other functions? There is no difference other than not having to load the dll manually, and you having to type a whole lot to get the rest of the functions ;)So its not like you MUST use this one, i just dont like it when other people get errors using my stuff when it should work. If it doesnt work for you, maybe others have trouble too so i want to fix it is all. |
| ||
| running the supplied test What test are you referring to? The one I posted above? This is the test I'm running...Framework grb.LibVLC Local vlc:TVLCInstance = libvlc_new( 0, Null ) Afraid I only have one machine available where I am right now. It should be a clean install 'cause I did that pretty recently. Still wondering what advantage there is to this version over the other..? As I said before, not that big of a deal. :) |
| ||
| The test i was referring to is libvlc_trackdesc_test.bmx in the mod folder, it just uses more of the api. But since not even your sample works there is no reason to run it really. Again, the only advantage is that it is already complete and uses the import section. Its just my personal preference to have actual functions and their relevant symbols defined (even if they are trampolines) instead of Global function pointers. And running the resulting executable without the DLL present will cause the OS Loader to complain even before the executable is run. Since it clearly wont work for you, and if you want to save yourself some typing. You could use a regex capable editor (or sed if your on linux) and change all the extern functions into globals doing GetProcAddress in a couple minutes :) Still, i do not like that it doesnt work for you :( .. Things like this really bugs me lol |
| ||
| will cause the OS Loader to complain even before the executable is run Ah yes, it would be nice to avoid that.Have no idea what the problem is. I'm running MinGW 5.1 - does that matter? |
| ||
| Ah yes, it would be nice to avoid that. Why? Unless the functionality is going to be optional youd have to error out yourself either way, otherwise doing the LoadLibrary+GetProcAddress dance is the only way of course.I use MinGW 5.1 (TDM) too, so its not that. And its not using MinGW anyway... That it works in a "clean room" environment should be rather telling IMO. I would suspect your doing something wrong, but i know this aint your first rodeo so thats out ;) The functions are defined in libvlc.dll.a and since its imported they should be there. So most likely there is some mismatch between the defined symbols and what blitzmax tries to load. As a last resort you could try to replace libvlc_api.bmx with this one. Both versions work for me, but i suspect that this one is more correct as the functions arent really stdcall anyway (il update the archive too). |
| ||
| unless the functionality is going to be optional Bingo! I'd like to keep that option open, anyway. :) know this ain't your first rodeo Appreciated, but I'm nonetheless open to that possibility. However, there isn't much to this and no errors are getting reported.Afraid the libvlc_api replacement made no difference. Later, I will try a clean build, but I'm a bit preoccupied at the moment because my Vive arrived literally five minutes ago. ;) |
| ||
| Hehe nice, im guessing youl be off for quite a while then :p Have fun! |
| ||
| Turns out that figuring out what's going wrong here may be key to wrapping the OpenVR module. OpenVR is just a .dll, but most of the functions are in a namespace and I don't know how to GetProcAddress them... :/ |
| ||
| Its probably not related, as C++ symbols cant be exported portably. And OpenVR has a C interface for this very reason, brain dead as it may be: https://github.com/ValveSoftware/openvr/blob/master/headers/openvr_capi.h The DLL only exports these symbols: VR_GetGenericInterface VR_GetInitToken VR_GetStringForHmdError VR_GetVRInitErrorAsEnglishDescription VR_GetVRInitErrorAsSymbol VR_InitInternal VR_IsHmdPresent VR_IsInterfaceVersionValid VR_IsRuntimeInstalled VR_RuntimePath VR_ShutdownInternal VRControlPanel VRDashboardManager VRTrackedCamera |
| ||
| I know, but there must be a way to access the needed functionality. The point of the c api is language bindings. There's a .lib, as well. I'm no expert at the wrapping business, though. Anyway, this probably belongs in another thread. :) |
| ||
| |
| ||
| @Guy Don't know what any of that means. @grable Forget what I said about this being related. My first attempt to access the openvr functions used the same technique that you're using with libvlc (same result). Of course, I can use LoadLibraryA and that works, but only for those base functions you mentioned. Instead, I used the LIB2A utility and converted the .dll (and .lib) to .a and imported that. I'm attempting to wrap more of the functions in an openvr_capi.cpp (with additions to .h), but it's slow going because I have very little knowledge of what I'm doing. :P Any tips would be welcome, of course! |
| ||
| If you guys wrap openVR so that we can make VR games for the Vive, that would be amazing. any luck with that ? |
| ||
| Its weird that the archive you get from LIB2A works, but the archive i generate does not. As LIB2A uses DLLTOOL just as i do with the virtualy the same arguments! You could run DLLTOOL yourself, and see if that helps. This is what i run: set DLL=libvlc.dll dlltool -C -D %DLL% -d %DLL%.def -l %DLL%.a %DLL% -k ranlib %DLL%.a The -C is supposed to produce compatible archives, doesnt make a difference here though. And running RANLIB isnt strictly necessary, it just optimizes the archive. |
| ||
| @grable Here's where I got it... https://code.google.com/archive/p/lib2a/downloads What you're running requires a .def file - where are you getting that? @RifRaf Just started, but this is for BlitzMax. I'll happy to make it available when/if it works. Keep in mind it will be a barebones library and anyone that uses it will have to connect it to whatever 3d engine they're using. NOTE: Created new topic HERE. |
| ||
| it's an example of a DECLs file. ~GF |
| ||
| This is for BlitzMax (sorry for the shift), so I created a new topic HERE. |
| ||
| EWWW! SCREW blitzmax! >< ~GF |
| ||
| @grable FYI, I used LIB2A to convert the libvlc .dll to .a and that did the trick. Using ".dll.a" doesn't work for me at all - on my home or office computers. EDIT: Does not seem to work in Debug mode, either. :( EDIT 2: Problem in Debug mode seems to be the pixmap type. Converting from PF_BGRA8888 to PF_RGBA8888 causes a failure, but works fine in Release mode. EDIT 3: Seems similar problem to this post, except it happens in Debug mode (haven't tried threaded). http://www.blitzmax.com/Community/posts.php?topic=91075#1036635 |
| ||
| I tried compiling the blitz3d exe on one of the examples but it throws out some load errors. This does not happen when I use Run Program. Also how do you save a screen capture to an mp4 file? Does it have a function like that? thanks |
   |