NG: How to import .a libs? aka. Newton Begging
BlitzMax Forums/BlitzMax NG/NG: How to import .a libs? aka. Newton Begging
| ||
| Is it possible to import/use .a libraries in bmx-ng? I've tried putting my .a lib in a /lib folder (I notice it doesn't exist by default) -- no joy there -- so am instead trying to load it from the local source directory, but using Import "libode_single.a" gives a bunch of "undefined references" (using 32-bit build with 32-bit .a), while Import "-lode_single.a" just says "cannot find -lode_single.a". Here's a test archive: http://www.hi-toro.com/temp/odetest.zip I'm basically trying to get ODE working at the most basic level -- the DLL version works fine with the two test functions, but it would be nice to be able to use the more cross-platform-ish .a method. |
| ||
| If I remember correctly, I have a previous post somewhere on how to do static linking, try doing a search here on this category. |
| ||
| I've had a good go at searching, but although there's the odd reference, I can't find anything that actually goes into static linking in any way (that's the term I needed!). http://www.blitzbasic.com/Community/_search.php?bbs=BlitzMax+Modules&forum=&terms=static&method=simple&case=no&results=topic&user=RustyKristi&submit=Search Off to play about again and see if I can get anywhere with this... |
| ||
| Just re-uploaded odetest.zip, as it had the .a from the DLL build, doh! (Guess that's a set of import definitions?) Still giving exactly the same results, though, with the correct static lib, ie. undefined references. Even used the 'nm' tool to list the functions (_dWorldCreate and dWorldDestroy both exist, tried defining as such in the Blitz code to no avail). |
| ||
| I've pushed an update to bmk, which should allow for direct imports of .a files now. The other way to do it is to put it in a module, and setup an LD_OPTS path, and a -l import. |
| ||
| Cool, will have a go later -- thanks, Brucey. Started writing a wrapper for ODE (was just going to use the DLL interface), so once I've got the C API basically converted to BMX I'll deal with the interface stuff. Mind you, a proper Brucey wrapper wouldn't go amiss! Was surprised to find there isn't really an open, cross-platform 3D physics library out there. |
| ||
| You know how to bait Brucey: give him a "known library" and "not wrapped for BlitzMax yet"... and you have really good chances to see it wrapped some time later... :-) bye Ron |
| ||
| Ha ha, worth a try! |
| ||
| and you have really good chances to see it wrapped some time later Nah. I have an unreleased Newton Dynamics module here. They both seem to do the same kind of thing, so there's little point me investing time in another one. |
| ||
| The most important keyword here is "unreleased" :-p bye Ron |
| ||
| Fair enough... so, obvious question: any plans to release your Newton mod?! I'm still plugging away with ODE anyway -- pretty simple C API, so hoping I can piece enough together to get it working... ish. Looking good so far! Good thing about ODE (other than being plain C interface-wise) is that there is previous Blitz history, with Blitz3D and the JV-ODE wrapper examples, which gives me (largely-unfounded) hope that I might be able to make it do something. |
| ||
| I've pushed an update to bmk, which should allow for direct imports of .a files now. nice! this should make importing much easier. :-) |
| ||
| Woo-hoo, got my ODE converter to output all of the C API and build a test using the DLL interface. Whether or not it'll actually work in practice (or if my C++ -> bmx type conversions are any good) remains to be seen. Finding out if there's much involved in translating ODE's world orientation to B3D's orientation is another matter entirely... Will have a go at making it work with static linking shortly. |
| ||
| Egads, there's a big difference between being able to call the plain C functions and actually doing anything that involves bmx/C++ interfacing, particularly dealing with structs... this stuff is clearly beyond my ken! :/ |
| ||
| That's nice James. Are you trying to use ODE with OpenB3D? Just curious and on a side note, do you have any examples or demo with ragdolls and ODE? thanks |
| ||
| No, I was hoping to, but see last comment -- failed dismally at the, er, hard part. We clearly need a man like Brucey to bring us a proper port! (Is your Newton port publish-able, Brucey, or just a private thing?) |
| ||
| Er, note to self... updating bmx-ng going great, except for, er, "vulkan.mod" crash on running openB3D example! Where the heck did that come from?! :/ |
| ||
| Hey James, I have all of the latest modules and your speccy example runs fine on a linux machine which uses the openB3D ( gl ) module too. What OS and error do you get with the vulkan module? |
| ||
| Is your Newton port publish-able um... it's incomplete, but the stuff that's done appears to work - although you'd need to integrate it into... whatever. Since I'm rubbish with 3D, I wouldn't know how to tie the two things together. "vulkan.mod" crash on running openB3D example Do you have some more details on the crash? Where the heck did that come from?! It's a new NG feature... something to do with Star Trek, if I'm not wrong? |
| ||
| Ha ha... I assumed it came from recent openb3d updates, but maybe not... ? Do you have some more details on the crash? Not really -- it builds with my ongoing bmx-ng, but today I finally got bcc, bmk and the pub/brl/sdl/maxgui and openb3d mods to build, and the end result of an openb3d test was that it gave an EXCEPTION_ACCESS_VIOLATION error, nothing more. (Deleted my build/s and .bmx folder.) I get your point about not knowing how to integrate into "whatever", as I imagine the matrices and position (z vs y, etc) could be different, but if you could find it in your noble heart to publish some sort of release, I'd like to at least cluelessly have a pop at getting it to match up with the *B3Ds, at least! I suppose I need to dig into my crash, though... :/ For what it's worth, here's my deadspeccyshader, though you'd probably have to have a matching ng setup to get the exact same problem. In theory, it's all the latest stuff, other than a fix to openb3d.mod\b3dsound.mod\b3dsound.bmx in line 320: HearingPoint = New HearingPoint -> HearingPoint = New ListeningPoint |
| ||
| BTW My latest attempts to build all-new bcc/bmk/pub/brl went remarkably smoothly today, no build errors at all, other than the aforementioned third-party b3dsound error -- all built perfectly, which was f'ing amazing! |
| ||
| No crash here my ole mate. On Linux x64. I'm not sure if this is any good for you but here's a raw port of the newton ( 3 I believe? ) functions that I found except the callbacks are commented out, I can't even remember why :/ Part 1 as the forum takes a dump when its posted as one code. If it no good for you then just throw it away, or Brucey may have even more of it - like the structs/types for eg. |
| ||
| Part2, just copy-paste to the end of part1 |
| ||
| Sorry, col, meant to comment on your comment re. Speccy/Linux last night! Glad/impressed to hear it worked! Nice work on the Newton commands, but I think it effectively puts us where my ODE efforts got us, ie. no doubt able to call the commands, but not actually able to do much with the library due to lack of interfacing with the C/C++ structs. I could be wrong, though, will have a look! (Of course, source-level, Brucey-style would be better, anyway, so we're not limited to Windows/DLL usage of the library.) |
| ||
| ....like the structs/types for eg Yeah, I tend to hide all the raw calls and pointers behind BlitzMax, because we aren't all so hardcore as col ;-) if you could find it in your noble heart to publish some sort of release I'll see if I can get something done this week. It's building from source, rather than using a dll, so I'll need to check it compiles across platforms... |
| ||
| Awesome, thanks Brucey! Going to have a play and see if I can make col's version do anything, but cross-platform sourceyness would be ideal. I'm sure someone will be able to figure out making it work with Blitz3D-style position/rotation if I fail (OK, "when" I fail). |
| ||
| Looking through the api source and a couple of tutorials it's slowly coming back to me :P Newton works via callbacks to set your 3D entity position and when you want to apply forces and stuff. A general overview of events... 1. Use NewtonCreate to get a 'world' instance into which all of your physics bodies will go. 2. Each 3D entity will need a newton collider to represent its collision shape. For a 3D sphere you would NewtonCreateSphere, for a 3D box you would NewtonCreateBox. There are a few different ones that should cover all needed collider shapes. 3. For each collider you need a dynamic body. This is where you may be tripped up as the NewtonCreateBody wants a 4x4 matrix to represent the 3D object initial rotation and position. If OpenB3D can't give you that then I suppose you could create one yourself. 4. You then NewtonBodySetMassMatrix for each newton body. 5. Then setup some callbacks. For each Newton body ( which also means each 3D entity ) use NewtonBodySetForceAndTorqueCallback with a function that will get called when Newton is ready to apply forces and torque. You would need to store the forces in blitzmax side variables so that when the callback gets called from Newton you can then give the variable datas to Newton in this callback function. Also setup a NewtonSetTransformCallback for when Newton wants to tell you of the new rotation/position of the newton body. The NewtonSetTransformCallback will give you a 4x4 matrix that you would use to position and rotate your 3D entity. So if OpenB3D doesn't give you the option of setting your 3D entity rotation and position via a combined 4x4 transformation matrix then I guess you need to extract the rx,ry,rz and tx,ty,tz parts from the matrix and set the 3D entity manually using OpenB3D EntityRotation and EntityPosition ( or equivalent ) commands. Of course you'd probably want to wrap up the Newton body stuff and use NewtonBodySetUserData to set say a 'TNewtonBody instance' to be associated with the actual 'newton body instance', then in the callbacks you can NewtonBodyGetUserData with the 'newton body instance' to get the 'TNewtonBody instance'. Each frame you do a NewtonUpdate. It's advisable to use a FIXED timestep here as opposed to a fluctuating one to keep everything in check. Newton will call the 'apply forces' callback for each 'newton body instance'. After all calculations are done in Newton it will then call the 'set transform' callbacks so that you can set the new rotation and position. As far as the transformation matrix stuff goes if OpenB3D doesn't let you set the rotation and position in 1 hit via a 4x4 matrix then it may be beneficial to dive into the source and write a couple of functions to allow you to do it - you know... just to make things easy(er) ;-) Hope it helps. |
| ||
Good write up! Sounds awfully complicated compared to ODE, but I just had a look inside OpenB3D, and the TEntity type does indeed contain a user-accessible matrix:Type TEntity Global entity_list:TList=CreateList() ' Entity list Field child_list:TList=CreateList() ' Entity list Field parent:TEntity ' returned by GetParent - NULL ' transform Field mat:TMatrix ' returned by EntityX/Y/Z (global) - LoadIdentity Field rotmat:TMatrix ' openb3d: used in EntityPitch/Yaw/Roll (global) - LoadIdentity ... ... and TMatrix contains a pointer to a 4 x 4 matrix and plenty of conversion methods (which may or may not address your last point re. applying a matrix directly): Type TMatrix Field grid:Float Ptr ' array [4,4] - LoadIdentity ... ... not that I really know much about matrix operations, but at least it should be possible to make them talk to each other without too much customisation. |
| ||
| Literally didn't know this existed until now! Blitz3D Newton wrapper and source I assume it's for 1.x, but should still give plenty of insight in to how to make them talk to each other. Weirdly, all of the links still work! Reasonably hopeful that Brucey's wrapper + *B3D will work with a bit of hacking and swearing. |
| ||
| Yeah, I had a look at that before I started work on wrapping it myself. https://github.com/bmx-ng/newton.mod There are currently two examples, which cover the newton starter tutorials. |
| ||
| Wow, thanks, Brucey. I've literally spent 2-3 hours playing about with the various Blitz3D physics wrappers -- newton and bullet, plus other bits and pieces -- all a lot of fun, so will go have a play with this right now. Really appreciate it! |
| ||
| I'm still padding out the API. Let me know what stuff you need next (that I've yet to implement) and I'll do that next. Although the API is not too large, it's all wrapped up in BlitzMax types, which is intended to make using it easier. So it takes a bit more work than just generating a list of function pointers :-p If you didn't need legacy BlitzMax support, I'd probably change the TNMatrix to a Struct and leave it out of the GC... |
| ||
| I don't need legacy support, personally -- my BlitzMax experience is now totally NG. Although the API is not too large, it's all wrapped up in BlitzMax types, which is intended to make using it easier. So it takes a bit more work than just generating a list of function pointers :-p Definitely noticing that after my attempt, but nice to know! First thing I need to try is to get something visual to work. The two examples certainly work fine here, and I reckon I've a fair bit to work with now to point me in the right direction... which doesn't necessarily mean success! |
| ||
| It's probably easy enough to plug into OpenB3D if you know what you are doing.. unfortunately, I don't... If someone ever builds a small example, I can include that in the distro... or we could end up with, say, a newton-openb3d module, which takes on the nitty gritty work itself... and you just use it with your 3D stuff. |
| ||
| I may have drunk too much alcohol (I'm on holi-holi-day!), but will have a pop tonight/tomorrow. Can't be *that* hard to align a B3D entity to a Newton body, especially with all those 3rd-party examples out there! (He said.) |
| ||
| Much of a hangover? :D Cool stuff :-) In theory then you should be able to set the OpenB3D matrix from the one returned by newton and that's it. If it doesn't work first time then you may be that you need to transpose it - it could be column-major or row-major, that just depends on the Newton implementation vs the OpenB3D implementation of matrices. |
| ||
| Yeah, I'm kind of hoping it will be that simple (should find out in a bit) and not too bad, hangover-wise, ahem... :P |
| ||
| @James A big fry-up normally fixes me up if I feel a little rough :p @Brucey... Newton uses multiple threads which don't play well with the GC. Adding in ModuleInfo "CC_OPTS: -DDG_USE_THREAD_EMULATION" to newton.mod/dynamics.bmx should fix that problem. @all Here's a simple thrown together example for you to show one way to do it. With some textures, colour, a timer, make a variety of 'game boards' some rocking music... *ahem* yeah... then it could well pass up for the basis for a game ;-) |
| ||
| Newton uses multiple threads which don't play well with the GC. NG's GC? I thought Boehm was pretty solid with threading? |
| ||
| Hey Dave, question about your openb3d, are you using Munch's or Brucey's version? |
| ||
| NG's GC? For sure, yes. I get gc issues with 3rd party libraries that spawn their own threads which then callback into BMax code from those 'untracked' threads, it's the same for threaded and non-threaded builds. I think the GC error mentions that its being called from an unknown thread or something similar. I'v never really worried about it. In the past I can still create a new Blitz thread in which I then use the single threaded 3rd party lib, so any app can still be multi-threaded. Other than that I've had to keep the 3rd party accessing code totally separate from 'Max code - NG or Legacy. @RustiKristi The same one that BlitzSupport posted for his speccy shader demo. It has a copyright with Bruceys name in it so I'd assume it's Bruceys version ;-) |
| ||
| Right. I'm just confused with the openb3d. So far this is what I understand OpenB3D (angros47) Main Project (ported on Blitzmax/PB,etc) see below OpenB3D Wrapper (Munch) discontinued, latest. works great on vanilla OpenB3D Wrapper Bah (Munch/Brucey) Less files, compatible with NG In short, is munch the vanilla version and Brucey's on NG? |
| ||
| are you using Munch's or Brucey's version? I really need to update mine to get it more in-line with the work that's been done on the other one. |
| ||
| Right. I'm just confused with the openb3d. Yep, that's one of the joys of open source ;-) |
| ||
| I really need to update mine to get it more in-line with the work that's been done on the other one. Mind you though, the latest commit is an unfinished attempt for GLES, so find the working commit a few steps back. Yep, that's one of the joys of open source ;-) haha yes this happens a lot. :D |
| ||
| Weird, I didn't know Brucey had an OpenB3D version! Mine's the one from here, specifically the commit highlighted by RK a while ago: http://www.blitzbasic.com/Community/post.php?topic=106512&post=1311173 Great job there, col, looks like I was actually on the right track with MemCopying the matrix earlier (had to go out), so I'm impressed with myself too! Will dissect with interest, many thanks to all... |
| ||
| -DDG_USE_THREAD_EMULATION I'm thinking for NG, I can change Newton (dgThread.h/cpp) to register its thread with the GC. In theory that would solve the issue of "unknown threads"... |
| ||
| That sounds like a winning solution there Brucey, awesome! One of the other joys of open source eh. so I'm impressed with myself too! As was quoted in Kellys Heroes... 'have a little faith baby, have a little faith' ;-) |
| ||
| Woo-hoo, this is awesome! Heavily based on your example, col, though I've been fiddling about to make things more "me-friendly". This is a bit of a jumble for now, just experimenting... Numbers of 500-1000 cubes are good for my system, gets a bit bogged down at 2000, but this might well be dependent on update rate as well. [* See EDITs below!] Code below, runnable Win32 executables (32- and 64-bit) here: http://www.hi-toro.com/blitz/newtontest.zip Thanks again, guys, great new toy! A lot to figure out, but this was encouraging... If you set NUM_CUBES into the tens of thousands, it can take minutes to set up (exponentially slower as more cubes are added, guess as it cross-checks each body). I've had 20,000 going successfully -- not quickly -- and had a crash at 50,000, though I suspect that might have been OpenB3D, not sure, as it 'built' all 50,000 cubes before crashing. EDIT: Just realised having thousands of shadows on the go probably doesn't help the frame-rate! Comment-out CreateShadow cube.mesh in the AddCubes function to test without. EDIT 2: Yeah, CreateShadow was definitely slowing down the creation process, and I currently have 50,000 cubes on the go with shadows disabled! Runs in seconds-per-frame at first, then picks up to 'a few frames per second'... and uses 1.7 GB of RAM! |
| ||
| Further to EDITs 1 and 2: Tested with 100,000 cubes, 5.4 GB of RAM! Technically, it runs. Impressive test of both Newton and OpenB3D, if you ask me. |
| ||
| Dynamically add/remove cubes... seems to work, and learned to invalidate world after removing objects! (See comment inside If KeyHit (KEY_MINUS) check.) Win32 32- and 64-bit executables. |
| ||
| Here's my current state of play: http://www.hi-toro.com/blitz/newtonb3d.zip Ignore '*.mod' folders, not currently a module (aborted for now as PITA) -- just drill down into the /examples folder, run example.bmx source or 32/64-bit executables! Usage: Hit the [+] and [-] keys a few times to get started, then read the on-screen text! |
   |