Garbage Collector Issues with Lua Scripting
BlitzMax Forums/BlitzMax Programming/Garbage Collector Issues with Lua Scripting
| ||
| I've been playing more with my script engine and I think I've found some issues with the GC and Lua. When I first started reading up on Lua with BMax, I found a lot of information about being careful that Lua doesn't have objects which have gone out of scope in BlitzMax, because the GC won't manage Lua references to objects. Fair enough, I thought, makes sense. I seem to have the reverse problem though. BlitzMax *is* reference counting my Lua references to objects, and I don't want it to. The Lua VM has the ability to get lots of objects, for various purposes. I intentionally want as few restrictions on scripting as possible. I certainly *don't* want to force people to manage their objects properly. They have a right to expect me to do that for them. I mean it's not like they're actually creating objects. They're only retrieving handles for existing objects. It'd be very rude of me to expect them to free something they didn't even create. But how can I manage objects I don't know they have? Well ok, I know they have them, but I don't know where they are or how to break the references. I can't even fiddle with the internal reference count ( not that I would want to! ) because I have no way of determining whether a second call to get an object handle is going into a second variable or back into the first one. I'm already destroying the VM when the program finishes, but even that doesn't seem to be enough. the GC still seems to think there are references which exist, but I don't have access to them to free them. There has to be a better way to manage this? EDIT: I'm sending objects as LightUserData via HandleFromObject(), if that makes any difference. In Lua, the LightUserData is stored in a table. |
| ||
| Looking through again, it seems I was misinformed when I was told that HandleFromObject is not managed. Ok, so if I just call Release() on the handle as I send it then that should resolve the issue, right? Now it will only break if I try to use an object in Lua which has already been destroyed, right? EDIT: No, of course it won't. If the handle is Release'd, BlitzMax won't convert it back into an object even though the object still exists. |
| ||
| You will have to try it and find it out. Not many were "crazy" enough and tried to bind BM objects to something outside BM. The only one I know is Koriolis with BriskVM ... |
| ||
from the docs:Function HandleFromObject( obj:Object ) Returns An integer object handle Description Convert object to integer handle Information After converting an object to an integer handle, you must later release it using the Release command. |
| ||
| Skidracer: Yes, I referred to that in my second comment. It doesn't really help me very much though. There's no practical way for me to know when I can safely call Release. Dreamora: Yes, that sounds about right. If ever there are only two people stupid enough to do something, you can usually bet I'll be one of them. |
| ||
| Skidracer: Yes, I referred to that in my second comment. It doesn't really help me very much though. There's no practical way for me to know when I can safely call Release. This is more of an issue with you not implementing your handling of objects between the application and script safely. Your objects should have their own code to determine when they are still in use by Lua. If they are, Lua should call a function to tell the host that it's safe to release the handle. If not, you simply don't release the handle. |
| ||
| That's my point though. The idea of having scripting was to let people mod the game freely and easily. Making them call functions to release handles with every object they use isn't free or easy. They're inherently safe because the scope of scripts and objects is carefully managed by the engine itself. |
| ||
| Right, I think I've found a solution. It's not pretty, but I don't see a better solution. If anyone ever hits the same problem, here's my solution. Type ScriptableObject Field IntHandle:Int Method Destroy() If IntHandle <> 0 Release(IntHandle) End If End Method Method GetIntHandle:Int() If IntHandle = 0 Self.IntHandle = HandleFromObject(Self) End If Return IntHandle End Method End Type Anything you want to send as an object to Lua, has to Extend ScriptableObject. The ScriptManager object calls GetIntHandle() on all objects when sending them to get the lightuserdata which you pass to Lua. It also has to call Super.Destroy() at some point when you want to get rid of it as well. Sadly, you won't be able to tuck that away in a delete method because it will never get called. You'll have to implement an explicit destructor, which is not ideal, but it's the cleanest solution I can find. |
| ||
| As a general rule you want to use regular userdata instead of the light variety. Userdata has the advantage that when the Lua GC collects it you can specify a destructor to free your resources. Lightuserdata is only a pointer. You can use it to pass userdata to the host. |
| ||
| I would actually make this a function. Why? it does not operate on the object and therefor allow the GC to take care of it (not possible with a method until it finished). I just realized that this would be a way to solve the problem you had before as well (with weak references) The main problem here is: HandleFromObject and the vice versa are EXTREMELY SLOW so don't do it with speed critical objects ... When I am not wrong, this 2 functions are mainly there for backwards compatibility of Blitz3D / BlitzPlus ... This is the main reason actually why using Int handles for banks for example drops the performance to 50% of banks and the like ... if you want to speed up that you could as well do your own mapping. A global array in the class which stores the object handles and you just send the array index as ID to lua and instead of release you just NULL it. This would even free you from the need to handle the reference in a list -> weak references for lua objects could theoretically be done this way. |
| ||
| I've actually moved over to managing most of my resources in the way you describe, only using a hash table and referencing objects by name instead of by integer index. There are a few objects, mainly HUD and GUI stuff which don't really suit being managed in this way though, and those are the ones I'm referring to here. I can't say as I've seen any improvement in speed though, so I'm not sure that HandleFromObject was particularly slow. Or perhaps in the limited context that these are being used it just wasn't going to matter either way. With regard to userdata vs lightuserdata, it sounds good in theory, but I'm pretty sure I only came to use lightuserdata because there was a big problem when using userdata. So I'll give it a look, but I think it might not work for what I want. |
   |