LuGI - Lua Glue Generator & Interface for BlitzMax

Community Forums/Showcase/LuGI - Lua Glue Generator & Interface for BlitzMax

N(Posted 2009) [#1]
Get the LuGI Source Code
(For those of you without Git, just click the 'Download' button next to its name and you can grab either .tar.gz or .zip archives)

LuGI - pronounced loogie - is a framework for building Lua glue code for BlitzMax, and provides much of the ground-work needed to get BlitzMax object support working quickly and painlessly in BlitzMax. The way LuGI does this is by firstly generating glue code for your project, and secondly by using its core API to expose that glue code in a manner that best mimics the way one would use the wrapped code in BlitzMax, thereby providing an almost seamless combination of BlitzMax and Lua.

The code is composed of a module for generating the required glue code to make use of LuGI and a very small core API: two functions to push objects and arrays, two functions to get objects and arrays, and an initialization function. The remainder of the API is hidden away by monks who cannot speak, for they have cut out their tongues. Incidentally, the API is also inside of a header file (lgcore.h) - however, the rest of the LuGI core code is strictly for those interested in writing their own code generators.

Alright, that roughly explains what LuGI is for and what it does. LuGI is free and open-source under the MIT license. You can use, modify, sell, etc. this code, and essentially do whatever you like with it, provided you follow the very simple terms of the license.

If you're not already familiar with Lua, I'll quote their about page for you, which should explain it fairly simply:
Lua is a powerful, fast, lightweight, embeddable scripting language.

Lua combines simple procedural syntax with powerful data description constructs based on associative arrays and extensible semantics. Lua is dynamically typed, runs by interpreting bytecode for a register-based virtual machine, and has automatic memory management with incremental garbage collection, making it ideal for configuration, scripting, and rapid prototyping.
(Emphasis mine)

There is already some documentation in the wiki, and I'll be working on improving that as I go along based on questions anyone might have about how to do something, how LuGI works, or what general rules there are for writing your own code generator. The goal here is that by releasing it, I'll be able to get some feedback on what areas need improving, try to locate bugs, and all that general stuff you get when people use your code.

The entirety of the core is written in platform-independent (or it is as far as I can see) C++. None of the 'complex' aspects of C++ are used, such as templates, metaprogramming, operator overloading, etc. just to keep things simple on the inside. If you know C++ and something about Lua, chances are you can tweak it to your liking. I may have also profusely commented the source, but I don't know how much that would help you. The generator doesn't rely on knowing how the core works internally, so you can change almost anything you want in either the core or the generator and it should not affect how the other module works. There are exceptions to this (e.g., the initialization routine), but pretty much anything in the core C++ is modifiable without hurting anything.

If you have any questions or comments, feel free to post them here. Should you encounter any bugs, I would encourage you to post an issue on the GitHub page (just click the 'Issues' tab at the top and click 'Create Issue') so that I'm notified about it more easily, but you can post about any issues you encounter here as well. Finally, if you have any questions that you cannot ask here or need to remain private, you're always free to e-mail me.

I apologize that this has no screenshots, but this doesn't do anything visual on its own, so you'll have to forgive me.


plash(Posted 2009) [#2]
Awesome. I'll be playing with this :)


rs22(Posted 2009) [#3]
This sounds awesome. Looking forward to checking it out. Thanks!


xlsior(Posted 2009) [#4]
Where should these files be installed to?

blitzmax\mod\lugi.mod ?


plash(Posted 2009) [#5]
Where should these files be installed to?
Correct.

From `core.mod/core.bmx`:
Module LuGI.Core



xlsior(Posted 2009) [#6]
Hm... compiles OK in normal mode, but not threaded mode for me:


Archiving:core.debug.mt.win32.x86.a
ar: creating C:/Code/BlitzMax/mod/lugi.mod/core.mod/core.debug.mt.win32.x86.a
Compiling:generator.bmx
Compile Error: Identifier 'TMutex' not found
[C:/Code/BlitzMax/mod/lugi.mod/generator.mod/utility.bmx;214;2]




N(Posted 2009) [#7]
Forgot to import brl.Threads in the module, it seems. Should be fixed now.


xlsior(Posted 2009) [#8]
That did the trick -- thanks!


Wiebo(Posted 2009) [#9]
I'm interested in this. But how would one integrate LUA for, say, AI for enemies and how would the advantage be from coding the AI in Blitz?


N(Posted 2009) [#10]
That's really down to the type of game you have and the way you code. The way I'd do it is similar to the way you'd handle entities back in the A5 (3D GameStudio) engine, where you'd attach a script function to an entity and then each entity would be updated using that function. How you do that part of the coding is really up to you, since there's not really any route that's defined as the best, as far as I know.


therevills(Posted 2009) [#11]
where you'd attach a script function to an entity and then each entity would be updated using that function


Hi Nilium, could you please post an example of this? Thanks!


Naughty Alien(Posted 2009) [#12]
..this is very nice Noel..thanks..


N(Posted 2009) [#13]
Hi Nilium, could you please post an example of this? Thanks!
I could post an example of a type that would handle it, but a full application would require me to essentially write a whole sample game- something I'd really rather avoid, especially considering there are a lot of different ways you could handle that. Anyhow, once I get some changes in on the core and generator, I'll see about putting up an example, since it would depend on an addition I'm making to the generator.


Wiebo(Posted 2009) [#14]
The way I'd do it is similar to the way you'd handle entities back in the A5 (3D GameStudio) engine, where you'd attach a script function to an entity and then each entity would be updated using that function.

I don't know 3D Gamestudio, but wouldn't that be the same as using a method containing the same functionality in a base type, and calling that from a derived type? I still fail to see the advantages of LUA in this. What can I do with LUA what I cannot do in 'normal' blitzmax? What is the big PLUS?

I can see one big benefit: modding the game. Using scripts from 'outside' the main code which can be altered without recompiling.


N(Posted 2009) [#15]
I don't know 3D Gamestudio, but wouldn't that be the same as using a method containing the same functionality in a base type, and calling that from a derived type?

Not exactly. The function could be changed at any time, generated at runtime, removed entirely, etc. It's more like if you were to add a callback field to an object and called that whenever you updated the entity. Even then, however, your update routines will all be hard-coded, so you'll be required to rebuild and update your binary whenever you need to fix something. With script, you could just leave the program running and reload the script at runtime, if you took the time to do that.

What can I do with LUA what I cannot do in 'normal' blitzmax?
Quite a lot of things. You could add and remove methods from objects (not BlitzMax objects- there are safeguards in place that will raise errors if you try to modify something that isn't defined for an object (unless you set BMX_TABLE_SUPPORT to 1, which I wouldn't recommend just because that code isn't tested right now), just generally speaking) at runtime, you can have operator overloading, anonymous functions, modify the environment that objects exist in, etc. Furthermore, due to it being open source, anyone savvy enough could easily customize the language to add their own features to it.

For an interesting, albeit odd at first, example of something you could do, you could chain a bunch of functions together to get a desired result:
function chainfunc(...)
	local chain={...}
	return function(...)
		local first = true
		local a,b,c,d,e,f
		for i,v in ipairs(chain) do
			if first then
				result = {v(...)}
				first = false
			else
				result = {v(unpack(result))}
			end
		end
		return unpack(result)
	end
end

math.randomseed(os.time()); math.random()

print(
	chainfunc(
		-- Add 'ern' suffix if the name is a compass direction
		function(v)
			local vl = string.lower(v)
			if vl == "west" or vl == "east" or vl == "south" or vl == "north" then
				v = v.."ern"
			end
			return v, y
		end,
		
		-- Add a random street number
		function(x)
			local streetnum = math.random(80)
			local lastchar = string.char( string.byte(streetnum, #tostring(streetnum)) )
			if lastchar == "1" and (streetnum == 1 or streetnum > 11) then
				streetnum = streetnum.."st"
			elseif lastchar == "2" and (streetnum == 2 or streetnum > 12) then
				streetnum = streetnum.."nd"
			elseif lastchar == "3"  and (streetnum == 3 or streetnum > 13) then
				streetnum = streetnum.."rd"
			else
				streetnum = streetnum.."th"
			end
			return x.." "..streetnum, y
		end,
		
		-- Add a random street suffix
		function(y)
			local names = {" Street", " Boulevard", " Road", " Crossing", ""}
			return(y..names[math.random(#names)]), x
		end
	)("East Hastings")
)


What is the big PLUS?
I can't answer this for you, since you seem to be more inclined to prove that Lua is not of use to you. It's up to you and you alone to decide what its use is, I'm just providing a tool to make the use of Lua easier (preferably significantly easier).

edit: Improved the above Lua example so that it better supports varargs functions.


Wiebo(Posted 2009) [#16]
I start to see the use of this. Thanks!


slenkar(Posted 2009) [#17]
HEY thanks Noel, this seems quite useful


JoshK(Posted 2009) [#18]
Once I started heavily using the lua reflection code, I found the speed to be very slow due to all the dynamic construction of objects. This code is a revision that uses a more optimal technique. I think this module has a lot of potential.

Reproduceable error:

luabug.bmx:


lua-gluefunctions.bmx:


test.lua:



Azathoth(Posted 2009) [#19]
Argh. IE8 won't let me download it.


N(Posted 2009) [#20]
I'm pretty sure I fixed the above bug Josh found- seems to be due to a missing return statement in the lugi_gc_object metamethod. I'm not sure what the return value of a function in C is when it doesn't have a return statement, but it was destroying the application under Windows (and possibly Linux, but Mac OS is unaffected for whatever reason), so I'm guessing it's a garbage value.

Anyhow, should be fixed.

Additionally, I haven't done any testing under Linux. I don't have a VM for it right now (not much excuse for that other than that I don't want to spend the time downloading an ISO) and I'm definitely not going to install it into its own partition since I've got limited space. If anyone has Linux installed, is actively using it, and is interested in this code, I'd appreciate any feedback on it from you. As it stands

Argh. IE8 won't let me download it.
That's just bizarre.


Edit: For those who used LRef/lua-reflection, and there were very few people I know of who used it, you might remember there was a bit of a hack in the way it handled methods and such that you could pass around methods and they'd just work without you ever needing to actually have the object on hand. While that's not possible out of the box with LuGI, I figure I'll offer up a solution to that. This script function will let you pass around the method with its object intact, and you can call it however you like:

function DelegateMethod(object, method)
	local _method
	if type(method) == "string" then
		_method = object[method]
	else
		_method = method
	end
	local _object = object
	local fn = function(...)
		return _method(_object, ...)
	end
	return fn
end



N(Posted 2009) [#21]
I've just added a new "category" metadata tag you can use to sort of filter out what you want to generate glue code for in LuGI. Categories are just names that you can use to identify sections of code that should have glue code generated for them. There's a brief explanation of it in the wiki's Metadata section. The category stuff is completely optional to use and it won't affect anyone already using LuGI.

Also added a lua_isbmaxobject function, since otherwise you may have trouble determining if an object is an object or just another userdata type. The type of userdata currently differs between regular and threaded builds as well, so this is the most reliable way to determine if an object on the stack is a BMax object or not. It works by checking the metatable of the value on the stack (after ensuring it's the right kind of userdata) and seeing if it's the generic metatable used by all BMax objects handled by LuGI. Not that you need to know how it works, but that should at least be some indication that you won't accidentally get a false positive.


Duckstab[o](Posted 2009) [#22]
Just installed the mod this looks really cool always wanted to add scriting to my game.

Now as a total newb I was wondering how to put it into practice
Type Character
        Global List:Tlist = CreateList()
	Global Stats$[]
	Field id:Int
	Field Name$
	Field Stat:Int[]
	Field Effect:Int[]
End Type


What i would like to do is

1. Via a Script add a array of stats to the Type so i could have HP,MP and at a later date add AP ect..
Stat and Effect are Dynamic arrays so they reflect the Size of the Stats$ array.


2. Create instances from a Lua script with Name and Stat Values

3. Use a Script to change the Value of effect on any given instance

any help welcome :)

btw thanks for the Mod o/


N(Posted 2009) [#23]
I think you'd be better server by not having that type in BMax/the host application. If you need it in the host, I suggest you write getter/setting methods for the arrays, because LuGI has somewhat limited array access (it has to push arrays as tables and convert tables to arrays- this behavior isn't likely to change).

At any rate, I'd just do it like this in Lua:
-- character.lua

Characters = {}
setmetatable(Characters, {__mode="v"})

function NewCharacter(name)
	local char = {
		Name = name,
		ID = 0, -- some number don't know what
		Stats = {
			Strength=5,
			Dexterity=5,
			Agility=5,
			Intelligence=5,
			Charisma=5,
			Luck=5,
			Health=30,
			Magic=10
		},
		Effect = {
			Poisoned={State=false, Timeout=0},
			Mute={State="gordon freeman", Timeout=-1}
		},
	}
	table.insert(Characters, char)
	return char
end

local char = NewCharacter("Gordon Freeman")
char.Stats.Aptitude = 5	-- set new stat


That way, 1 & 2 are non-issues because it is in the script. It also means characters could be more easily redefined for gameplay purposes, like if a player came along and wanted to introduce some new stat by default, they could (assuming you don't store all scripts as bytecode).


JoshK(Posted 2009) [#24]
This code will parse a source code and generate a type with methods for functions that aren't part of a type. Add 'hidden on the end of functions and includes/imports that you want ignored:



JoshK(Posted 2009) [#25]
How would you suggest handling multiple layers of commands? For example, my engine has its own command set in the module. Then I have an interpreter built on top of this, which enables some additional functionality in Lua. However, the interpreter generator redefines all the existing engine functions, as well as the new functions, so all the engine commands are listed twice. I'd also like to be able to override commands in the module, but it looks like the old command is used. Any suggestions?


N(Posted 2009) [#26]
I haven't got a clue what you mean.


JoshK(Posted 2009) [#27]
It's hard to explain. I think I am just moving the lugi generator code to the top level of whatever application is using it. It becomes the end user's responsibility to generate the lugi code, but that way there are no repeated commands.


N(Posted 2009) [#28]
I went ahead and updated LuGI with some code for working with arrays. Previously, arrays would be pushed onto the Lua stack as tables every time. This isn't particularly nice, because it means that if you access an array field of an object and modify the value you get from it, you won't do anything. So, that's been fixed. Arrays of arrays are a little bit iffy right now, and there are issues with auto-arrays due to a bug in BlitzMax where the created array does not have an actual type, but otherwise it seems to be working fine.


_Skully(Posted 2009) [#29]
Thanks Nilium

Yoink!


Armitage 1982(Posted 2012) [#30]
This module is extremely handy when it comes to Lua!
And it's even working with the recent LuaJIT 2.0 b10 from Zeke (you simply need to change the import version of Lua in the core module :).

I will probably rewrite my engine this year, and try to include a bit of Lua, at least for AI and Game object :)

I read there is a few problems while generating the glue file. Hope it doesn't make things too complicated.

Anyway, Thanks a lot for this module Nilium!