Concepts of modifying/scripting
Blitz3D Forums/Blitz3D Programming/Concepts of modifying/scripting
| ||
| I know this has been raised a few times before, but I wanted to look into the possibilities of allowing "user-scripting" with a B3D app in a bit more detail. Most solutions I have seen available tend to rely on a form of 'script parser' built-in to the engine which will read and action simple commands during runtime as and when required. To me this is far from ideal, in a large part defeating much of the purpose of implementing a scripting functionality. It would be an immense slowdown reasding lines of code (maybe a little less so if the scripts are written with a predefine editor which can save them to be read as data direct to banks maybe, yet then there's more of a conversion process required.) and then there's the integration with the standard code base. Unless every 's c criptable' eventuality is considered somewhewre with respect to global variables then errors will creep in. Yet of course, there is no a means of compiling external dependancies into an existing executable. So the options certainly seem limited. However, I propose a possible 'solution' which naturally requires a similar format to the above, yet with more consideratin given to runtime efficiency 'during gameplay'. Instead of the "scripts"' functions being read, parsed and actioned as and when required, potentially multiple times, the entire script library would be parsed early on, perhaps during the 'initialisation phase' of the main app. Instead of directly parsing functions into somee form of simple conversion If instead, the scripts were converted into a simpler structure, though paerhaps taking some real effort and time to produce something comprehensive and workable, but something more reminiscent of a higher level language in a way, where values were arbitrarily operated with their references implied only at critical points. Forgive me if this sounds a little confusing, but consider the above rough example, but now in this sense:
;The kind of argument list required to identify what's being done.
FUNCTION[x]="CHECKBULLET"
::
OBJ[1]="BULLET"
OBJ[2]="PLAYER"
LINKFUNCT[1]="KILLPLAYER"
VALUETYPE[1]="HEALTH"
VALUETYPE[2]="NONE"
VALUE[1]=OBJECTVALUE(OBJ[2],VALUETYPE[1])
VALUE[2]=INTEGER("0")
COMP[1]=COLLIDE(OBJ[1],OBJ[2])
COMP[2]=LESS_THAN_OR_EQUAL(VALUE[1],VALUE[2])
OPERATOR[1]=DECREASE(VALUE[2])
ACTION[1]=CALL(LINKFUNCT[1],OBJ[2])
IFCOMP(COMP1)
{
DO(1)
{
OPERATOR[1]
IFCOMP(COMP[2])
{
DO(2)
ACTION[1]
{
}
ENDIFCOMP(COMP[1])
}
ENDIFCOMP(COMP[2])
}
;And a special addition which allows us to 'attactch this function to an event, so the main program knows when to call it.
CALL_FLAG=1
::
Whilst it would need some careful work in setting up and understading the limited syntax, The advantage here, is that repeated functions, such as those that check for objects etc. need not be repeatedly parsed in entirety. A range of operator types and comparison types can be converted in B3D with something like:
Function ScriptValue(ValueReference%,Value%)
CurrentScriptFunctionValues(ValueReference%)=Value%; Making use of a a global array
End Function
Function Comparison%(ComparisonType,Comparison_Value_1%,Comparison_Value_2%)
;in a sense, this is similar to how one might overload operators, though of course, we're not actually using the b3D operators.
;Some predefined constants
Select (ComparisonType)
Case CON_COMP_TYPE_ADD: Return CurrentScriptFunctionValues(Comparison_Value_1%)+CurrentScriptFunctionValues(Comparison_Value_2%)
Case CON_COMP_TYPE_SUBTRACT: Return CurrentScriptFunctionValues(Comparison_Value_1%)-CurrentScriptFunctionValues(Comparison_Value_2%)
Case CON_COMP_TYPE_MULTIPLY: Return CurrentScriptFunctionValues(Comparison_Value_1%)*CurrentScriptFunctionValues(Comparison_Value_2%)
; etc...
End Select
This is an extremely simple example, and certainly far from complete, but what's everyone's thoughts? Is this a good approach? Are there better alternatives (and if so , please share! :) ) If this is considered to be useful or has some potential, then I'l get really cracking on it. The basic functionality as I have described above is kinda already done and works in short, but nowwhere near enough to be useful or to identify it it's particularly fast enough. As you can imagine, it will take a great deal of work to get this up to a decent standard, but if the concensus is that it's wrth it, I will get cracking on it and of course keep you all up to date with the source :) |
| ||
| You mean compiling to bytecode? Take a look at: - GameScript, by John J. (Open source - BSD) - My signature link (I haven't kept the website updated but will happily share newer sources if you want - it's much more like Java than C++ now) - Quake C, by John Carmack (Open source - GPL) - UnrealScript (closed source but lots of documentation) - Java Very few serious attempts at scripting have used direct interpretation, because you're right - it's far too slow to do anything useful. That, and if you're not storing structure data anywhere, it makes life very difficult for even managing structured programming, let alone proper procedural programming (and OOP is right out). |
| ||
| Wow, that's great Yasha.. yes, it seems like the bytecode approach was the kinda lines I was thinking on (I just didn't know it had a name/existed :) ) I looked at your link, some wikipedia articles on the suggestions there, the Quake C one I need to read a bit more*, but Do you have any details on how this kinda stuff is implemented? Is it a (relatively :D) laborious conversion, and are functions effectively duplicated? How does, say "Java" 'talk' to a blitz exe? *edit: [quote="Wikipedia"] QuakeC also suffers from the fact that many built-in functions (functions prototyped in the QuakeC code but actually defined within the game engine and written in C) return strings in a temporary string buffer, which can only hold one string at any given time. In other words, a construct such as SomeFunction (ftos (num1), ftos (num2)); will fail because the second call to ftos (which converts a floating-point value to a string) overwrites the string returned by the first call before SomeFunction can do something with it. Other prominent examples of these quirks include the fact that QuakeC does not contain any string handling functions or file handling functions, which were simply not needed by the original game. [/quote] This kinda issue is part of the problems that my ideas hopefully should overcome. By (after eventual conversion) storing arguments, parameters, handles and such in arrays (or types maybe? depending on circumstance). |
| ||
| The implementation is actually pretty simple... Your compiler will spit out a list of VM instructions, same as a native compiler spits out processor instructions (from a mathematical/CS point of view, there is no difference between the two kinds of compiler). The simplest kind of VM then simply iterates over this list of instructions, and chooses an action for each one. eg. a = a + b might, given the right compiler, turn into load 0 load 1 add store 0 for a stack machine, or simply add a b a for a three-address code random access machine. Part of the fun is deciding on the machine model - stack-based is simplest (and easiest to use with complex tasks like OOP), random-access is fastest for software interpretation, register-based is closest to how a real CPU works. (Note that the point of bytecode is that you'd still be representing these as integer constants, not as strings.) Choosing the action could be as simple as this:
Select instr
Case add
; (Do whatever add operation)
Case sub
; (Subtract operation)
Case mul
; (Multiply operation)
Case div
; (Division operation)
...
Case call
; Call a native function
End Select
...although Select isn't optimised in B3D so it's not the fastest way to do this (using a binary chop - "if instr > MIDPOINT then if... else if ..." has time complexity O(log n) rather than O(n) ). The important part is being able to access native functions: you can have machine codes for the really common ones (no reason to limit yourself to machine codes used on your CPU), and leave others flexible. I would do this by assigning each function a UID and then having another select structure within the "call" instruction; in other languages (or with the help of a userlib) function pointers would be good for this, too. Anyway, somewhere there has to be a way of registering your B3D functions with the VM, otherwise it can't do anything useful in the main program. Err... I'm not sure I understand what you mean by "conversion" or "duplicated". EDIT: Ah, I wouldn't bother about the QuakeC parameter issue - that's because QuakeC was - in my opinion - implemented really badly, using a totally counterintuitive VM that couldn't store temporary variables - no implications for other scripting languages. What they mean is that you had to do this: temp1 = myQCFunction(a, b, c); temp2 = myQCFunction(d, e, f); result = myOtherFunction(temp1, temp2, X, Y); rather than result = myOtherFunction(myQCFunction(a, b, c), myQCFunction(d, e, f), X, Y); ... like you can in B3D and any serious language, because the second call to myQCFunction would overwrite the parameters of the first, and the result. |
| ||
| ...although Select isn't optimised in B3D so it's not the fastest way to do this Really? that's interesting to know.... Anyway, it shoulkd only be a small switch, say, for add, subtract, multiply, divide.. other operators could have different "labels" so could be sent to a separate function so as not to have a huge switch (select/csae) of every possible operator. _________ You are addressing the parsing of the code on a deeper level which is great, and therefore clearly an imporvement on my initial suggestion. The use of identifying specific memory pointers for the functions would be crucial as you say, to making a script "truly integrated", however, when faced with the fact that once compiled, all the functions and procedurality is quite 'distorted' due to being compiled and condensed into their machine code equivalents, soeven if the RAM memory locations could be identified, not only would they be inconsistent, but also, promne to issues with use of the paging file, maybe? Therefore I opt for a less ideal idea, but one that should still warrwant the same level of "understanding" by the main app. So I am still converting what is read as "bytecode" into a format to be parsed into various functions which can action the script functions/commands "almost as efficiently"(I would hope) as a standard default function of the main app itself. _________ When I say 'conversion' and 'duplication', I mean for example that a function in the script might be something along the lines of: (just a kinda pseudocode, but hopefully eequivalent to my earlier examples) Function CheckBullet(Bullet,Object) If Collided(Bullet,Object) Then DecreaseHealth(Object,1) If (GetHealth(Object)<1) Then Kill(Object) End Function For this to be interpreted and actioned in the main code, it would be converted too a format as suggested kinda reminiscent of Assembly or such, however, in doing so, it would also strip away any kind of "individuality" or identity of the function. If the script contains something like this: Function CheckTimeLimit(Bomb,Object) If (FuseLit(Bomb)) Then DecreaseTime(Bomb,1) If (TimeLeft(Bomb)<1) Then TimeOver(Object) End Function essentially, it's identical, but would be parsed entirely by the intepreter and therefore duplicating where only one function framework is really needed: A_Function(Ref) If (Boolean(Ref1)=True) Then Operator(Ref,Val) If (SubFunction(Ref))<1) Then CallFunction(Function(Ref)) End Function In typically available 'scripting' libraries (I am talking specifically those desinged for Blitz3D ) this duplication bloats the code, slows it down and makes for a lot of unncecessary string-checking. In my proposition, the interpreting of the script would know that it's the same thing, since "Laser" and "Bullet" are just some form of handle/label/reference. When the script is parsed, all the 'engine' needs to know is: Arguments-2 Reference1 (Tye=object reference), Reference2 (type=object reference) Function :: FIRST ALL THE DATA AND DATATYPES ARE 'DECLARED':: ---Values=3 ------Value1 (bool) ------Value2 (int) ------Value3 (int) ------Value4 (int)=1 ------Value5 (int)=1 ------Value6 (bool) ---Operations=2 ------CollisionCheck(bool) ------IntegerManipulation(int) ------IntegerComparison(bool) ------Actions=2 ---------CallFunction ---------CallFunction :: THIS IS TEH ACTUAL GUTS OF TEH FUNCTION:: ---Operator ---What Operation? ------Boolean ------What Boolean? ---------CollisionCheck ---------What Am I Operating? ------------Reference1 ------------Reference2 ------------Where Do I Put The Result? ---------------Value1 ------True? ---------Action ---------What Action? ------------CallFunction ------------What Am I Actioning? ---------------GetHealth(Reference2) //Okay this makes things a little more complex but still, you get the idea :) ---------------Where Do I Put The Result? ------------------Value2 ---------Operator ---------What Operation? ------------Integer Manipulation ------------What Integer Manipulation? ---------------Subtraction ---------------What Am I Operating? ------------------Value3 ------------------Value4 ( = 1 ) ------------------Where Do I Put The Result? ---------------------Value3 ------False? ---------{NOP} ------{End Boolean} ---Operator ---What Operation? -----Boolean ---------What Boolean? ------------IntegerComparison ------------What IntegerComparison? ---------------LessThan ---------------What Am I Operating? ------------------Value3 ------------------Value5 ( = 1) ------------------Where Do I Put The Resullt? ---------------------Value6 ------True? ---------Action ---------What Action? ------------CallFunction ------------What Am I Actioning? ------------KillObject(Reference2) ------False? ---------{NOP} ---------{End Boolean} EndOfFunction So in a byte-wise sense, say, 65536 individual UNIQUE steps of the process above could be represented, as in assembly. The process is remarkably procedural, though perhaps some problems may occur with more complex functions and nested loops / conditionals etc. For example, "Where Do I Put The Result?" might be given a value of '16', it doesn;t really matter WHAT value this is, only that the same value will always apply, and ONLY apply to the step "Where Do I Put The Result?" the following ("STANDARD DEFAULT NUMBER OF BYTES") byte(s) will identify the answer to that question specifically for each situation. |
| ||
| Wow! It took me an hour to post that! o.O :D |
| ||
| I wouldn't worry about the duplication issue. It causes bigger binaries, true, but it won't cause slowdown as you only need to examine the requested function. Writing a compiler that could identify common function structures would be a nightmarishly tough job (although I won't stop you trying...). Really this is up to the script author, in the same way that it's up to B3D authors not to write a separate "jump" function for each NPC or whatever. Otherwise, looks like you're on the right track - although bear in mind that existing scripting libraries (such as mine or John J's) don't use string comparison either! |
| ||
| Yeah, sorry if it was misunderstood, the duplication is something that is automatically taken care of, provided the scripted function is 'read' :) I know how mammouthly daunting this sounds, and that your approach (I've yet to look in detail at on's too) are great examples to go by. I already for see issues with multiple brackets and such and I don;t know how far I could get with the 'complexity' of the script language to be honest. That's really why I posted in the first place, ythat you suggest I/'m, on the right track is a great incentive, though since you've enlightened me to existing alternatives, I'll seriously have to conssider how worthwhile my attempt would be. Certainly I wont trash it right off, though, so I am willing to give it a shot, at least, at the absolute weorst, I already conceive how I may have something of a 'realtime debugger' functionality that could allow changing of certain values during runtime for testing purposes :D |
| ||
| You could run with something like LUA |
| ||
| LUA would be ideal, in fact, any existing scripting solution would be ultimately pprefereable to coding my own :D Problem is, LUA is incompatible with the structure of B3D (According to another thread on here somewhere...) It was partly the benefits of a scripting ability combined withe multitude of LUA libs and such appearing for BMax codearcs that prompted me to get started on this whole train :D |
| ||
| The Lua VM isn't compatible with Blitz3D, no. However, Lua is open-source with a permissive licence (allowing commercial use)... so it'd be easy to a) modify the compiler very slightly to suit B3D, and b) either heavily modify the VM or rewrite it altogether in B3D (wouldn't be as fast, but it would at least work). I considered doing this at one point... might be worth reconsidering, actually. Mainly didn't because I didn't want to use Lua at the time... (also because my language-implementing attention is kinda spread thin already, between NoodleC and Ravioli) |
| ||
| I for one would really like to see that heappen :) It's certanly beyond my ability I think to change the LUA VM. I first encountered LUA about 3 years ago, but it rapidly seems to have grown more popular, or maybe I'm just more aware of it, though it certainly seems comprehensive and versatile I've had every little experience and/or opporttunity to really 'play around with it'. If this is siomething that you do perhps consider in the futture, Yasha, counrt me in if you would like any help or just an 'accomplice' I'd be willing to put the time and effort in where required for something as potentially useful as this :) |
| ||
| Does anyone else have anything to add on this? I'm suprised actually thought there may be more interest overall... |
| ||
| I've got stuff to add! As if there wasn't enough already... As people may have noticed, I'm really focusing a lot on language/compiler stuff at the moment (including devising a whole new language for my MSc). I'd therefore be more than happy to contribute to/lead any scripting language projects for Blitz3D - I've investigated wrapping JavaScript and Lua, and done around two thirds of the work on my own language (NoodleC - not connected to my thesis language) as well, so in a couple of weeks (after exams!) I'd get on board such a thing (and am willing to share any existing information on request). I too think this is an important part of the B3D armoury that hasn't been addressed properly. As far as I'm aware there are no free, OOP or FP scripting solutions for B3D at the moment (Procedural is no good for scripting, in my opinion). |
| ||
| That's fantastic to hear, and I would gladly hand the mantle over to you, since you clearly have a lot more experience and talent. I wont mislead you that to me this is very ambitious and will really stretch my level of ability, but I feel you have a much better grasp of the concept and the ideal way to go forward, but certainly I will do all I can to help, just let me know what I can do to help and I'll do my best... In the meantime, I shall press on with my current fcus just to see how it goes and maybe if it's something that might be compatible with what you decide or have already done :) |
   |