BlitzScript3D

Blitz3D Forums/Blitz3D Programming/BlitzScript3D

Techlord(Posted 2004) [#1]
BlitzScript3D is a Scripting System with a BASIC Syntax for use in Blitz3D made games. Based on Mark Sibly's Blitz Compiler Code, the Scripting System Lib will contain a Compiler and Virtual Machine.

The Scripting System Compiler will open and parse *.bsc files, generate opCode that resembles x86 assembler instructions, and convert opCode into smaller Pcode (opCode represented by a integer value), saving the Pcode files with *.bso extension. Pcode files are loaded and executed by a Virtual Machine. The Virtual Machine simulates a cpu processing the Pcode files.

The Scripting Language Supports:
1. Keywords: let, print, end, if, then, else
2. Blitz3D Commands
3. Operators: binary +, -, *, /, <, =, >, <=, <>, >= and unary -
4. Precedence : *, /, >>,<< have precedence over +, -, which have precendence over <,=,>,<=,<>,>=
5. Subexpressions grouped by Parenthesis .
6. Comments
7. Preprocessing (New)

To Be Implemented:
1. Multi-Dimensional Arrays (wip)
2. Functions
3. Conditional Loops
4. Select...Case

Last edited 2011

Last edited 2011


jfk EO-11110(Posted 2004) [#2]
I guess a Script Interpreter and a Compiler that generates Assembler Souces are two completely diffrent things. Probably you should think about to use the already existing virtual Blitz Machine, this would save you a lot of work. Also Noel started a Script Interpreter if I remember right.


Techlord(Posted 2004) [#3]
jfk

I'm aware of BVM, however, the goal is to produce another Scripting alternative with Blitz. I have developed 2 scripting systems using Marks Compiler code as the basis. Had some pretty good results. Just decided openly write one.


N(Posted 2004) [#4]
Started one, never finished it.

http://blitzbasic.com/Community/posts.php?topic=38036#420588

Personally, I prefer mine to this sort of one, but then I'm an odd fellow. I don't really consider this one a scripting language until I see it do something other than convert one language to another.


Techlord(Posted 2004) [#5]
Noel

Actually, there is no scripting language yet. Just a basic compiler that can parse some basic-like syntax into opcode.

It currently takes the basic program
let a=1
let b=2
let c=a*b+a/b
if a=b then print a else print b
end

and generates these opcodes
//tbasic startup...1
  sub  edx,edx
//let a=1 
  mov  eax,1
  mov  [_a],eax
//let b=2 
  mov  eax,2
  mov  [_b],eax
//let c=a*b+a/b 
  mov  eax,[_a]
  push eax
  mov  eax,[_b]
  pop  ecx
  imul eax,ecx
  push eax
  mov  eax,[_a]
  push eax
  mov  eax,[_b]
  mov  ecx,eax
  pop  ecx
  shl eax,ecx
  pop  ecx
  add  eax,ecx
  mov  [_c],eax
//if a<b then print a else print b 
  mov  eax,[_a]
  push eax
  mov  eax,[_b]
  pop  ecx
  cmp  ecx,eax
  slt  eax
  and  eax,255
  and  eax,eax
  jz  __1
  call print:64
  mov  eax,[_a]
  jmp  __2
__1:
  call print:64
  mov  eax,[_b]
__2:
//end 
  ret

_a:
  dd  0
_b:
  dd  0
_c:
  dd  0
Done!
Pcode is merely a numerical representation of these opcodes
;opcode
mov eax,2

;pcode equivalent
01 2


The VM simulate a cpu containing the necessary LIFO Stacks (Array), Registers/Stack Pointers(Variables), and Functions to process Pcode. Generating opcode may be optional.

Using 'compiled' Pcode files will provide greater performance than 'interpreting' the scripting language on the fly. Compilation is only required once.

This is a good start, lots to add.


MSW(Posted 2004) [#6]
Sense the VM simulates a CPU, also think about a muti-threading approch with Pcode(script) instanceing, parent/child linking, and messageing between those instances.

You could concieveably create a bunch of object scripts...a script for a particular type of gun, another for a particular type of armor, particular type of movement (running, walking, jumping, etc)...and pieceing them together in parent/child relationships under an AI script...and instance them with a particular game character/enemy/whatever...in this way even simular enemies can have different weapons, and effects at thier disposal, and completely customizeable at run time without haveing to generate bloated AI scripts that take all variations into account.

Obviously this would require a little more planning and work in terms of the VM...effectively a bit like writeing a OS for the VM in that you would need the equivelant load utility to get scripts into memory for VM use, a resource/process manager to maintain the scripts in memory and introduce them once loaded to the VM, as well as the appropriate garbage collection utilities to clear unused scripts from memory and general clean-up duities....then of course the VM would need to work with time slices, or other means of multi-processing control, as well as a means to swap between executeing processes...but it shouldn't be that hard.


Techlord(Posted 2004) [#7]
.


Techlord(Posted 2004) [#8]
MSW,

I'm a lil foggy on the concepts muti-threading approch with Pcode(script) instanceing, parent/child linking, and messageing between. Please provide more detail as im very interesting in adding these features.


Techlord(Posted 2004) [#9]
Work In Progress. I have divided the Scripting System into two sections: Compiler and VirtualMachine. The compiler's job will be to load and compile 'BlitzScript' into Pcode. The Virtual Machines Job will be to load and execute Pcode. I have made several changes to Mark's Original Source. Mostly dividing the source into smaller 'code modules' and renamed all vars, functions, etc, with 'bscvm' to ensure the lib will function without interference with existing code.

Get the
WIP ZIP. Included are code modules, I generated with TypeWizard. Many of the modules contain unnecessary functions to be removed when the system is working properly. Included are also some other code modules used in other projects. The WIP ZIP contains:

bscvm.bb - Main Program
stack.bb - An old stack lib I've used in many projects.
bscvmVar.bb - Manages a look table for Vars used by the compiler and virtual machine.
bscvmLabel.bb - Manages a look table for Labels used by the compiler.
bscvmCompiler.bb - Manages Compiler Object, compiles 'blitzscript' *.bbs files to pcode files *.bsc
bscvmThread.bb - A component of the Virtual Machine, supports psudeo-multithreading execution.
bscvmVirtualMachine.bb - Virtual Machine Object - executes pcode files.


Techlord(Posted 2004) [#10]
.


Techlord(Posted 2004) [#11]
With the current design, the compiler and virtual machine are independent. The VM will be able to exec multiple script 'threads'. If you take a look at the compiler and VM modules you will notice two large Select...Case structures. These structure quickly map function calls and parameters to blitz commands. Custom functions will operate differently, the will be bahave similar to scripts with ability to pass parameters. Not sure how its going to be implemented, i just know that i will.


MSW(Posted 2004) [#12]

MSW,

I'm a lil foggy on the concepts muti-threading approch with Pcode(script) instanceing, parent/child linking, and messageing between. Please provide more detail as im very interesting in adding these features.



Okay...this may be a bit confuseing:

First you will need a resource manager...this handles the loading of scripts, allocating of memory banks, etc...

The pcode itself is stored in bank...and when the resource manager is called upon to load a script into memory it first checks to see if it hasn't already been loaded...if not then it creates a resorce tracking record (basicly a type) which contains the handle for the Pcode bank, the name of the pcode, info about how much variable space the pcode requires and all that...then a script instance is created (another type) it is given a name along with handles to both the pcode bank, and a bank for pcode variables, messages, and bytecode pointers....if the resource manager finds that the script already exists, it simply creates another script instance of that type.

basicly there are two scirpt instance lists. and the VM works with the "active" list.

messages are basicly special stacks that parents can push data onto, with the script instance owning the particular message list being the only instance able to pop data off.

the parent/child stuff works quite simply.

basicly you have something like this:

Type script_inst

;basic stuff

 Field Name$
 Field ID%                   ;used for classifying scripts 
 Field Pcode                 ;handle to a bank
 Field variables             ;handle to another bank
 Field Messages              ;handle to another bank  
 Field pcodepointer%
 
 
;VM active/inactive list stuff

 Field flags%                ;is script active, etc..
 Field previous.script_inst  
 Field next.script_inst

;instance parent child stuff

 Field parent.script_inst
 Field child.script_inst

end type


typicaly only the highest level parent (its parent = NULL...or possably the game control script) scripts are active...this means thier children will likely be on the inactive list.

Say the parent script decides to fire its gun...the vm then searched through the scripts children for the first matching gun ID..finds that the child is inactive...changes the childs flag to active and puts it on the end of the active list adding "fire" to its messages

the lists are structured like this:

Dim active.script_inst[1]
Dim inactive.script_inst[1]


active[0] points to the first script in the list
active[1] points to the last.

tis' basic linked list stuff...I think you prolly have a handle on that.

anyway..when the VM gets to the particular gun script...first thing it may do is check it's messages...sees "fire"...and does so by calling upon the resource manager to create a new script instance...in this case it's "bullet 1" or some such...the resource manager creates the appropriate instance...and the gun script becomes it's parent, putting it on the active list, and giveing it messages informing it of what direction to travel in, etc...then the gun script tells the VM to put it on the inactive list and reset it's pcode pointer.

course the bullet script is then run, gets it's messges, sets the appropriate variables, moves the bullet mesh into possition, etc...it remains on the active list...until the game control script that is parent over everything finds that it has collided with a wall, and gives it the appropriate message...it reads it, and decides to generate a smoke script...does so...then tells the VM to delete it (parent/child, previous/next lists are updated in the process)

hopefully that makes some sense.


Techlord(Posted 2004) [#13]
MSW,

Well, I think I'm on track with your suggestions. The 'bscvmThread' object is the equivalent of 'script_inst'. I'm handling processing and management different but, should achieve the same results.


Techlord(Posted 2004) [#14]
Looking for ideas on how to deal with functions.


DrakeX(Posted 2004) [#15]
CALL and RET shouldn't be too difficult to implement. you'd have to make a call stack.

also - is there any reason you're restricting yourself to x86-style opcodes and registers if it's not real assembly? i mean, if i were you, i'd use something like r0-r15 for registers.. one of the problems with x86 assembly is the sheer lack of general-purpose registers, and it doesn't really make much sense to limit yourself, especially in a scripting language, when you need speed and can't afford to be shifting registers around all the time.

just my 2 cents :)


Techlord(Posted 2004) [#16]
DrakeX,

A Call Stack? Please elaborate as im sure there are many ways to implement one. The x86 Opcodes were merely a starting point.

Found some good articles for others interested in writting there own Scripting Engine. Flipcode's Implementing a Scripting Engine: Part 1, Part 2, Part 3, Part 4, Part 5, Part 6, Part 7, Part 8, Part 9

Note: Part 9 discusses how to deal with functions.


Techlord(Posted 2004) [#17]
The latest changes: Wrote some help docs on BlitzScript which be a basic-syntax language very similar to Blitz3D. Still trying to sort out the handling of functions as there are many ways to go about it. I dont think object-oriented stuff is going to make it into BlitzScript.


N(Posted 2004) [#18]
Frank: Write a preprocessor for the scripts to allow usage of object oriented programming features. I did that for Blitz3D and it works fine. Just spits out a .BB file that has code compatible with Blitz.


Techlord(Posted 2004) [#19]
Noel,

OO is way on the bottom of the feature list at this point and time. However, figuring out implemention of standard variables and functions are giving me clues for implementing oo.

Write a preprocessor for the scripts to allow usage of object oriented programming features. I did that for Blitz3D and it works fine. Just spits out a .BB file that has code compatible with Blitz.
I'm not sure exactly what you mean here as BSCVM is a in-game script compiler & processor that uses scripts at runtime. The method you describe requires you to convert your script -> .bb and then compile the *.bb with blitzcc. Please correct me if im wrong.


N(Posted 2004) [#20]
I mean when you compile the script file, preprocess it (either via some means such as a virtual file system or via a temporary file (bscvm.SWAP for example)) and then process the preprocessed version. That would, basically, result in a pseudo-object oriented environment. g++, as far as I'm concerned, is a big preprocessor that converts to C and then builds the preprocessed file- presumably from an existing stream such that no extra files are neccessary.

Does it quite nicely, too.

Rambling there, my brain is jelly at the moment.


Techlord(Posted 2004) [#21]
If I'm to add OO features they will be implemented into the Compiler, just as the standard variable and function features are.
I've checked out your AAAP. IMHO performing preprocessing is unnecessary. Truly these features should be incorporated into the compiler.


Techlord(Posted 2004) [#22]
BlitzScript3D Compiler & Virtual Machine (BSCVM) will making a debut in Project PLASMA FPS! BSCVM will be the scripting solution for Project PLASMA FPS.


Strider Centaur(Posted 2004) [#23]
Im not sure a scripting language needs a VM.

I see several people are trying to make a BB scripting language, But Im not really sure what advantage this has of native BB other than the obvious ability to influance a sceen or game engine coded in blitz without having access to the actual source and need for recompile.

While those are good reason for scripting, it seems to me that BB would make a terible scripting language. Its a great programming language for what it was designed for, but its bulky and not very intuitive. It also has alot of restrictions on how you can access data. A good scripting language should be light, easy to learn, and provide plenty of support for accessing all types of data, even using methods not supported by the main application, in this way you can make scripts that extend the application in ways you normally would not be able too.

There are some excelent scripting languages out there, and many are free to use, everything from Embeded python DLL to LUA.

Given that many of these scripting languages are opensource and supported by huge comunities, one would think that using them would be a pleasure.

As usuall Im not a fan or doing what others have already done, Im just to lazy to reinvent things. :)

BTW hasn't somone aleady done a LUA DLL for use with blitz?


Techlord(Posted 2004) [#24]
Strider,

The inclusion of a scripting language into PPF is to influence Game Objects in the engine without having to access to the engine source and recompile with blitzcc. Thus, games developed with the 'engine' can be MODed without the need for Blitz3D compilation.

In respect to PPF the scripting language will be focused around high level interactive game objects: Camera, Players, Weapons, Projectiles, Reactors (Particles), Powerups, Triggers & Keys, Waypoints, Platforms, Flares (lens flares), Foilage, Terrain, Maps, Soundfields, Warps, and GUI & Chat.

The scripting language will be a BASIC Syntax and offer a limited number of control loops, conditions, variables and arrays, and functions. Scripts are compiled to pcode for fast loading and execution. The VM process pcode.

Network Core, Rendering Optimizatons, and other 'behind-the-scenes' task are managed by the engine coded with native blitz. All modern, complete game engines offer a method of programming game behavior via some sort of scripting language and PPF will follow suit.


Techlord(Posted 2004) [#25]
Anyone interested in a Scripting Engine?

PS: A Free One that is.


@rtur(Posted 2004) [#26]
yes! of cource! :)


Techlord(Posted 2004) [#27]
BlitzScript3D Engine is coming!

To Be Continued...


Techlord(Posted 2004) [#28]
...


Synchronist(Posted 2004) [#29]
@Frank, a worthy project to be sure! I was seriously looking at Java, Java3D and the BeanShell for a project. Your timing perfect! Thx!


Techlord(Posted 2004) [#30]
@Sychronist, Thanks.

There is more to do. But, once complete it should be an ideal scripting solution developed with Blitz3D for Blitz3D!


Synchronist(Posted 2004) [#31]
@Frank; this is great! I can see you've got your work cut out for you in extending the features but it sure looks like you're on the right track!


Techlord(Posted 2004) [#32]
New BlitzScript3D Update


N(Posted 2004) [#33]
Are Frank Taylor and Synchronist the same person? They sound so much alike.


Synchronist(Posted 2004) [#34]
@Noel, we're two different people. It's just that Great Minds think alike. 8^)

@Frank, looking good! I'm messing about with it by coding a solar system sym in B3D, BasicGL and BSVM. It's interesting.

A suggestion? You might consider adding some version numbers to the filename...


Techlord(Posted 2004) [#35]
Sychronist,

hehe. Your right. Changes are very rapid on both code and docs. Its usually best to replace all files. I'll startup a worklog with the most recent version changes.

Noel,

Now why on earth would I purchase two copies of Blitz3D just to chat to myself on the forum? LOL


Uhfgood(Posted 2004) [#36]
I always wondered what the real purpose of scripting langauges were, when you could just use the development enviroment you have, recode and recompile. I guess modding for people that just have your game and not access to blitz (or any other de for that matter), would be a good use for scripting. If you're a programmer, is there anything scripting does for you if you're already working with blitz3d and/or some other language/enviroment?


Synchronist(Posted 2004) [#37]
One the good things about Frank's system is that you can change a script and reload it into the application while it's still running. This allows you to modify program behavior to some extent without having the "modify/compile/run" cycle each time. It might not amount to much time saved in a rather static enviroment, but it you were tweaking say, AI algorithms, the time saved could be substantial.


Techlord(Posted 2004) [#38]
A Scripting System provides a encapsulated programming interface to the app/game core. This allows you modify and extend the applications behavior without modification to core components. Mods, Plugins, and Macros, are all good examples of the benefits of a scripting system.

Many modern software packages now include a scripting system: Quake - QuakeC, Unreal - UnrealScript, Microsoft Office - VBScript, 3DStudio Max - MaxScript, Torque - Torqescript, A6 - Quake C, Maya - Mel, Scriptable Installers, and many others. IMHO, pratically all software can benefit from a Scripting Engine of some sort.


Uhfgood(Posted 2004) [#39]
YEah I know most games have a scripting system in them, always just curious about it... writing a program in another program, sounds odd to me.


Techlord(Posted 2004) [#40]
Multi-Dimensional Arrays Are Coming! I've work out the math to convert multiple indexes into a linear index for use with Blitz Arrays (ie: mytype.alien[100]). The code below pertains to the BlitzScript3D Engine but, may be useful for others.

This code snippet, illustrates the formula that will compute the offset into memory for an array with any number of dimensions:
Graphics 1024,768
n=Input("Number of Array Dimensions: ")
ascii= 96
element$=Chr(ascii+loop+1)
dimensions$="a"
For loop = 1 To n
	Print "Dim Array("+dimensions+")"
	Print "linear_index=base_index+"+String("(",loop)+"a_index)" + expressions$ + "*element_size_in_bytes"
	Print 
	element$=Chr(ascii+loop+1) 
	expression$="*"+element$+"_size+"+element$+"_index)"
	expressions$=expressions$+expression$
	dimensions$=dimensions$+","+element$
Next
Application of the formula in the BlitzScript3D Engine.



Techlord(Posted 2004) [#41]
...


Techlord(Posted 2004) [#42]
Preprocessor

BlitzScript3D Engine now features a Case-Sensitive Mass Text Replacement Preprocessor. Although extremely simple, the Preprocessor allows you to code with BlitzScript3D in very interesting ways. It also works hand in hand with 'Const' keyword allowing you to replace labels with values, strings, and expressions.

Multi-Dimensional Arrays are pratically complete and will be available in the next update. Once they're in and have been fully tested, I will be implementing functions. I have the framework for functions in place, just have to wrap my brain around dealing with local vars.

BlitzScript3D is looking really good and the performance isn't very shabby.


Synchronist(Posted 2004) [#43]
Lookin' good Frank... 8^)


Conan(Posted 2004) [#44]
Yep! Real cool!


Drey(Posted 2004) [#45]
Yea, scripts are sweet for level editors. Say u're making a level and u want the AI to change objectives and object to start moving when you flip a switch. And editor with an scriptor should make this easy if i'm understanding this right. Say, u want a wave of enemies to bust down from a roof and turn on once u stop on some trigger. Well, on the trigger, u could put "code" "in" to it. Then once the collision happens, it'll kick the sequence u have programmed in there. There's ways to hardcore it, but u can make modifications really easy with editor and scriptor. The point of a script isn't to replace hardcoding. It's to allow flexiblity and sweet events to happen without having to give a blitz3d compiler to the world(which u'll get in trouble for).

So far, BlitzScript3D seems to be pretty similar to what i was preparing for myself. But the biggest difference is, it would also have functions that are in my engine. control over the Actor, Collision, Health Modifiors, Armor, Lighting, Particles, Sound, Movers/Waypoint Systems.
So it'll also have say "Actor_LightPackage"

Event Collision_Box_5( ActorHandle )

Actor_SetLightPackage(Actor, (New Light Package) ) ; sets new lights for that room that will be applied to the actor.

End Event

I'm doubting that BlitzScript3d could access my types that are hardcoded in my engine. So i'm probably still have to create a wrapper for those correct?


Techlord(Posted 2004) [#46]
I'm doubting that BlitzScript3d could access my types that are hardcoded in my engine. So i'm probably still have to create a wrapper for those correct?

SortOf, considering the BlitzScript3D Engine is compiled with your Blitz App. You will have to 'hardcode' access to them in BlitzScript3D Compiler and VM to expose them within scripts. To do this, you will have to:

1) Add a "Command_Label" and Command Index to the bscvmCompilerCallIndex. This allows the Compiler to recognize the 'command' within a script as a valid function.

2) Add a Command Call to your function in the bscvmThreadCallIndex to be executed by the VM. This tells the VM what command to execute when processing the compiled script.

Thats it. These functions are just two big select case structures. They provide a lot of flexibility and they're fast. You can get a greater idea if you look at the code modules, but, I'll explain here as well.
bscvmCompiler.bb:  Function bscvmCompilerCallIndex

Purpose: Compiler Call Selector. Makes Commands Calls recognizable by compiler from scripts.
The Return values match the Command Index# called in the bscvmThreadCallIndex. You can easily
 add new commands to the scripting engine by adding a case here and the bscvmThreadCallIndex.

Add a New Command:
	Case "your_command_label" Return Index#
		
Parameters: bscvmCompiler Object

Return: bscvmThreadCallIndex#


bscvmThread.bb: Function bscvmThreadCallIndex 

Purpose: VirtualMachine Call Selector. Called by bscvmThreadProcess to executes Command calls.
Parameter values are stored in the this\arg[0-15] and the Return value is stored in this\ax. 
The Select Case Values are matched to the command labels in the bscvmCompilerCallIndex. 
You can easily add new commands to the scripting engine by adding a case here and the 
bscvmCompilerCallIndex.

Add a Command w/ Arguments:		
	Case Index# Your_Function(this\arg[0])

Add a Command w/ Return Value:	
	Case Index# this\ax=Your_Function(this\arg[0]) 

Add a Command with no Arguments:
	Case Index# Your_Function()

Add a Command w/Return Value and no Arguments:
	Case Index# this\ax=Your_Function() 

Parameters: bscvmThread Object
Return: None.
The VM passes and returns string, float, and integer values with functions. If your functions work with types internally, you may have to use Object/Handle command to pass or return a specified type or use globals. I personally use a Array_Of_Types Indexing System. Anyhow, the two steps above is all that is needed to expose your 'hardcoded' functions, arrays, variables, etc within script.

I'm going to eventually append all the code modules into one file, so all you have to do is Include "BlitzScript3D.bb", BlitzScript3DStart() {initialize}, BlitzScript3DUpdate() {main loop}, and BlitzScript3DStop() {shutdown}.


Drey(Posted 2004) [#47]
yea, i figured i'll have to do something like that. that's fine tho. Hmm, very interesting. I have your system on my computer. I'm checking it out.

It's pretty much like the system i wanted to make. So i figure i'll just use it. I just want short lil rounties and start ups occur when events occur. If u programmed with visual basic before, then you'll pretty much have an idea on how i want the editor to work. Timers and all that sweetness.

I knew making something a flexible has i want would make a while, but your scripter took a big part of the work out. Thanks for making it free. :-D.


Drey(Posted 2004) [#48]
I've been thinking about the scripting system. I guess it really depends on the developers goals. All this examples are just thoughts. I haven't tried them out yet.

If one's goal is to make there game modible, then the Script System is good because it shouldn't get you into legal troubles.

If you're only consider about making the game yourself tho, no community. There's other options too. I was thinking this.

So the idea has 3 levels. Engine Style Includes, Main, Custom Hardcoding for levels.

Basicly, the idea is u can have an .exe per level if u're cool with that. U'll have a StartUp.exe. That's like your main menu or whatever. Then, when u load a game or pick a stage. The custom exe gets called and StartUp.exe is ended.

In your custom exe, U'll probably have this
Include "Engine" ; The custom types and functions u've made.
Include "Main" ; controls and such

**now you put your custom per level coding here"



This would be faster than using a script system. Just bigger overall because blitz3d doesn't make DLLs( :( ).

If blitz3d has memory addressing for it's types( not this array handle stuff ). U could actually have 2 .exes running per level. So say u're loading your level, while the types and entities are being made, u send there address to a .txt file. Then in your other .exe(the script), u load up the address from your main.exe into your level(x).exe and start your checks( this probably could work for entites, not types tho ).
Because level.exe doesn't generate graphics on it's own screen and such, it shouldn't slow anything down really.

Again, DLLs would make this easier and cleaner, but i think u guys get the idea.

Just wanted to share some thoughts.


Techlord(Posted 2004) [#49]
Drey

I intend to use scripting for Interactive Logic and Macros in my Games and Applications. Rendering and other performance critical functions will be hardcoded. So in other words, the Script tells the engine "StartParticle(fire,x,y,z)" and the Engine generates sprites, textures them, orientates them, and culls them when out of view.

Most modern day games comprise of 3 elements: Engine, Media Data, and Logic Data. The 'Engine' manages network, rendering, collision, and sound. 'Media Data' provides the external Game Content. 'Logic Data' defines Interactive GamePlay. Scripting provides a means of of creating Programmable Logic Data.

Each element is encapsulated so that they can be combined to create new games. If you hardcode any two of these elements together you will have to restart from scratch if you want to use them in a another project. Even if your making the game for yourself, you may want to employ the 'Engine' in one project, Media Data and Logic Data in others.


Techlord(Posted 2004) [#50]
New BlitzScript3D Update 123004


Techlord(Posted 2005) [#51]
Was digging through some old code and found by old ACTIONBuilder code. The purpose of the ACTIONBuilder was to scan *.bb files, find Functions, and generate a Function Call Select...Case Tree from them. Saves lots of time adding a large number of functions to the FunctionCall Select...Case.

I've decided to add this utility to BlitzScript3D as well. I'm totally redesigning it. It will make adding functions to the Scripting Engine extremely simple. I've separated the compiler and vm function callindex functions into independent files. These functions are just two large select case structures used in Script Engine to identify commands in script and execute them in the VM.

The new bscvmCallBuilder utility will generate a compiler and vm function callindex file. So no messing around with the code and tracking index numbers. Just run bscvmCallBuilder and it will parse the *.bb and *.h (prototype) generate compiler and vm function callindex files for you.

Found it extremely useful when adding other code lib functions to the scripting engine.


Jeremy Alessi(Posted 2005) [#52]
How was that assembly code working?


Techlord(Posted 2005) [#53]
bscvmCallBuilder complete and totally out performs its predecessor ACTIONBuilder. This utility will pratically automate adding functions and macros the scriptengine.

How was that assembly code working?
Im not sure what assembly you refering too. Please elaborate. Thnx


Jeremy Alessi(Posted 2005) [#54]
You posted opcodes above ... looked like assembly language. Is that just for the virtual machine?


Techlord(Posted 2005) [#55]
Those opcodes were the starting point for my VM. The opcodes instuct the processor how to move data on the stack and registers. The VM mimics a processor consisting of a stack array and register variables.

I dont use the code specifically, but they give a idea what direction to go in. I'm quite happy with me results. just have to get functions in.


Techlord(Posted 2005) [#56]
I found several bugs in the scripting system that have demanded my immediate attention. The If...Then...Else is limited and failing. This is basis for simple logic and must work with no errors. Select...Case on hold. As soon as the resolve the If...Then problems I'll resume getting functions in

In the meantime, I've added 'BlitzScript...EndBlitzScript' Keywords. These keywords allow you to put several individual scipts in a single file verses being forced to have a individual file for each script. This can substitute for Functions for now.


Techlord(Posted 2005) [#57]
...


JoshK(Posted 2005) [#58]
This is stupid. BVM works perfectly. Use it.


Techlord(Posted 2005) [#59]
Josha,

Remember the good ole days of Metal?


N(Posted 2005) [#60]
Frank,

Remember the good ole days of all your community projects never being finished?


Techlord(Posted 2005) [#61]
Noel Cower (Posted 2005-01-16 16:13:00)

Frank,

Remember the good ole days of all your community projects never being finished?
This isn't one of them.


N(Posted 2005) [#62]
This wasn't a community project either.


Techlord(Posted 2005) [#63]
Nope. I only worked on Project Plasma FPS and recently started PW3D as a Community Project. So when folks start blabbing about how many unfinished Projects there are, all I can do is chuckle out loud. As I told many before a project never really dies, it just changes names.


JoshK(Posted 2005) [#64]
Remember the good ole days of Metal?

Yes:
http://www.idigicon.com/products/pdetails.asp?id=80


Techlord(Posted 2005) [#65]
Excellent!


JoshK(Posted 2005) [#66]
Anyways, the point is, I finished it.

Now if only I had the same art budget that they had to design the box with.


Ross C(Posted 2005) [#67]
lol


Bouncer(Posted 2005) [#68]
Are Noel Cower and Halo the same person?? They both sound like retards.


Techlord(Posted 2005) [#69]
I manage to hunt down and kill the If-Then-Else Bug that was infesting BlitzScript3D. The problem was between both the compiler and VM in the assignment of the instruction loading.

Now that its resolved I can really see how limited my current implementation of 'bscvmLabels' really is. You cannot do anyting fancy. In fact, your limited to evaluating a single expression at a time which makes for pain in the arse scripting.
Example:
If n=1 Then dostuff()
If n=2 Then dostuff()
If n=3 Then dostuff()
;you get the point
This has taken priority over implementing functions because without Labels working properly you cannot jump back and forth between instruction offsets. Jumping is needed for Unconditional Loops, Conditional Loops, Nested Conditions and Functions. I'm to work on expanding this functionality asap. When complete you should be able to support Nested If..Then...Else...EndIf and 'Label & Goto' Keywords.

In the meantime, I've added 'BlitzScript...EndBlitzScript' Keywords. These keywords allow you to put several individual scipts in a single file verses being forced to have a individual file for each script. This can substitute for Functions for now.

The bscvmCommandInstaller completey and totally out performs its predecessor ACTIONBuilder. This utility will automate adding functions and macros the scriptengine and its huge work saver.

Expect an update within the week.


Clarks(Posted 2005) [#70]
Frank Taylor:

im in the works of creating my own virtual machine and i already ran into problems with if statements.

ex.(if n = 3)

how exactly would that piece of code be turned to byte code with out some sort of coruption. the compiler needs to know that theres a comparison going on and not an assignment.

load n
pushint 3
comp
if_false_jump (to someline)

how exactly does the compiler know what line the vm should jump to.

and what about the keywords( and, or , xor, not)


Techlord(Posted 2005) [#71]
Clarks,

Well, this is going to deserve an in depth answer and it would be easier to explain if we could compare notes. I have one question: is your language compiled or interpreted?

I started with Mark Sibly's Compiler Code because it had a excellent way of parsing expressions and tokenizing BASIC statements, keywords, etc and output instructions. Once you are able to tokenize the statements and write instructions with your compiler, its easier to construct the VM. It took a lil research on assembly to get a good understanding of what the instructions do.

Armed with a some info on Assembly, I designed a simple VM that consists of a couple Arrays (Stacks), a couple of Variables (Registers), and Constants (Instructions). The VM moves values in / out of variables, push/pop values on and off the Stack, performs simple compares (=,<,>,etc), perform simple math operations (*,/,+,-), and perform instruction jumps. The compiler outputs the instructions to do this and these instructions are the VM's Constant Instruction Values.

how exactly does the compiler know what line the vm should jump to.


So with this background info I can tell how labels work in BlitzScript3D. The compiler outputs an instruction called BSCVM_OP_LABEL and a operand that is the ID of that Label. When the VM loads the compiled script and comes across the BSCVM_OP_LABEL instuction, it creates a label, assigns it the id and stores the Instruction Pointer value to the label.

When the VM executes each instructoin stored in the Instruction Array. Instruction Pointer is incremented to the next instruction. So when the VM runs across the BSCVM_OP_LABEL, BSCVM_OP_JZ or BSCVM_OP_JMP instructions, the it accesses the Label's ID and sets the Intruction Pointer valued stored in the Label.


__# = Label ID


Clarks(Posted 2005) [#72]
I also started with marks code too.
my compiler takes the source and compiles it to byte code.
the compiler is not fully developed yet. the vm will take the byte codes and executed it. so its interpreted.

label for labels, jmp for jump, what does the jz stand for.

So with this background info I can tell how labels work in BlitzScript3D. The compiler outputs an instruction called BSCVM_OP_LABEL and a operand that is the ID of that Label. When the VM loads the compiled script and comes across the BSCVM_OP_LABEL instuction, it creates a label, assigns it the id and stores the Instruction Pointer value to the label.

When the VM executes each instructoin stored in the Instruction Array. Instruction Pointer is incremented to the next instruction. So when the VM runs across the BSCVM_OP_LABEL, BSCVM_OP_JZ or BSCVM_OP_JMP instructions, the it accesses the Label's ID and sets the Intruction Pointer valued stored in the Label.



can you explain this a little bit more.


Techlord(Posted 2005) [#73]
Clark,

A little reading on 80x86 Assembler Codes will help a great deal. I'm curious as to how you are executing instructions. If you havent downloaded Blitzscript3D, do so, and look over the bscvmCompiler.bb and bscvmThread.bb files. Heres what my VM looks like (ignore the hex codes their useless):

Heres the actual VM object. Each script gets its own self-contained VM 'Thread' consisting of its own stacks, registers, and flags.
Type bscvmThread
	;Purpose: bscvmThread Script Object.
	Field id%
	Field parentid%
	Field typeid%
	Field label$
	Field filename$
	Field instruction$[BSCVMTHREAD_INSTRUCTION_MAX%]
	Field instructionPTR%
	Field instructionsize%
	Field stack$[BSCVMTHREAD_STACK_MAX%]
	Field stackPTR%
	Field arg$[BSCVMTHREAD_ARG_MAX%]
	Field call%
	Field ax$
	Field cx$
	Field zero%
	Field run% ;state -1:Infinite loop, 0:halt, >0:Specified Loops  
	Field schedule%
	Field scheduleCTR%
End Type
The instructions are stored the bscvmThread\instruction[] Array. When 'executed' the VM loops through the elements in the instruction array. This is handled by the bscvmThreadUpdate() Function
Function bscvmThreadUpdate()
	;Purpose: Updates all Virtual Machine bscvmThreads see bscvmUpdate Function 
	;Parameters: None
	;Return: None
	For this.bscvmThread=Each bscvmThread

		If this\schedule>0
			this\scheduleCTR=this\scheduleCTR+1
			If this\scheduleCTR=this\schedule 
				this\run=1
				this\scheduleCTR=0
			EndIf	
		EndIf
		
		If this\run
			Repeat
				bscvmThreadProcess(this)
			Until this\instructionPTR>this\instructionSize
			this\instructionPTR=0 ;reset instruction
			If this\run>0 this\run=this\run-1
		EndIf	
		
	Next
End Function
The bscvmThreadProcess() Function selects the Instruction and manipulates the Instruction PTR, Stack, AX, and ZeroFlag.

I simplified the Compiler code generated for comparisions.
cmp  ecx,eax
slt  eax
and  eax,255
and  eax,eax
Is equal to
BSCVM_OP_SLT


BSCVM_OP_JZ or ("JZ") stands for 'Jump if Zero'. As you can see the VM uses a variable that represents the Zero Flag in a CPU. The IF and ELSEIF keywords generate comparision codes: SEQ, SLT, SGT,SNE,SGE, SLE and jump code: JZ.

When the VM executes the Comparison Code it performs the comparison on the values in the AX and CX 'Register' vars. If the comparison fails it sets the Zero Flag to 1 otherwise it remains 0. When the VM executes the JZ instruction it will perform a JMP if the Zero Flag is set True, otherwise it will not. JMP is an unconditional Jump.

The confusing part is that VM performs the comparisions in a inverted fashion to set the Zero flag. So SEQ doesnt mean if ax=cx zero=True, It means if ax<>cx zero=true . It took a while for me to get this to work. I'm currently rewritting this entire portion of the Compiler, because IF...ELSE. expressions are very limited. You can not compile multiple statement conditions, ELSEIF, and Nested IF...ELSE, etc. The VM doesnt require any rewrite because the instructions generated are the same, just the order is different. As soon as I get it working I'm going to make an official update. If you want the 'unofficial' wip.zip let me know and I'll post a link for DL.


Clarks(Posted 2005) [#74]
80x86 Assembler Codes:

i took a look at them, my head started to hurt.

i already had downloaded your blitzscript3d, and viewed the demo, impressive.

im not executing any instructions yet, i didnt even start coding the vm. im just looking for ways to compile a file very efficiently and offer flexiblity and power in the language. trying to make sure that the codes produced are efficient and effective, easy for the vm to understand and execute as fast as possible. for instance, the (cmp ecx,eax) is not how i had plan to do it, but your overview and your source opened my eyes up some.

best luck on BlitzScript3D, if i continue with my project ill give you an update.


Techlord(Posted 2005) [#75]
i took a look at them, my head started to hurt

Clarks,

My head still hurts - hehe. That probably wasn't the best article. I read several articles before I really started to get a feel for it.

I'm nearly complete with this version of BlitzScript3D. I'm very interested in watching your project evolve as well. I plan on using the BlitzScript3D Engine as the scripting solution for several Games and Applications.

Ultimately, I hope others find BlitzScript3D as useful learning tool. I've learned a great deal about programming from working on this project. My view on coding is much different. I feel enlightened. I'm sure you will experience the same enlightenment if you continue your project as well.


Techlord(Posted 2005) [#76]
New BlitzScript3D Update 012005


Techlord(Posted 2005) [#77]
Anyone using BlitzScript3D? I will be gearing up to add features on the TODO List in the Worklog. Would like to get some feedback if possible.


John J.(Posted 2005) [#78]
I would like to have used BlitzScript3D in some of my projects, but unfortunately it doesn't support the features I need. BlitzScript3D would be much nicer if it had function, custom type, and conditional loop (while..wend) support.

Since it didn't look like BlitzScript3D was ever going to support these, I actually just began writing my own scripting engine just a couple of days ago. Currently, my compiler (I haven't even began programming the VM yet) supports number+string expressions, and if..then statements, in addition to custom type support with New and Delete keywords. I soon plan to impliment select..case conditions, conditional loops, overloadable functions, and an OPcode optimizer (I already know exactly how the optimizer is going to work).

If you would like to continue developing BlitzScript3D, I might be able to help you a little with some of these features if you plan to impliment them, although I doubt I could be any help since my compiler is probably totally different from yours (my scripting language's syntax is more BlitzMax style than BlitzBasic style).


Techlord(Posted 2005) [#79]
John J,

I took a short break:) Functions and Conditional Loops are on the agenda. I need these features as well.

I'm very interested how you are designing your Scripting Engine. I would like to compare notes. Although different in code, they may be very similar in operation. I'm completely open to any advice you have on adding custom types, select..case conditions, conditional loops, overloadable functions features to BlitzScript3D.


John J.(Posted 2005) [#80]
I would do select..case conditions and conditional loops basically the same as I did if..then conditions: the "TOKEN_IF" case takes care of the entire if..then..else..endif" and all it's statements just like Mark Sibley's example. Here's my if..then..else..endif code (single or multi line):

	Case TOKEN_IF
		GSC_ParseExpression()
		If GSC_TokenType <> TOKEN_THEN Then
			GSC_AddError("Expected "+Chr(34)+"Then"+Chr(34))
			GSC_SkipLine(): GSC_NextToken()
			Return
		End If
		LabelElse = GSC_UniqueLabel()
		WriteLine GSC_OutputFile, "    jz    _" + LabelElse
		
		GSC_NextToken()
		If GSC_TokenType = TOKEN_TERMINATOR Then
			;Multi-line IF statement
			Repeat
				GSC_ParseStatement()
				If GSC_Done Then
					GSC_AddError("Expected "+Chr(34)+"Else"+Chr(34)+" or "+Chr(34)+"EndIf"+Chr(34))
					Return
				End If
			Until GSC_TokenType = TOKEN_ELSE Or GSC_TokenType = TOKEN_ENDIF
			If GSC_TokenType = TOKEN_ELSE
				LabelEndif = GSC_UniqueLabel()
				WriteLine GSC_OutputFile, "    jmp   _" + LabelEndif
				WriteLine GSC_OutputFile, "_" + LabelElse + ":"
				GSC_NextToken()
				If GSC_TokenType = TOKEN_IF Then GSC_AddWarning("Ambiguous coding style (may cause unexpected errors - see language manual)")
				Repeat
					GSC_ParseStatement()
					If GSC_Done Then
						GSC_AddError("Expected "+Chr(34)+"EndIf"+Chr(34))
						Return
					End If
				Until GSC_TokenType = TOKEN_ENDIF
				WriteLine GSC_OutputFile, "_" + LabelEndif + ":"
				GSC_NextToken()
			Else ;EndIf
				WriteLine GSC_OutputFile, "_" + LabelElse + ":"
				GSC_NextToken()
			EndIf
		Else
			;Single-line IF statement
			GSC_ParseStatement(False)
			If GSC_TokenType = TOKEN_ELSE
				LabelEndif = GSC_UniqueLabel()
				WriteLine GSC_OutputFile, "    jmp   _" + LabelEndif
				WriteLine GSC_OutputFile, "_" + LabelElse + ":"
				GSC_NextToken()
				GSC_ParseStatement()
				WriteLine GSC_OutputFile, "_" + LabelEndif + ":"
			Else
				WriteLine GSC_OutputFile, "_" + LabelElse + ":"
			EndIf
		End If
		
	Case TOKEN_ELSE, TOKEN_ENDIF
		GSC_AddError("Keyword is out of place (there are no open "+Chr(34)+"If"+Chr(34)+" statements)")
		GSC_NextToken()
		Return

Note: As you can see, GSC_NextToken() does not return anything! Instead of returning the token type, I find it cleaner and more logical to store the token and token type in global variables (GSC_Token, GSC_TokenType).

I haven't gotten to overloadable functions yet, but I don't think I'll have much trouble with the overloading part; just search all functions for the best match. For the actual function part I'm not sure - probably just as simple as a call stack and "scoping" variables properly.

To impliment custom types I had to "invent" a few special opcodes. These opcodes move a type field to/from another variable/eax/ecx:

mfc  a, b
  Moves the contents of the ecx."b" (type ecx, field "b") into "a"

mfa  a, b
  Moves the contents of the eax."b" (type eax, field "b") into "a" (same as mfc except uses eax instead of ecx)

xfc  a, b
  Moves the contents of "b" into ecx."a" (type ecx, field "a")

xfa  a, b
  Moves the contents of "b" into eax."a" (type eax, field "a") (same as xfc except uses eax instead of ecx)


Then the compiler can use these to compile code like this:
Type Car 
  Field a:Int, b:Float
EndType

Local test:Car, tmp:Float

test = New Car

test.a = 23

tmp = test.a * 10

Delete test


Into this:

;test = New Car
    new   eax,{car}
    mov   [test], eax
;test.a = 23
    mov   eax,23
    mov   ecx, [test]
    xfc   <a>, eax
;tmp = test.a * 10
    mov   eax,[test]
    mfa   eax,<a>
    push  eax
    mov   eax,10
    pop   ecx
    mul   eax,ecx
    mov   [tmp], eax
    mov   eax,[test]
;Delete test
    del   eax

    end



N(Posted 2005) [#81]
Frank, have you read this?


Techlord(Posted 2005) [#82]
Noel,

Thanks for the article it was very informative. My reasoning behind developing BlitzScript3D was similar to Rik's with one additional facet. I wanted the Scripting Engine to be written in Blitz3D. No Dlls. This would serve as a learning tool and show that Blitz3D can meet the challenge. This is ultimately my goal with all my projects.

John J,

My idea of implementing SELECT...CASE is also similar to the IF...ELSEIF...THEN.
get varname			;select							
If var=value		;first case
ElseIf var=value	;next case
ElseIf var=value	;ditto
.
.
.
Else				;default case
EndIf				;end select
SELECT CASE will most like be my next feature to add. I noticed that your IF...THEN can handle multiple lines. However, the following line concerns me.
If GSC_TokenType = TOKEN_IF Then GSC_AddWarning("Ambiguous coding style (may cause unexpected errors - see language manual)")
.This suggest that the implementation does not support nested IF..THEN. Please correct me if I'm wrong. Handling Nested SELECT...CASE is also something to consider.

Conditional Loops. I do have a primitive form of looping using labels and goto commands. Its easy to home brew your own conditional loops but, too unconventional. I plan to implement FOR...NEXT, Repeat...Until, While...Wend loops based labels, jumps, and if...then contructs. I will also going to experiment with Hyper SELECT...CASE based on Label Jumping. The reason behind this is to reduced the number of instructions to evaluate conditions.

I haven't quite figured out how I will implement Functions less alone overload them. So I'm going to think tank for this. With BlitzScript3D being datatype-less, it may be difficult to implement overloading. The only way i can see this is if i implement them like methods in class and it will be limited (based on number of parameters vs type of parameters).

The way you implemented Custom Types looks very interesting. This is were I can see BlitzScript3D datatype-lessness making implementation easier. If I do pursue types, I will most likely pursue simple object classes in which you can package properties and methods under a object name. I always been a fan of the (.) syntax:
 
object = New model
object.entity = entityload("dude.b3d")
object.rotate()

vs

object.model = New model
object\entity = entityload("dude.b3d")
object\rotate() ;doesent exist
I would also build in ID Reference Management for fast access to individual objects. Custom Type implementation depends on how it impacts performance.

I'm very interested in how you plan to implement the VM. I assume your compiler will convert scripts to opcodes and the VM loads in opcodes for processing. My compiler converts scripts directly into pcode (a byte representation of the opcodes). This is the compiled version of the script. BlitzScript3D can either recompile the script at loadtime or just load the compiled script. This allows you to totally remove human readible scripts and just used the compiled versions.


John J.(Posted 2005) [#83]

However, the following line concerns me.

If GSC_TokenType = TOKEN_IF Then GSC_AddWarning("Ambiguous coding style (may cause unexpected errors - see language manual)")


.This suggest that the implementation does not support nested IF..THEN. Please correct me if I'm wrong.


This isn't exactly what it seems. Testing the next token for "If" only detects if the programmer typed something like:

If <condition> Then
  <statements>
Else If <condition> Then 'This is not elseif to the compiler
  <statemtnts>
EndIf

, which is the same as:
If <condition> Then
  <statements>
Else
  If <condition> Then 'Unclosed if

EndIf


, since I haven't implimented any ElseIf behaviour. The reason an "Else" followed by an "If" on a new line would not generate this warning is my compiler returns TOKEN_TERMINATOR when a new-line character (chr(13)) is found OR ";" is found (the equivelant of Blitz3D's ":").

I intend the VM to read Pcode and the compiler to write to Pcode as well (I mistook OPcode and Pcode as being synonomous). The file "manipulation" scheme I plan to impliment is slightly similar to BlitzScript3D:

The compiler writes the date/time stamp of the script file to the Pcode file. When the VM finds only the Pcode file, it will run that. If it finds only the Script file, it will compile and run that. But, if it finds both the script and Pcode file, it will check to see whether the Pcode file is up-to-date (if the date/time stamp of the script file equals the one writen within the Pcode file). If the Pcode is up-to-date, there's no need to recompile and can simply load that. If not, the souce will be recompiled (and therefore will not be recompiled until the source code's date/time stamp is changed by an edit).


Techlord(Posted 2005) [#84]
John J,

I like the idea of checking the timestamp on the script to determine recompile. I did not consider this. For BlitzScript3D you have to explicitly tell it to recompile, just in case you want to work on script but not modify the compiled version. However, a timestamp check is better.

IF Statements. I was concerned about nested IFs
If <condition> Then 
	If <condition> Then 
		If <condition> Then 
			value = True
		EndIf
	EndIf
EndIf
This usage is not the same as Else...If. I ran into great deal of headache implementing nested conditions until i realized the power of labels and jumps.


John J.(Posted 2005) [#85]
My implimentation supports nested "If" statements since the "If" code calls ParseStatement() for each line between "If" and "Else", "Else" and "EndIf". And since ParseStatement() handles "If"s, nested "If" statements are automatically handled (just as long as the label names don't clash - and they don't due to the GSC_UniqueLabel() function). What my previous post stated was that that line simply helps avoid confusion about "Else If", since I haven't implimented "ElseIf" behavour.

The idea about checking the timestamp isn't new, I don't think, since that's the way I thought most compilers worked (most C compilers only recompile source files that have been modified).


skyfire1(Posted 2005) [#86]
i tried blitzscript3d. now i'm off on a quest to figure out how i'll need it.


N(Posted 2005) [#87]
You won't need it.


skyfire1(Posted 2005) [#88]
why not?!!!


Techlord(Posted 2005) [#89]
John,

I follow you now and Nested IF's looking good. I found a very bad bug in running compiled scripts without recompilation:( I almost had to redo the entire variable handling.

Its not looking good for Custom Types. I dont have a clue as to how i want to handle the dynamic mem allocation. Instead of drilling my brain on them, i moving on with Select...Case Implementation.

skyfire1,

A Scripting Engine can be used for 'Modable' game engines, Macros, and other User Programmable behavior without modifying the actual Game/App Source Code.

If you take a look at some the most powerful game engines and applications, they all include a scripting system.


John J.(Posted 2005) [#90]
Today I added function support! Functions now compile perfectly, supporting both Local and Static variables. It supports blitz's style of functions (with or without parenthisis enclosing parameters).

The way it works is actually simple. Before a function is called, all Local variables of the function are "pushed", all the parameter values are "pushed", the execution pointer is pushed to the call stack, and it "jmps" to the function. Within the function, it "pops" the parameters into the function's parameter's variables. When it returns ("pops" the execution pointer off the call stack), it "pops" back all the functions local variables.

I probably didn't explain that very well, but it basically uses the stack to store the parameters, and to restore variables when using recursion techniques.

Unfortunately, I didn't add overloading support. It turns out it wasn't as easy as it seemed. To add function overloading I would have to add a third file parse pass, and I didn't think the advantages of overloadible functions justify the added compile time delay. Currently, one pass reads in variable and type declarations, and the second pass compiles the actual code. I've timed it and it compiles at about 50-70 KB of code per second.

So, it looks like conditional loops like "For..Next", "While..Wend", and "Repeat..Until" are next, then I'll be pretty much done with the compiler, and I'll begin work on the VM :)

skyfire1: Noel says you won't need it probably because he doesn't personally see any use for it. That's perfectly fine, except I would encourage you to research the advantages/disadvantages of a scripting engine in your project. I am making my own scripting engine because I think many aspects of my game would benifit from it greatly (especially AI), in addition to the fun I have programming things like this :)


Techlord(Posted 2005) [#91]
John,

Your explanation on Functions make perfect sense. I push and pop off args to regular functions in the same manner. I just havent quite figured out how to handle local variables at the moment. I knew that function overloading would be tricky.

I use a very crude preprocessor (mass text replacement) on the 1st pass of the script and compile it to pcode on the second. If you can accomplish a 3rd pass within reasonable time, i say go for it. Afterall, once the script is compiled it loads first with no additional passes.

So you're moving on to conditional loops. I have to assume you already have Select...Case working? I'm real curious as to how you worked out the opcode sequence.

PS: Dont forget Multi-Dimensional Arrays and Logical conditions.


John J.(Posted 2005) [#92]
Select..Case compiles from this:
Local a:Float

Select a
Case 1
  a=2
Case 2, 0
  a=1
Default
  a=0
EndSelect

, to this:
;Select a
    mov   eax,[a]
    mov ebx, eax

;Case 1
_Flow_2
    mov   eax,1
    cEQ eax, ebx
    jz  _Flow_3

;a=2
    mov   eax,2
    mov   [a], eax
    jmp _Flow_1

;Case 2
_Flow_3
    mov   eax,2
    push  eax
    mov   eax,0
    pop   ecx
    or    eax, ecx
    cEQ eax, ebx
    jz  _Flow_4

;a=1
    mov   eax,1
    mov   [a], eax
    jmp _Flow_1

;Default
_Flow_4

;a=0
    mov   eax,0
    mov   [a], eax
    jmp _Flow_1

;EndSelect
_Flow_5
_Flow_1

    end


It's pretty simple, really. The hardest part was writing the parser (it wasn't really that difficult, it just took a bit of thought). I'd show you my Select..Case code, but you'll probably want to program yours yourself :)

About logical operators (and, or, xor, not), I implimented those long ago. As to miltidimentional arrays, I haven't even given them any real thought yet, but I'll probably impliment them last.


Techlord(Posted 2005) [#93]
John,

WOW! You totally comprehend Opcode. Its hard to believe you just started a writting your own Scripting Engine less than a week ago. I struggled for weeks just implementing basic IF...THEN statements (BlitzScript3D has yet to support Logical Conditions.) If you can work out the opcodes, its no trouble to work the vm.

I'm going to reintroduce myself to the BlitzScript3D Compiler Opcode:)


John J.(Posted 2005) [#94]
Yep, I learned how the opcodes work the first day I started my scripting engine by studying Mark Sibley's compiler code. Before then all I knew about opcodes were they moved memory around and used some thingey called a "stack" :)

Anyway, I now have While..Wend, Repeat..Until, Repeat..Forever, and For..Step..Next loops working fine.

Here's how the loop code compiles:
Local a:Int, b:Int

'--------------------------------------
'While..Wend
'--------------------------------------
While a<10
  a=a+1
Wend

'--------------------------------------
'Repeat..Until
'--------------------------------------
Repeat
  a=a-1
Until a=0

'--------------------------------------
'Repeat..Forever
'--------------------------------------
Repeat
  a=a-1
Forever

'--------------------------------------
'For..Step..Next
'--------------------------------------
For a = 1 To 10 Step 2
  b=1
Next


Compiled:

'--------------------------------------
'While..Wend
'--------------------------------------

While a<10
_Flow_2
    mov   eax,[a]
    push  eax
    mov   eax,10
    pop   ecx
    cLT   ecx,eax
    jz  _Flow_1

;a=a+1
    mov   eax,[a]
    push  eax
    mov   eax,1
    pop   ecx
    add   eax,ecx
    mov   [a], eax

;Wend
    jmp _Flow_2
_Flow_1


'--------------------------------------
'Repeat..Until
'--------------------------------------

;Repeat
_Flow_4

;a=a-1
    mov   eax,[a]
    push  eax
    mov   eax,1
    pop   ecx
    sub   eax,ecx
    mov   [a], eax

;Until a=0
    mov   eax,[a]
    push  eax
    mov   eax,0
    pop   ecx
    cEQ   ecx,eax
    jz  _Flow_4
_Flow_3


'--------------------------------------
'Repeat..Forever
'--------------------------------------

;Repeat
_Flow_6

;a=a-1
    mov   eax,[a]
    push  eax
    mov   eax,1
    pop   ecx
    sub   eax,ecx
    mov   [a], eax

;Forever
    jmp _Flow_6
_Flow_5


'--------------------------------------
'For..Step..Next
'--------------------------------------

;For a = 1 To 10 Step 2
    mov   eax,1
    mov   [a], eax
_Flow_8
    mov   eax, [a]
    push  eax
    mov   eax,10
    pop   ecx
    cLT   eax, ecx
    jz    _Flow_7

;b=1
    mov   eax,1
    mov   [b], eax

;Next
    mov   eax, [a]
    add   eax, 2.0
    mov   [a], eax
    jmp _Flow_8
_Flow_7


    end



Techlord(Posted 2005) [#95]
John J.

I'm totally amazed. I sure could have used this insight with BlitzScript3D initially. I can visualize the cover of your book entitled: "How to write a Scripting Engine in 7 days with Blitz3D."

I create some weird opcodes for handling labels and jumps. Your opcode output is very elegant and holds true to nature. I'm gonna leave things in that area alone for now, but, i will definately take a second look at it in the future. At least for optimizing purposes.

I getting used to looking at BlitzScript3D code now. Its time to start adding these constructs:)

Frank Taylor's Distorted Opcode for Select...Case

;Select expression
mov eax,<expression>
push eax
jz start
 
;dummycase
 
;Case <expression>
label_next
mov eax,<expression>
seq eax  ;auto pops stack -> ecx and compares
push ecx ;push ecx back to stack
jz next
;statements
jmp last
 
;Case expression$
label_next
mov eax,<expression$>
seq eax str ;auto pops stack to ecx and compare strings
push ecx ;push ecx back to stack
jz next
;statements
jmp last

;Default
label_next
;statements

;EndSelect
label_last



John J.(Posted 2005) [#96]
I'm going to begin work on the optimizer next. It will probably be seperate from the compiler; instead of optimizing as it compiles, it will optimize after the compile.

One method of optimization it will use is a mass-text-replacer. For example this might replace this:
pop ecx
cEQ eax, ecx

for this:
seq eax  ;auto pops stack -> ecx and compares

Or possibly simplify other complex operations that are performed often.

The other optimization method I planned out could reduce these rendundant instructions:
mov eax, 10
mov [a], eax
mov eax, [a]
mov [b], eax
mov eax, 5

to this:
mov [a], 10
mov [b], 10
mov eax, 5


How it works: First, it finds a group of "mov"s. Then, it lists all variables/registers used and their values internally like this:

eax = eax
a = a
b = b


Then it steps through each "mov" assignment and changes the value of the variables.

Step 1: "mov eax, 10"

eax = 10
a = a
b = b



Step 2: "mov [a], eax"
set a=eax ... eax is 10, so set a=10:

eax = 10
a = 10
b = b



Step 3: "mov eax, [a]"
...

eax = 10
a = 10
b = b



Step 4: "mov [b], eax"

eax = 10
a = 10
b = 10



Step 5: "mov eax, 5"

eax = 5
a = 10
b = 10



Done! Now translate the final data back into "mov"s:

mov eax, 5
mov [a], 10
mov [b], 10


That's just a simple example, but it shows the versatility of the "mov" optimization algorithm. I've tried the algorithm (by "hand") on code like "a=a" and other redundant code, and it works quite well.


Techlord(Posted 2005) [#97]
John,

My optimizations are in both compilation and VM. I would highly recommend developing the VM prior to the Optimizer. My reason being is, you already have compiler output, the next step is to get the output to run. I'm certain you will get a better perspective on optimizations once you see the code run.


John J.(Posted 2005) [#98]
I took your advise, and started the VM first. First, I converted the compiler output to binary. It outputs instructions like:


(1 byte)Instruction code
(1 byte)Parameter code
(x bytes)Parameter(s)

(1 byte)Instruction code
(1 byte)Parameter code
(x bytes)Parameter(s)

...
...



Anyway, the VM is coming along fairly well.

I also just added the "automatic" compile system that uses the source file's date/time stamp to check if the compiled file is up-to-date. I used (this) method by turtle1776 to get the modified time/date of a file (it also can get the created, and last accessed time, although I didn't use those). It's works perfectly, although it does require the user to put a small decls file in the userlibs folder (for access to kernel32.dll).


Techlord(Posted 2005) [#99]
John,

Thumbs Up! My instructions are formatted in nearly the same fashion. I dont output all data in bytes though. I use shorts and strings for parameters. I have yet to find a need for more than two parameters.

I'm going to omit the auto-recompile for now. If there are significant requests for the feature I'll add it later. I do not desire to use a DLL, so I will most likely place the time stamp in the compiled file or special project config file as you suggested earlier.

You wouldn't believe how hard im struggling with implementing Select Case. Its all due to my fancy-spancy opcodes:( I'm going to keep working at it til it works. I'm trying to use the opcodes I have in place without creating new ones. I'm not looking forward conditional loops either.


John J.(Posted 2005) [#100]
About the auto-recompile: I don't use any DLL's really, except for a standard windows DLL that all blitz applications probably are already using anyway (kernel32.dll). Since blitz doesn't natively support getting the date/time stamp of a file, that's what turtle1776's code does.

My code now uses this to take the date/time of the source, and write it to the compiled file. There's really no other way to do auto-recompile unless you can get the date/time of the source file (it's the only way you can know if it has been modified or not).

In my opinion, putting the date/time in the compiled file is much better than putting it in a project config file, since the source and executable are totally independant of each other and any other files (more fail-safe and bug-safe).

My recompile check compares the compiled executable's written date/time with the date/time stamp of the source file. If it is EQUAL, no compile is necessary. If it is not equal, it recompiles. This method, opposed to not compiling when the executable is "Greater than or Equal to", can avoid the possibility of having a later-dated executable with an identicle as a existing source file being used. In other words, source files (if existing) take precedance over executables.


Techlord(Posted 2005) [#101]
Getting Closer to resolving Select...Case. I'm designing it with Nested Select...Cases in mind. What I noticed in your Select...Case opcode segment above
;Select a
    mov   eax,[a]
    mov ebx, eax
is that you use a ebx register which holds the value or pointer to [a]. BlitzScript only uses eax and ecx.

Taking Nested Select...Case into consideration, the ebx register's contents will have to pushed and popped off the stack as needed with each Case comparison otherwise it will be overwritten if nested Select.Case is encountered.
;Select a
    mov   eax,[a]
    push eax

;Case 1
_Flow_2
    mov   eax,1
    pop ebx
    cEQ eax, ebx
    push ebx
    jz  _Flow_3

   {statements}
    jmp _Flow_1
This is completely dependent on how the VM Stack is implemented... I only use one.


Techlord(Posted 2005) [#102]
Yeah-hoooooooooo!

Nested Select...Cases are in. Doesn't yet support cases with multiple arguments, but its working!


John J.(Posted 2005) [#103]
I'm glad to hear you're making progress!

I didn't notice that problem with my Select..Case - thanks for finding that! I fixed it so it uses the stack instead of ebx to store the "Select" expression, similar to your example above. Instead of popping ebx, comparing, and pushing ebx when ebx is needed, I use a new opcode called "vpop" (virtual pop) which pops the top stack item into a variable without modifying the stack contents. At the end of the "Select", I "xpop", which removes the top stack item (so the Select expression doesn't stay in there causing a stack leak).

I added multiple case items a while ago by reading expressions seperated by commas, producing code as if the commas were "Or" keywords


Techlord(Posted 2005) [#104]
Moving on to Control Loops. I do have code in place to handle logical operators: OR, AND, XOR, NOT. But, I have yet to put the code in operation. I'm going to move on to Control Loops for now.


Techlord(Posted 2005) [#105]
John,

I'm curious about how you are managing labels. I noticed that your label# are ordered from the highest to the lowest. I'm not sure how you are able to do this, unless you're using a stack to manage the label ids.

BlitzScript Label IDs are assigned during load time. I believe your method to be more effecient.


John J.(Posted 2005) [#106]
The label IDs are just calculated with a UniqueVariable() function. The reason the IDs may appear in reverse order some times is my code sometimes needs to pre-calculate a label (so it can jump to the end of a Select..Case structure with the Exit keyword, for example).


Techlord(Posted 2005) [#107]
John,

Hows the VM coming along?


John J.(Posted 2005) [#108]
I haven't done any work on it in a little while. I've been recently working on my game. I usually have a few projects "in-progress" so if I get tired of one, I can move over to another. I seem to work bests in "bursts", switching projects occasionally so I don't get bored.

My latest project is documented in my worklog , if you're interested.

Anyway, when I do get around to it, all I have to do it fill in the Select..Case structure which execures all the opcodes, create a call stack and variable stack, and that's about all.


Techlord(Posted 2005) [#109]
John

I too switch project to project as well. Working on each diligently as long as the motivation allows. Check out my latest project. Sounds like the VM is all ready to rock and roll.

Took a gander at your MegaMesh worklog. This is very much needed. I was actually working on LOD and VIS Rendering Optimizations for Project PLASMA FPS before I took a pause. However, I was taking a precalculated VIS (occlusion) and prerendered LOD approach.


John J.(Posted 2005) [#110]
Over the past week, I've spent some of my time finishing up GameScript. I thought you might be interested in my latest progress.

Currently, GameScript is 98% functional (custom types, function, static vars, local vars, global vars, select case, while, wend, etc. etc.). The entire language works flawlessly. Luckily, I had to make almost no changes to the compiler when getting the VM to work. I simply need to add an assembly code optimizer, and (not so simple) arrays. I also will need to find a way to integrate the scripts with the application better.

When a runtime error occurrs, a runtime error window is displayed, after writing a CrashLog.txt file containing useful information about the crash, including a dis-assembled portion of the code where the error occurred.

Currently, it runs this script perfectly, giving a list of prime numbers:
'Prime number search

Local prime:Int, i:Int

Print "Prime Numbers:"

prime = 1
For i = 1 To 20
  prime = NextPrime(prime)
  Print prime
Next

End



'Prime number search function
Function NextPrime:Int(current:Int)
  Local i:Int
  Local found:Int

  While Not found
    current = current + 1; Found = True
    For i = 2 To current-1
      If (current Mod i) = 0 Then
        Found = False
        Exit
      EndIf
    Next
  Wend

  Return current
EndFunction


My scripting engine currently runs at 3-5 times less than the same code directly compiled in a blitz EXE (both with debug mode off). Because of the structure of the VM (lots of nested Select..Case's because of the multi-variable type), running in debug mode causes scripts to run extremely slow (10 times slower than when debug mode is off). Anyway, I hope my optimizer will be able to increase efficiency and possibly get it down to an average of only 3 times slower.

I'm also thinking about making a debugger program to debug scripts. It would be very possible to create a simple app to allow you to step over/step into code from the original script file (if each instruction code has a line index to use).

Let me know if I can help in any way with your scripting engine.

P.S. When you get to functions, remember that local/global variables makes no difference to the VM. A global variable named "a", for example, is totally different from a local variable in some function named "a", since the compiler should automatically do this. When the compiler comes across "a", it checks what function it's in, and handles the compilation accordingly. Also, for recursion to work, all a functions variables should be "push"ed before calling any function, then "pop"ped after the function call.


Techlord(Posted 2005) [#111]
John,

Your scripting engine is looking real good. It is interesting how you achieved a Scripting Engine with a BlitzMax syntax written with the Blitz Basic language. Your speed results are good, BlitzScript3D runs about the same.

I've have yet to implement loops and logical operators. I've taken a break to work on a game. I will return to it in the near future and will definately seek your help.