BCC NG compiler

BlitzMax Forums/BlitzMax NG/BCC NG compiler

col(Posted 2015) [#1]

Had a quick play with this for the last hour, I'm liking it. Really cool so far!

I loaded up an emulator that I'd written to build with your new version, you know... just to see :-)

First up... a little snag with the If statement...

Function testme()

If Not testme() End

responds with 'Endif' without matching 'If', whereas this works...


Function testme()

If Not testme()

Seems quite straight forward as to whats happening there...

In my parser this one baked my noodle at first. I handled the parsing quite differently. I notice that you have a 'am i in a nested If statement' parameter in the parseif method. I used a different approach to handling nested ifs in that I created an ElseIf statement and the IfStmt would have an array of ElseIfStmt, instead of trying to keep everything within the same 'If' statement. This made it really easy by comparison. Just a thought.

Before pummelling you with other bugs and code suggestions... Are you looking to get 100% compatibility with existing code bases? how up to date is the github repo compared to your latest local? And also, how dirty can we get our hands with offering any help? As you may have noticed I lean on the windows environments so if anything needs to be done there...

bcc-ng is cool. I like it... a lot :)

Derron(Posted 2015) [#2]
My current test case for "ifthen":

... is still compiling perfectly, so it will be the "end"-handling.

So I opened up our test scenario "end_01.bmx" which contains your setup already:

If 1=0 End

... and it fails checks - having the same error as you got.

Somehow this means it is a regression bug: worked once but got bugged during changes. Will check which one made it fail and edit that into this post afterwards.

This commit breaks the end-handling:

Think the problem is, that brucey added a check on "endif" and "end" without checking for the next token to be "newline" (singleline if) ... I am not sure but it might be Bruceys fault as he came up with his own rewrite of that portion (we simultaneously rewrote that if-handling portion). ... but maybe it is my fault. I prefer to be innocent until caught ;-).

EDIT2: I created a patch for this bug, feel free to test my commit for other problems.

For now some of our tests fail but I will mail Brucey about that separately (list, inherited function, freetype ...) because I do not know what of them are failing because of their specific test structure (maybe module interna changed).


col(Posted 2015) [#3]
Ahh, the old regression bugs, don't you just love them. I do remember getting in a spin when parsing if statements, so I understand how easy it is to break this one. I think I rewrote it like a dozen times before I got it right.

Narrowed another bizarre bug down to...
Type Mode
	Method Create:Mode()

Building untitled1
Compile Error: Syntax error - expecting '('.

Rather strange is that it reports linking and executing, obviously it isnt.
The problem seems to be with returning a typename that begins with 'Mod' as changing the return type and everything carries on as expected, including catching errors etc.

Derron(Posted 2015) [#4]
Compilation: did you compile the "untitled1" before? In that case the "c"-code (.bmx-directory) exist and the linking is done - instead of just quitting out.

Normally things are spit into tokens. So it is not doing a "nextchar = m, nextchar = o, ..." to check for "mod", the token is "mode" or maybe "mode()" and then split even more.

I will see if I can find a reason for this - else it is up to you to walk a bit in the dust of the bcc code - or leave it up to brucey :p


col(Posted 2015) [#5]
ref compilation - Yep, I see, I understand whats happeneing there.
ref 'mode' - Yep again, I understand about lexical analysis, tokenisers, parsing etc.

As to what's going on... I don't mind putting on my walking boots :-)

Derron(Posted 2015) [#6]
it happens for all other registered keywords too..

so a Type "SarFuBar" would be problematic too ("sar") .. tokenizer might need to distinguish between non-alphanumeric symbols (":+") and ":sar | :shl | ..." the last ones should be handled as "inbuild commands" ... or something in the likes.

Added a commit changing tokenization of alphanumeric symbols (:shl, :shr, ..).

local a:int = 10

still does not compile while being valid in vanilla BlitzMax.

For the example above: seems the ":mod" gets interpreted differently with the ".." concencated without space ("a:mod .. VS a:mod.."). "ParsePrimaryExpr" is run for this line and somehow tokeType "TOKE_IDENT" is used.

I am quite sure Brucey will find the correct spot to lay on his fingers.


col(Posted 2015) [#7]
On a different note.
I see a considerable difference in compilation times between the 2 compilers. I understand the two fold process of producing c and compiling that too but I'm talking a huge difference. For eg the vanilla compiler will build bcc (ng) in a fraction of a second, but the ng compiler takes well over 10secs. Is this a known issue and one for 'ron ( later on )?

Different note again :-)
I want to help debug the compiler, what method are you using to do that? I keep getting a mysterious 'debugger error:scope kind unknown' or something similar just before returning from ParseArgs.


ref what youre saying with :sar token as a single token etc. i would have expected to find those 'self modifying' operators to be 'constructed' in the parser as opposed to the lexer no? i guess it doesnt matter so much as long as they are recognised correctly.

Derron(Posted 2015) [#8]
@compilation times

I do not see that long compilation times... it is similar to what I have with "vanilla" - and both utilize "caches" so recompilation is only one when forced.

with MaxIDE you have the menu entry: "Program -> Command Line".
If you enter something there, it is added to the call of the compiled programme.

-g x86 -v -r -f brl.standardio -p linux -o "/home/ronny/Arbeit/Programmieren/Projekte/Apps/Dig64/external/libxml/.bmx/libxml.s" "/home/ronny/Arbeit/Programmieren/Projekte/Apps/Dig64/external/libxml/libxml.bmx"

- when you now open bcc.bmx and compile+execute it (debug mode) it will try to compile the above URI and on errors you can use the debugger of MaxIDE.

I did not want to rewrite things Brucey did (or reused in the case of Mark Sibly written code) ... else there are surely some things which would benefit from getting a rewrite/overhaul.


Brucey(Posted 2015) [#9]
It probably depends what you are compiling, and the size of a "single" .bmx (after includes) source file.

It may also depend on your C compiler/system setup. For example, C-compiling the main digesteroids .bmx file takes a long time on the raspberry pi, but the other files are pretty quick.
The initial bmx -> C generation of the file was very quick, on the other hand.

Compiling all the modules with ng, I see very little noticeable difference. It feels quite nippy actually, considering the amount of work going on.

The final runtime binaries it creates, depending on what's in the code, can be faster than Mark's generic hand-coded ASM. For example, I found TMap to work more quickly in an NG binary than legacy bcc here on OS X (32-bit, of course - since legacy bcc doesn't do 64-bit at all ;-).

Brucey(Posted 2015) [#10]
I want to help debug the compiler, what method are you using to do that? I keep getting a mysterious 'debugger error:scope kind unknown' or something similar just before returning from ParseArgs.

Ho ho ho :-)

BlitzMax expects bcc to be located in the correct location (BlitzMax/bin) for everything to build properly. But obviously, when you are running a debug version of it from a different location, that rule fails.

You can work around this by adding a bcc.conf file to the folder where your bcc debug stuff lives.
In that file, you have a line like this :

which NG bcc will read and set when it runs - this overrides the one that is set by your debugging session.

(in this case, BlitzMax_NG is a BlitzMax area just for the NG stuff, with all the appropriate modules and so on that the new compiler requires)

col(Posted 2015) [#11]
Need an idiots guide here :P

In the BlitzMax folder, everything would be the standard files except...

BlitzMax/bin/bcc.exe (ng version)
BlitzMax/bin/bmk.exe (ng version)
BlitzMax/mod/ all ng modules

BlitzMax_NG/mod/ ( all the ng modules )

Is this correct?

Derron(Posted 2015) [#12]
You will have to use bmk ng too. Bmk-ng contains a readme about additional files (make.bmk core.bmk...) which you need too.

But else ...yes...you do not need much.

To compile you need a working mingw setup...or you copy it into a specific directory within the blitzmaxng dir...mingw32 or so.

Bcc.conf is needed for adjustments regarding the paths ...


Brucey(Posted 2015) [#13]
In the BlitzMax folder, everything would be the standard files

It depends what you are trying to accomplish.

My BlitzMax folder has all original stuff in it - original bcc, bmk, mods, MaxIDE etc.
My other BlitzMax area is for working with NG, so everything there is NG specific.

My bcc source happens to be somewhere else. I debug it using the original compiler/MaxIDE. The bcc.conf file points to the NG BlitzMax area.

When I am not directly testing bcc itself, I drop a release version of bcc into my NG BlitzMax/bin folder, and use another MaxIDE from there - everything I am building/testing with this MaxIDE uses all the new stuff. (for example, cross-compiling to Android, etc)

I generally only debug bcc when there is an issue to fix, otherwise I'm using the release build of it.

If you create a MinGW32 folder inside your BlitzMax dir, the new bmk/bcc will use that, rather than rely on files in bin/lib. (The latest 64-bit TDM MinGW appears to work well - the binaries are 32-bit so will work fine on a 32-bit system, although allow you to build 64-bit binaries)

I need to make a wiki area on github for bcc to document all of this really.

Derron(Posted 2015) [#14]
I also would suggest we create a first "release" (some alpha stuff).

-> create a branch (eg. "v0.29")
this branch is there to connect a "relase" to a specific revision state

-> create a new "release" and link to the new branch
this release may contain some release notes - but the most important: you can drop in a zip file (or multiple).

This zip file could contain:
- a snapshot of bcc / bccMac / bcc.exe + bmk / bmkMac / bmk.exe
- snapshots of the needed conf/bmk
- snapshots of brl.mod-ng and bmk.mod-ng (NOT precompiled - so a clean checkout)
- in the case of including windows, a whole whopping MinGW32-folder (or a MinGW32-folder containing a .bat which downloads/unzips the MinGW for you ... or just a readme)

This would be then package to start with ...


col(Posted 2015) [#15]
Excellent, thanks guys, I'm sure to get the ball rolling now...

col(Posted 2015) [#16]

I have narrowed down another one for you...


Function TestFunc(Str:String Var)
	Local valA:Int = 10
	Local valB:Int = 20
	str = Float(valA) + Float(valB)

Local Test:String
Print Test

Building untitled1
C:/BlitzMax_NG/tmp/.bmx/untitled1.bmx.console.release.win32.x86.c: In function '_untitled1_TestFunc':
C:/BlitzMax_NG/tmp/.bmx/untitled1.bmx.console.release.win32.x86.c:5:10: error: incompatible types when assigning to type 'BBSTRING' from type 'float'
Build Error: failed to compile C:/BlitzMax_NG/tmp/.bmx/untitled1.bmx.console.release.win32.x86.c
Process complete

Derron(Posted 2015) [#17]
If you have a github account I would ask you to open an issue for this on the bcc-project-page.
So Brucey can properly close the bug - and it does not get forgotten.

I assume Brucey will be able to fix that bug ... he just have to know about it.


col(Posted 2015) [#18]
Sure, will do.