Some NG errors I've found

BlitzMax Forums/BlitzMax NG/Some NG errors I've found

GW	(Posted 2015) [#1]

Here are a few that I've encountered

#1 (generates some invalid cpp)

SuperStrict 
Framework brl.retro

Print FloatHex2(123.456)

Function floatHex2$(num#)
	Local fp:Float Ptr = Varptr num
	Local ip:Int Ptr  = Int Ptr fp
	Return "0x" +Hex(ip[0])
End Function

'' this one too
Function FloatHex2:String(num:Float) 
	Local p:Float Ptr = Varptr num
	Local bp:Byte Ptr = Int Ptr Int(p)
	Local out:String = ""
	
	out:+Right(Hex(bp[3]), 2)
	out :+ Right(Hex(bp[2]),2)
	out :+ Right(Hex(bp[1]),2)
	out :+ Right(Hex(bp[0]),2)
	Return "0x" + out
End Function

#2
http://www.blitzbasic.com/codearcs/codearcs.php?code=3162

This is my old 2d lightmapper code. This causes BCC to crash. I tracked it down to the fact that I was using a data label with the same name as a global variable.
If you change the data label from '#map' do something else unique the program works.
I didn't even realize bmax accounted for this and It's surely bad form to do it. but currently it causes BCC to crash.
On another note, When BCC crashes, the compilation process just continues on resulting in Running the older [possibly valid] exe if it exists.

If I find new issues I'll continue to post here in this thread.
Thanks.

Brucey

(Posted 2015) [#2]

Thanks for your input :-)

GW	(Posted 2015) [#3]

Thanks for making Bmax-NG!
I should point out for reference I'm using the Binary release BlitzMax-NGwin32_0_62_3_08 from bmx-ng.com on Win7-64.

Brucey

(Posted 2015) [#4]

Interestingly, this mangling of pointers - "Int Ptr Int(p)" - may eventually lead to a crash on (OS X at least) 64-bit , as you are casting an 8 byte pointer into a 4 byte Int, then casting back into a pointer, which is now a corrupted value, then trying to view data at said location (which may lie outwith the readable memory of your application).

Not an issue on 32-bit, obviously.

Dabhand

(Posted 2015) [#5]

bmx-ng.com on Win7-64.

That will be getting updated tomorrow with Bruceys release... The one on there only changed the android building setup slightly, which I think Mr H has fixed now... So, these little issues will still be on the git version of ng

Dabz

Brucey

(Posted 2015) [#6]

these little issues will still be on the git version of ng

Which issues?

Brucey

(Posted 2015) [#7]

Fixes for #1 and #2 have been pushed to git.

Dabhand

(Posted 2015) [#8]

You too quick! ;)

Sod! :P

Dabz

degac

(Posted 2015) [#9]

As I found this thread about 'NG-errors' I re-post what I found.
Better have one single 'place' where to post.
Win7-64 bit

Cheers

Hi

just tested the latest release 66.03.07 and toying with mojo2 module.
It seems to run everything fine, except for RenderDemo

If I try to uncomment
'canvas.SetViewport( 0,0,GraphicsWidth(),GraphicsHeight())
'canvas.SetProjectionMatrix( Mat4Ortho( 0,640,0,480,-1,1 ) )

or set Graphics 640,480,0

After 2 light's loop I get 'Incomplete Framebuffer' error.

Brucey

(Posted 2015) [#10]

@ Incomplete Framebuffer
There was an issue where it was constantly creating new images/textures on each render - rather than creating one and reusing it. Should be working now. (in latest source from github)

GW	(Posted 2015) [#11]

I've encountered a situation that generates invalid C code.
I've trimmed the code as much as I can to just the relevant part that can be easily duplicated without altering the offending method.

SuperStrict
Framework brl.Basic

Const NONE:Int = 0
Const INP:Int = 100
Const CONS:Int = 101
Const OP:Int = 102
Const OP_A:Int = 113

Type tErrTyp
    Const ITYPE:Int = 1
    Const INP1:Int = 2
    Const INP2:Int = 3
    Field MAXLEN:Int = 64
    Field comp:Float[,]
    
    Method New()
        comp = New Float[MAXLEN, 4]
    End Method
    
    Method MakeSlot(i:Int)
    End Method
    
    Method GetString:String(Terse:Int = False)
        Return "!"
    End Method
    
    Method Malformed()
        Local oldStr:String = Self.getstring()
        Repeat
            For Local i:Int = 0 Until MAXLEN
                If Rand(1, 100) <= 50 Then
                    Select Rand(1, 3)
                        Case 1
                            MakeSlot(i)
                        Default
                            Select True
                                Case comp[i, ITYPE] = INP
                                    comp[i, INP1] = Rand(1, 10)
                                    
                                Case comp[i, ITYPE] = CONS
                                    If Rand(1, 100) <= 50 Then
                                        comp[i, INP1] = Rnd(1, 10)
                                    Else
'/// invalid C generated Here //
                                        comp[i, INP1]:+Rnd(-1, 1)
                                    EndIf
                                Case comp[i, ITYPE] >= OP_A
                                    Select Rand(1, 3)
                                        Case 1
                                            comp[i, ITYPE] = Rand(1, 10)
                                        Case 2
                                            comp[i, INP1] = Rand(0, i - 1)
                                        Case 3
                                            comp[i, INP2] = Rand(0, i - 1)
                                        Default
                                            RuntimeError("!")
                                    End Select
                                Default
                                    RuntimeError("!")
                            End Select
                    End Select
                End If
            Next
        Until (Self.GetString() <> oldStr) And Self.GetString(True).Length > 2
    End Method
End Type

And the C code generated is a duplicate definition:

BBINT* bbt_7=((BBARRAY)o->__untitled_9_terrtyp_comp )->scales + 1;
BBINT* bbt_7=((BBARRAY)o->__untitled_9_terrtyp_comp )->scales + 1;

Thanks

Brucey

(Posted 2015) [#12]

This is fixed in the latest source on github.

GW	(Posted 2015) [#13]

Thanks!

GW	(Posted 2015) [#14]

I frequently encounter this error: ABORT("Too many heap sections")

I don't think I'm doing anything particularly abusive to the GC. It seems to happen regardless of compiler settings (debug,MT,etc) and always in a slightly different place during the runtime.

Up till now, I've been able to blind-ly rewrite the offending functions to bypass the error, but now I'm not able to.

Edit: So far it seems limited to 32bit builds, I'm not yet seeing the error in 64bit builds.

I see it's defined in "blitz.mod\bdwgc\include\private\gc_priv.h"
how would I go about dealing with this? Can I change the # heap sections?

Brucey

(Posted 2015) [#15]

I've no idea. I've never encountered this issue. It seems you are running out of memory.
It appears to have been an issue with Unity too, according to a search on the error text.

blind-ly rewrite the offending functions to bypass the error

Presumably you have an idea what you are doing that is causing the problem?

Although it appears to have been an issue with the legacy code too at some point .

Brucey

(Posted 2015) [#16]

If you have a look at blitz.mod/blitz.bmx :

?win32
ModuleInfo "CC_OPTS: -DGC_THREADS -DPARALLEL_MARK -DATOMIC_UNCOLLECTABLE"

and change that line, adding "-DLARGE_CONFIG", like so :

?win32
ModuleInfo "CC_OPTS: -DGC_THREADS -DPARALLEL_MARK -DATOMIC_UNCOLLECTABLE -DLARGE_CONFIG"

Let me know if that helps, and I can apply it to the "desktop" targets - it is not recommended for the embedded (iOS/Android/Pi) targets as they have more limited memory. If you were considering those targets in the future, you would probably need to redesign your application.

GW	(Posted 2015) [#17]

The number of heap sections is defined in that .h file. The computer has ~10gig of free ram vs ~200meg used by the NG gc.
The number of heap sections seems a bit arbitrary too. (256meg for 32bit?)

When I say that I blindly rewrite the function, here is an example:
Previously I was loading a large csv file by calling LoadString() or LoadText(), then split the file-string into a string array on the Crlf, then process each line.
This technique never given me problems for years. After getting the Heap error deep down in the procesing code, I rewrote the function to use a file pointer and call Readline() until eof() and process the file that way. It eliminates the heap error, but it's an inferior technique and although it bypasses the error, I have no idea why it was aborting in the first place or why my rewrite fixed it.

I suspect that as people start writing/porting larger programs over to NG this will come up more often.

EDIT: Just saw your second reply, I'll try that and see if it helps.
Thanks

Derron

(Posted 2015) [#18]

I assume this is not just a NG-problem but more or less a application design problem.

With your rewrite to use "per line reading from file" you might have saved a lot of RAM.

Imagine you got a normal 2gig apache access.log (kind of csv formatted), say we got 50 Mio hits to ressources this month, this means 50 million lines.

steps:
a) LoadText()
-> loads at least 2gig into RAM

b) For each line:string = EachIn Bla.Split("~n")
-> that "split()" of course creates an 50million sized array + neglectable overhead of being an array (some extra fields)
-> we are now already occupying 2*2gig of RAM
-> iterates 50 million times (dunno when GC is called then ...)
-> (maybe a manual GC trigger every X lines might help then ?)

c) maybe you do an other line.split(",") within that for loop: this _again_ uses some RAM (depends on when the GC kicks in)

The problem might be, that you
a) are loading a big file "as one" into the app
b) process the big file "as one" into your app
c) do not let the GC kick in some how (dunno how it works, if it gets triggered by a "max amount of trash" or "time" or "...")

The common sense to handle big files is how you do it now: on a "per chunk" base.

In a line-based format you could read "by line", XML/HTML/...-files need a different handling then (nonetheless you can still read the information on a "by chunk"-base).
In some cases you trade "RAM" (having all information read _once_) with CPU-time (analyzing the document first - check if "opener" equal to "closer" tags etc.).

Question:
What sizes do your CSV have?

bye
Ron

Brucey

(Posted 2015) [#19]

Indeed, I've done some text file processing at work in BlitzMax (NG) with files > 4TB.
I ended up creating a buffered stream reader, and had it pushing the files through the parser in seconds.

Of course, it all depends what you want to do with the data, that you might end up requiring to have a lot of it in RAM at the same time.

GW	(Posted 2016) [#20]

I tried updating my NG build today and came across this error when trying to rebuild the LuaJit module.

"Compile Error: Extern Types can only contain methods.
...luajit.mod/luajit.bmx;754;0]

I hope it doesn't invalidate using the luajit mod going forward as I use it in a few projects with NG.

Brucey

(Posted 2016) [#21]

Just an oversight. Sorry.
I've updated the repository.

GW	(Posted 2016) [#22]

Awesome!
Thanks!

GW	(Posted 2016) [#23]

I tried updating my NG setup today and started encountering some errors building modules related type conversions.
Does NG no longer support automatically converting primitive types?

Compile Error: Unable to find overload for mix_volume(Int,Float). Argument #2 is
 "Float" but declaration is "Int".
[C:/Dev/Lang/BlitzMaxNG/mod/sdl.mod/sdlmixeraudio.mod/sdlmixeraudio.bmx;324;0]

LT	(Posted 2016) [#24]

I've recently encountered this, as well. NG now has method overloading and this means that implicit conversions are no longer applied.

Derron

(Posted 2016) [#25]

Use ~~bmo~~ bmk -w when compiling ... and fix the listed warnings to make it compile without "-w".

(Edit: fixed bmo-typo)

Bye
Ron

GW	(Posted 2016) [#26]

what is 'bmo -w' ?

NG now has method overloading and this means that implicit conversions are no longer applied.

I really hope this isn't the case because that would be a terrible idea.

LT	(Posted 2016) [#27]

Think that was a typo.. it's bmk -w.

Derron

(Posted 2016) [#28]

Yeah, was a typo ... typed it on my smartphone with a very small-scaled textarea while having 2 of my eyes concentrated on watching my son play :-).

Excuse me.

"bmk -w"
The w-param was added to Brucey's BMK some days ago, so make sure you use the most current BMK revision (built from code).

As said BCC-NG (with overloading) defaults to _no_longer_ convert implicitely (I think this should be configurable via a bcc.conf-base-configuration to set "defaults", same for "debug").
On behalf of my ~~begging~~ request Brucey changed the "errrors" to "warnings" (aka adding "-w"). Regardless of this I would more likely see "-w" to turn into something useful to analyze your code (like "lint", checking for obvious mistakes, unused variables ...) but this might be better done in an extra tool (similar to the mentioned "lint").

There is also a switch ("-s" I believe) to adjust the strict-settings (extending modules using "nothing"-return with functions returning "int" - which works in vanilla, but fails in BCC-NG because of the existance of "void").

bye
Ron

GW	(Posted 2016) [#29]

well that's disappointing. Half the mods won't even compile now, even with the -w switch. I guess I'm stuck to the older version.
Thanks for the help though.

Derron

(Posted 2016) [#30]

You might note here, which modules are failing now.

bye
Ron

GW	(Posted 2016) [#31]

Maxgui and pub.directx is as far as i got. basically 200 lines of this:

In file included from C:/Dev/Lang/BlitzMaxNG/mod/maxgui.mod/win32maxguiex.mod/.b
mx/tom.bmx.release.win32.x86.c:1:0:
C:/Dev/Lang/BlitzMaxNG/mod/maxgui.mod/win32maxguiex.mod/.bmx/tom.bmx.release.win
32.x86.h:57:2: error: unknown type name 'pub_win32_com_IUnknown_QueryInterface_m
'
  pub_win32_com_IUnknown_QueryInterface_m m_QueryInterface_pbppb;
  ^

LT	(Posted 2016) [#32]

There are NG versions of pub, brl, and maxgui that compile. They are available at github.com/bmx-ng...

NOTE: The maxgui module compiles with -w, but gives a lot of warnings.

Brucey

(Posted 2016) [#33]

The maxgui module compiles with -w, but gives a lot of warnings

The latest version of maxgui should clean-build now.

LT	(Posted 2016) [#34]

should clean-build now

Just tried it, but I'm still getting two related warnings:

win32maxguiex.bmx;1999 : AdjustWindowRectEx - Argument #3 is "Int" but declaration is "Byte Ptr"

Same thing, line 1643.

GW	(Posted 2016) [#35]

I grabbed everything from the NG github that's newer than Feb for rebuilding.

If primitive type conversions really are gone, that would explain why all the mods fail to compile.
In my case, I'll have to stick to my old NG build going forward. compatibility with vanilla bmax is feature I need.

Brucey

(Posted 2016) [#36]

If primitive type conversions really are gone, that would explain why all the mods fail to compile.

To support overloading, the compiler needs rules to understand how types can be converted to other types. These "widening" rules allow smaller types to convert to larger types, but not the other way around. So an Int can widen to a Long, but a Long can't widen to an Int. This way you can write the following :


Method overloaded(arg:Int)
...

Method overloaded(arg:Byte)

...

Local s:Short = 10
call.overloaded(s)

The compiler uses these rules to choose the correct overloaded method, in this case overloaded(arg:Int).

Since older code did not require such rules, the compiler knew that there was only one method of a given name, and could convert types as needed.

The -w option simply raises warnings for cases where a widen rule fails, and assumes that the method you want to cal is the one that it found (since it has the same name). It will then convert the types as previously.
The build should complete with generated warnings.

If you get actual errors, then there may be other issues that I am unaware of.

Bobysait

(Posted 2016) [#37]

First, sorry for reviving a 5 monthes old topic.

So, to prevent overloading errors (double specified while a float is expected and so on), can we use long and double for any int/short/byte and float typedef in functions and methods arguments ?

(my engine is about 3500 functions and methods ... and almost all is using Float, Int and Byte (as boolean))
And ... would it lower performances on function calls ? (8 bytes to send instead of 4, I know it's obvious, but is it a drastic loss or almost insignificant ?)

ps : If it's not implemented yet, what I'd really like to see in NG is
the "Condition ? Value_If_True : Value_If_False" syntax

This syntax is commun and very usefull

(for example)

' pX point to the vector from a,b,c which holds the highest X
Local pX:Vector = a.X>b.X ? (a.X>c.X ? a : c) : b.X>c.X ? b : c
Local pY:Vector = a.Y>b.Y ? (a.Y>c.Y ? a : c) : b.Y>c.Y ? b : c
Local pZ:Vector = a.Z>b.Z ? (a.Z>c.Z ? a : c) : b.Z>c.Z ? b : c

Derron

(Posted 2016) [#38]

Ternary operations (x ? y : z) are not backwards compatible.

Even without "overloading" you should be able to to code it on your own

Function TernaryInt:int(condition:int, b:int, c:int)
  if condition then return b
  return c
End Function

Function TernaryString:int(condition:int, b:string, c:string)
  if condition then return b
  return c
End Function

...

To prevent overloading errors - use the param to convert them to "warnings".

I converted my code "step by step" to make it truly compatible to NG.

bye
Ron

Bobysait

(Posted 2016) [#39]

Even without "overloading" you should be able to to code it on your own

Yep, sure, I already did it, but it would have been nicer with the original semantic

the param to convert to warnings is "-w", isn't it ?

Derron

(Posted 2016) [#40]

	-w
		Warn about function argument casting issues rather than error. (NG only)
		With this warning enabled you may have issues with method overloading.

yes.

bye
Ron

Bobysait

(Posted 2016) [#41]

Thanks, and, I feel like I should apologize, I just add to run bmk in the console to show the parameters -_-, so ...

Sorry for loosing your time with trivial stuff.

Derron

(Posted 2016) [#42]

Also think of of making things like "void" and "int" function results consistent.

In my code I often extended types and their methods:

base: method meth:int() ..
extended: method meth() ...

leading to warnings (or errors :-))

bye
Ron