Operator Overloading

BlitzMax Forums/Brucey's Modules/Operator Overloading

Brucey(Posted 2016) [#1]
Hallo, there's a new feature for testing if anyone would like to give it a try. It adds support for operator overloading.

Operator overloading adds what is known as syntactic sugar to your code, hiding method calls behind standard operators.
Supported operators include
* / + - & | ~ :* :/ :+ :- :& :| :~ < > <= >= = <> mod shl shr :mod :shl :shr


The syntax is in the form of a method name with the prefix Operator, and the suffix of the operator you want to overload.
Return types for conditional operators are usually Int, for assignment-style operators (like :+ , :* etc) there is no return type, and for the rest, your user-defined Type.

Here are a couple of examples.
For assignment addition ( :+ ) where you are adding a float to your user type,
	Method Operator:+(f:Float)
		x :+ f
		y :+ f
	End Method

For assignment addition ( :+ ) where you are adding a another object to your user type,
	Method Operator:+(v:Vec2)
		x :+ v.x
		y :+ v.y
	End Method

For adding ( + ) two objects together.
	Method Operator+:Vec2(v:Vec2)
		Local n:Vec2
		n.x = x + v.x
		n.y = y + v.y
		Return n
	End Method


And so on.

Here's a working example :
SuperStrict

Framework brl.standardio

Local v1:Vec2 = New Vec2(10, 10)
Local v2:Vec2 = New Vec2(15, 20)

Local v3:Vec2 = v1 + v2     ' addition of two vectors
Local v4:Vec2 = v1 + 150.0  ' addition of vector and float

v3 :+ 100.0                 ' addition of float to vector

Print v1.ToString()
Print v2.ToString()
Print v3.ToString()
Print v4.ToString()

Struct Vec2

	Field x:Float
	Field y:Float

	Method New(x:Float, y:Float)
		Self.x = x
		Self.y = y
	End Method

	Method Operator+:Vec2(v:Vec2)
		Local n:Vec2
		n.x = x + v.x
		n.y = y + v.y
		Return n
	End Method
	
	Method Operator+:Vec2(f:Float)
		Local n:Vec2
		n.x = x + f
		n.y = y + f
		Return n
	End Method

	Method Operator:+(f:Float)
		x :+ f
		y :+ f
	End Method
	
	Method ToString:String()
		Return "Vec2(" + x + ", " + y + ")"
	End Method
	
End Struct

Resulting in the output
Vec2(10.0000000, 10.0000000)
Vec2(15.0000000, 20.0000000)
Vec2(125.000000, 130.000000)
Vec2(160.000000, 160.000000)

In this example, it is using Structs, but the same applies to standard Types. (Structs are stack-based, non-garbage managed/collected objects).

Of course, you can always write in equivalent using methods :
Local v3:Vec2 = v1.Add(v2)

But you may prefer an arguably cleaner syntax of :
Local v3:Vec2 = v1 + v2



You don't have to use it, but it is available for those that want to... and choice is good, as they say.

Anyhoo, all feedback is welcome, as usual :-)


col(Posted 2016) [#2]
I've not tried it yet, but this is an awesome feature to have!


BlitzSupport(Posted 2016) [#3]
Yeah, I only recently grasped the point of this feature from learning C++!

Are structs brand-new in bmx-ng? (Out of curiosity, have they come from Monkey2, which now features structs?)

How do you free them? I've tried a couple of ways to no avail:


Struct Test

	Field x:Int
	
	Method Add:Int (a:Int)
		x = x + a
	End Method
	
End Struct

Local t:Test = New Test

t.x = 1
t.Add 1

Print t.x

' ************** OK, LET'S DELETE! **************

t = Null		' Nope! error: incompatible types when assigning to type 'struct _m_untitled1_Test' from type 'BBObject * {aka struct BBObject *}'

't.Delete		' Nope! undefined reference to `_bbObjectDtor'

End


Just been looking through maxide-ng to try and find how to add Struct as a keyword, but literally no clue found! Looks like it's defined in bcc, but how do you get the IDE to pick it up and highlight, capitalise, etc?


Derron(Posted 2016) [#4]
as it is no module defined function name you surely have to edit MaxIDE to support "struct".


@ freeing a struct
Hmm, maybe it is just not implemented because nobody made Brucey aware of the missing functionality ;-)


bye
Ron


Bobysait(Posted 2016) [#5]
Hey, that's the kind of feature (almost the last one for my concern) that was really missing to blitzmax to make it really confortable !

So, it would probably just miss the "property" style syntax now and I think BlitzMax would be complete (for the OOP part)

I hope BlitzMaxNG will fully replace the vanilla version some days, so I don't need to make portability between the two versions and fully use the NG features for myself :p
But this won't happen until it becomes easier to install and compile.
Maybe there is still a lot of work in that sense
Like a simple installer that will get the stuff and batch install everything without the needs to set variables, compile stuff manually etc ...
(it's not that hard, but it's not very convenient)

Anyway, it's already a very good step, so thank you for making support for overloading !

[edit]
Damned ! searching the forum an "How to" install the whole thing, I 've found a topic which link to an old "last version" on github.
So, at this moment I said myself "hey ! that's pretty old, but it surely means there is binaries released ?! there must be a category to look for updated version ?"
And I click on the bmx-ng section to go to the parent section ... and nothing ... not a damned binary, just the source again. but there Must be something somewhere ... else I wouldn't see the previous page ...

And it just comes to my eyes after 5 minutes looking every chars on the page



(for me, the highlighted in orange section was actually not a section ... it just looks like a label -> some kind of general overview "There is 9 releases" , it doesn't look like something "clickable" so my brain skip it as fast as my eyes go on it)

Maybe I'm a bit stupid, I don't know ... But for me, this is all but explicit.
So, maybe there should be a sticky post on the forum to mention it for other idiots like me :)


BlitzSupport(Posted 2016) [#6]
I've downloaded and built bcc_op, replaced my bin\ copy of bcc, but this doesn't work here:



Says "Compile Error: Operator + cannot be used with Objects." Do I need to update anything else?

One thing I don't get from the original example is why the test declares two "New Vec2"s, while the operator just declares "Local n:Vec2". How come the operator doesn't need to 'New' it?


BlitzSupport(Posted 2016) [#7]
Hmm, the example works. Just tried converting it to use Type, and with New Vec2 added to the methods, it works. Will have to compare...


BlitzSupport(Posted 2016) [#8]
Ah, I did a stoopid:




BlitzSupport(Posted 2016) [#9]
This is pretty cool! 3D demo using CSG add/subtract operations with operator overloading:



This is with the following added at the end of mod\openb3d.mod\openb3dex.mod\inc\TMesh.bmx:



32-bit Windows demo at http://www.hi-toro.com/blitz/temp/csgoo.zip

(Will rotate for two seconds after adding/subtracting the cubes, then reset to the separate (overlapping) cubes.)


col(Posted 2016) [#10]
Just for completion...

Structs in BMX-NG are allocated on the stack. So say you create an instance of a struct that's a 'local' in a function then when the function goes of out scope the struct instance will also go out scope and that stack allocated struct will also be gone. Therefore you don't need to 'free' it and you also don't need to tell Brucey that he's missed anything :p

Stack allocations are much faster than dynamic memory management as its simply a case of updating a cpu register ( the stack pointer ) to accommodate for more needed 'local' memory. Dynamic memory management requires hitting up the GC for an allocation which is extremely expensive by comparison - to the point of being incomparable.

Looks like a good demo there James!


Bobysait(Posted 2016) [#11]
I wouldn't have think about CSG functions using this, but it's a good idea :)

You could probably add this too
	Method Operator&:TMesh (add_mesh:TMesh)
		Return MeshCSG (Self, add_mesh, CSG_INTERSECT)
	End Method

Intersect acts just like a "&" in theory (the part of the 2 meshes that exists for both)


BlitzSupport(Posted 2016) [#12]

Structs in BMX-NG are allocated on the stack. So say you use a struct in a function then when the function goes of out scope the struct instance will also go out scope and that stack allocated struct will also be gone


Oh, I see... I think! How does that relate to the New versus non-New calls, though?


Intersect acts just like a "&" in theory (the part of the 2 meshes that exists for both)



Oh, good call -- I'll add that!

EDIT: Added now -- little Win32 demo with source that shows it:

http://www.hi-toro.com/blitz/temp/csgoo2.zip


Brucey(Posted 2016) [#13]
Hallo :-)

@Structs
How does that relate to the New versus non-New calls, though?

Using New with Structs is optional...
Struct MyStruct
  Field x:Int
  Field y:Int
End Struct

Local s1:MyStruct
Local s2:MyStruct = New MyStruct

s1 and s2 are initialised in the same way.
You can always just use New as you do with Types, and not be concerned about it being optional ;-)

A Struct instance is not a pointer to an object - like you have with an instance of a Type. It is simply some space allocated from the stack. Since they are not tracked by the GC, they are quick to allocate, and don't need to be freed. They are useful for small data - like a vector - but not so much for larger data, because the stack is limited.
When you pass a Struct into a method/function it is passed by value (i.e. the content is copied onto a new part of the stack). When you pass a Type, you pass by reference.
Because they are not part of GC, you shouldn't have fields of Types/Arrays/Strings on them - unless you know what you are doing.
If you are not sure, just use Types :-)

Another Struct example
Struct MyStruct
  Field x:Int
  Field y:Int

  Method New(x:Int, y:Int)
    Self.x = x
    Self.y = y
  End Method

  Method New(s:MyStruct)
    x = s.x
    y = s.y
  End Method
End Struct

Local s1:MyStruct = New MyStruct(10, 10)
Local s2:MyStruct = New MyStruct(s1)



BlitzSupport(Posted 2016) [#14]
Thanks, Brucey. I suppose I'm sort of aware you can't really free stack memory within a function, though it's only recently that I've got a proper handle on what stack memory is, so I guess it makes sense that it's only released on exit from a function! [EDIT: Or is it? Isn't it just something that's globally allocated per-function when the program starts, based on the declared variables within? Does it actually get freed at all?]

I was kind of thinking it's like manually-allocated memory, but of course it's not.


If you are not sure, just use Types :-)


I might just do that, though I understand Structs are much quicker since the stack memory is already allocated at startup (I suppose), just like with basic local variables.

Anyway, glad to have more of a clue about operator overloading now, as it's something I always thought was pointless and confusing, misunderstanding that it was for changing how add/subtract and co worked on basic ints, floats and so on!


Derron(Posted 2016) [#15]
@ Free
Sorry for the wrong assumption of a "might have been missed" situation. We (James and me) just exposed some missing knowledge.
Good to have you on board to explain that subject to us.


@ Overloading
Whats the real benefit of having it - compared to Methods like "myVec2.AddVec2(otherVec2)" ?

In both cases "myVec2" must be aware of how to handle the incoming "other thing" - similar to the operator overloading.


@ Overloading add/subtract int floats ...
As much as I remember what Brucey explained: you can only overload operators for types in this NG-implementation. So you need "Methods-capability". Int/Float/String do not allow for such things - and therefor are not available for operator overloading (in the sence of "a:int + b:int").



bye
Ron


BlitzSupport(Posted 2016) [#16]
@Derron: yeah, I always misunderstood operator overloading to mean you could change how basic type addition, etc, worked, hence didn't get the point; I get that it's not actually meant for that now. (Does C++ even allow it?)

As for the benefits, I guess it mainly just allows for a snappier syntax, without the need to remember/look up methods, as in "obj1 = obj2 + obj3".


seriouslee(Posted 2016) [#17]
Saw this post so I downloaded the branched bcc and gave it a try. Love it, Brucey!!!! Are you planning to merge this into the master branch?


Derron(Posted 2016) [#18]
It is already merged into master for some days now.


bye
Ron


seriouslee(Posted 2016) [#19]
Well... I thought I had tried it but I must have not formatted the Operator part correctly. Duh. Glad it is! Expressions are clearer in my opinion with them for the most part.


col(Posted 2016) [#20]
Thanks for implementing this feature!!!

I'm really using it a lot :P


Brucey(Posted 2016) [#21]
I'm glad it appears to work ;-)

I really want to add support for generics, but I'm still trying to get my head around it...


FireballStarfish(Posted 2016) [#22]
For some reason, it doesn't work for me.
If I try to run the example from post #1, I get this:
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c: In function '__m_untitled3_Vec2__add_TVec2_TVec2':
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c:68:11: error: conversion to non-scalar type requested
  ((struct _m_untitled3_Vec2)bbNullObjectTest(o));
           ^
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c: In function '__m_untitled3_Vec2__add_TVec2_f':
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c:110:11: error: conversion to non-scalar type requested
  ((struct _m_untitled3_Vec2)bbNullObjectTest(o));
           ^
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c: In function '__m_untitled3_Vec2__addeq_v_f':
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c:152:11: error: conversion to non-scalar type requested
  ((struct _m_untitled3_Vec2)bbNullObjectTest(o));
           ^
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c: In function '__m_untitled3_Vec2_ToString':
C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c:182:11: error: conversion to non-scalar type requested
  ((struct _m_untitled3_Vec2)bbNullObjectTest(o));
           ^
Build Error: failed to compile C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled3.bmx.console.debug.win32.x64.c


This seems to be related the struct rather than the operators though, because if I change the Struct into a Type it works fine, and because I am also having issues with this simple test:
SuperStrict
Framework BRL.Blitz

Local s:STest = New STest("a")

Struct STest

	Field s:String
	
	Method New(s:String)
		Self.s = s
	End Method
	
End Struct

C:/Programmierung/BlitzMax-NG/tmp/.bmx/untitled1.bmx.console.debug.win32.x64.c:91:47: error: '_m_untitled1_STest' undeclared (first use in this function)
   bbt_s2=__m_untitled1_STest_New_S_ObjectNew(&_m_untitled1_STest,&_s0);
                                               ^

I did update to the newest versions of bcc and BRL.Mod; anything else I need to do?


Brucey(Posted 2016) [#23]
Those two issues should be fixed now.


TomToad(Posted 2016) [#24]
This is absolutely cool :)

Edit: cause it is even cooler


Derron(Posted 2016) [#25]
	
	Method operator:+(i:Int)
		t :+ i
	End Method
	
	Method Operator+:TTest(i:Int)
		Return New TTest(t+i)
	End Method


How does the compiler know which of both you are calling (param types need to differ - at least this is what I thought).

Bye
Ron


TomToad(Posted 2016) [#26]
The first is Operator followed by :+ . Note the : before the +. this is called whenever you do Type :+ Type.

The second is Operator followed by +, then a return type with :TTest. Note the : is after the +. It is not part of the operator, but separating the operator from the return type. This will be called whenever you do Type = Type + Type.

So basically, you have two different functions, Operator:+ and Operator+. Both take an Int as a parameter, the second returns a TTest type.


Derron(Posted 2016) [#27]
I should not write here with 1 eye open ... just saw "operator +" and "operator + ttest".


BTW: so
> [...] This will be called whenever you do Type = Type + Type.
is a bit incorrect and should be
Type = Type + 1


Why is that ":+" even needed or allowed to get overridden? For me ":+" should always be threatened as shortcut to "a = a + x". So in our example the second override should be:

Method Operator+:TTest(i:int)

and should override
test = test + 1
and
test :+ 1

because even the second one is "implicitely" returning "TTest".


What do I miss / misunderstand?


bye
Ron


TomToad(Posted 2016) [#28]
Method Operator+TTest(i:int) doesn;t consider the LValue (the identifier on the left side of the = sign). It calls the operator method of the right test, adds i then returns test. You need 2 different versions because you might not want to return a TTest as a result. For example
Type TTest
	Field t:Int
	
	Method New(i:Int)
		Self.t = i
	End Method
	
	Method Operator+:Int(i:Int)
		Return t+i
	End Method
End Type

Local test:TTest = New TTest(10)
Local j:Int = test + 10
Print test.t
Print j


Notice in this example, I overridden + for TTest, but the result returned is an int. So doing test :+ 1 would not work as it would expect a TTest as a result.


col(Posted 2016) [#29]
What do I miss / misunderstand?

You can set up the 'operator overloading' to return what ever you want it to return. Its up to you make sure the returned value is of the expected type - you are overriding default behavior so clarity in your code is up to you to maintain.

Personally, I'd keep the returned Type from any method the same as the Type that the method is within. Doing anything else is just asking for some serious brain hurt further down the line, however, as I say, it's up to you, I'm sure there are cases where it would sense to change the returned type but I can't think of any right now.

Strict

Type TTest
	Field t:Int
	
	Method New(i:Int)
		Self.t = i
	End Method
	
	Method Operator+:TTest(i:Int)
		t :+ i
		Return Self
	End Method
End Type

Local test:TTest = New TTest(10)
Local j:TTest = test + 10
Print j.t



TomToad(Posted 2016) [#30]
Matrix multiplication often results in matrices of different sizes. For example, a 1x3 matrix multiplied by a 3x1 matrix will result in a 1x1 matrix, and a 3x1 multiplied by a 1x3 will result in a 3x3 matrix. Such as the example below


In this case, a Matrix1x3 :* Matrix3x1 would not work as the result is not a TMatrix1x3. Since a TMatrix3x3 * TMatrix3x3 results in a TMatrix3x3, I can define a :* for it.

You could argue that you should make a general purpose TMatrix type which can be defined with any dimensions, but that might be unnecessary overkill when only dealing with a couple of matrix sizes. However, it is nice to have both options available to you so that you can choose whatever is clearest and works best for you.


Derron(Posted 2016) [#31]
Thanks for the clarification.

And yes - for matrices/vectors you are right (vec2+vec3 = vec3). This did not came into my mind. But I am with "col" that I am used to "int + int = int" and "string + int = string" so until it is only used for "internal/encapsulated things" (kept private in a type) I think it could add a lot of convenience.


bye
Ron


TomToad(Posted 2016) [#32]
having more fun. Chemistry simulation shows what you get when mixing different elements with oxygen. (just an example, probably not scientifically accurate)



FireballStarfish(Posted 2016) [#33]
Is, or will there be, a way to overload operators on builtin/primitive types, or on already existing types in general?
I am mostly asking because this would be required to make any operators involving them act commutatively. For instance with
Type TTest
	Method Operator+:Int(i:Int)
		' ...
	End Method
End Type
I can define the result of "New TTest() + 1", but what about "1 + New TTest()"?


Derron(Posted 2016) [#34]
I doubt that, as this would mean to "lazy load" how things are added.

For now you know on compilation: "int + int = int", if you now have a source file which overrides this behaviour, this would only be valid for files importing the source file which overrode the behaviour.
(excuse this "odd" sentence, not my mother tongue). So some modules use "int + int = int" while you additional files might have changed this.

For custom types you define the new behaviour right where the type is defined. No problem.


What might be possible is for combinations of "internal + custom" - therefor Brucey would have to create another "definition style" (like "method Operator_Int:+TList()". But somehow this feeld "hacky whacky".


bye
Ron


FireballStarfish(Posted 2016) [#35]
Well, it probably shouldn't be possible to override Int + Int, that's just asking for trouble.
But if, as you say, there was a way to define an operator "backwards", maybe by creating a method named something like "OperatorReverse+:TTest(i:Int)" in addition to "Operator+:TTest(i:Int), then that would still enforce the custom type being one of the operands. There would be a conflict if two custom types tried to define the same operator on each other (two types T1 and T2 both trying to define T1 + T2); but that could only happen if the two types are visible to each other in the first place, so it wouldn't be possible to accidentally alter the behavior of any already existing code by importing a new source file or module, right?


TomToad(Posted 2016) [#36]
How about Method Operator+:TType(I:Int) Commutative. With the Commutative keyword, you could have the operands in any order, TType + Int or Int + TType. Leave off the Commutative keyword and order will be enforce, TType + Int only.

That is probably over complicating things though, having two checks at each operator might slow things down too much.


Derron(Posted 2016) [#37]
> That is probably over complicating things though, having two checks at each operator might slow things down too much.


I would prefer to have only the "vice versa" option to go. If someone then "crosses" a previously overloaded operator, the compiler should issue an error.

WHY?
Imagine you wrote that operator overloads for "typeA + int = typeA" in your framework.
Now somebody is using your framework and knows "ahh, adding a number to typeA-instances leads to typeA" and writes then "(1 + myTypeA).ToString()". It will fail as it might now be a int instead of typeA.

So what do I suggest is something in the likes of:
(incomplete, at it limits functionality too much)
- you overload only "+/-/..." but not ":+" as this should be the same
- you overload "a + b" but also "b + a"
- if something already was overloaded "b + object" then you cannot do that, or your overload must return a compatible type (eg. in both cases a ":object" or ":TBase")

Of course you loose the detailed adjustability but you win on "you know what happens". As you then loose the ability to return "matrix_1x3" instead of "matrix_3x1" we could also:
- enforce that both "orders" have to be defined (typeA + typeB = typeA -and- typeB + typeA = typeB)

Nonetheless, having "different return types" depending on the order of arguments is something which should behave consistently over all objects. It will just introduce a lot of headache when not playing with matrices but "other objects". Do not forget: others might not know the return type in your certain scenario (you might have overridden it in an extended object) so they assume "normal behaviour".


Regarding "commutative" - subtracting vectors is not commutative, so it is no "generic" keyword suitable for every object-"concat".


bye
Ron


FireballStarfish(Posted 2016) [#38]
Another, simpler feature request: could we get overloading support for the unary versions of +, - and ~, as well as for ^?
(Curiously, :^ doesn't seem to exist in BlitzMax, but perhaps it could be added?)