Inline code..

Community Forums/Monkey2 Talk/Inline code..

EdzUp(Posted 2015) [#1]
I know I suggested this in another post but I would like to get everyones opinions here so mark can get ideas:

I would like something like:
Function MyFunc:Int( Value:Int, Value2:Int )
Local Val:Int
InlineCpp {
   #includes <iostream, stdio, conio>

   cout << ( Value *Value2 ) << endl;
   Val = Value *Value2
}
return( Val )
End Function


You could also inline ASM code as well this would be parsed straight into the end source file apart from the #includes lines where it would be broken down into seperate include files as required and is there so the parser knows whats required to run the source.

I think its a good idea mainly as sometimes its hell to convert code from c to max etc this way we could just inline the c routines we want without major modification.


ziggy(Posted 2015) [#2]
As I said in another thread, I think this is a very bad idea.


nullterm(Posted 2015) [#3]
Gives me a flashback to inline assembly on DJGPP and Watcom C/C++. Worked great there when you needed it.


marksibly(Posted 2015) [#4]
I can think of one cool use for this - resolving monkey names, eg:

global sum:int

function add:int( val:int ) native "c++"
   @sum=@sum+val; 
   return @sum;
end


The compiler would replace the '@sum' identifiers with the correct natively munged name.

I think a good use for inline code is writing 'stub' files for interfacing with external libs, eg:

#Import "blah.cpp"
#Import "blah/INCLUDE/*.h"

alias WCHAR:short
alias DWORD:int
alias LPSTR:bye ptr
alias LPCSTR:byte ptr
alias LPWSTR:short ptr
alias LPCWSTR:short ptr

extern

function _InitBlah:void( name:LPCWSTR Ptr )="initBlah"

public

function InitBlah:void( name:String ) native "c++"
   WCHAR *_name=@...();
   _initBlah( _name );
   @name.Discard( _name );
end



Nobuyuki(Posted 2015) [#5]
I can see how this can be useful, if most basic types can be exported/easily converted, making wrappers would be a bit easier. It does look a bit unfriendly to me, though. Definitely a "here be dragons" feel akin to external win32 declarations in vb6...


EdzUp(Posted 2015) [#6]
Yeah I dont think its everyone's cup of tea but it will as you say make wrappers a lot easier and sometimes converting from c to another language that doesn't have all the functions or operators in the same way can be a pain.


ziggy(Posted 2015) [#7]
It introduces lots of inconsistency and also ties the language to C++ as a backend , unless you're doing the same for any potential compiler backend, which would introduce even more inconsistence. Also, you will need to handle MX2 reserved words into C++ code, and C++ reserved words into MX2 code in some way. Not to mention that making any kind of refactoring with mixed languages is usually very time consuming. While I understand how this could seem like a useful "shortcut", I honestly think it would degrade the language quality enormously, it makes the code less readable, harder to maintain, and very platform dependent.

If that's not enough, you could break the MX2 generated code very easily and the compiler would not be able to notice this unless it gets to the C++ compilation phase, which makes finding and correcting errors a mess, as they will be reported in the wrong language (generated C++ code) instead of Monkey code.

I read also that this could be used to inline assembler into c++ inlined code. I was wondering which assembler would it run on iOS, Android, x86 and x64 PC... !! Unless you want to feed the surrounding C++ code with loads of IFDEF or similar for each potential architecture, which would make things even worse.


Gerry Quinn(Posted 2015) [#8]
I would just like classic inlining that would inject the Monkey code of the inlined function into the surrounding code:

For example:



Method New() turns into:




Samah(Posted 2015) [#9]
God no.

It's bad enough having to extern C++ in places. All this will do is introduce coupling between the platform-dependent and platform-independent code.
Please don't even consider this. It's just going to complicate things and make Monkey source code look ugly.

@Gerry Quinn: I would just like classic inlining that would inject the Monkey code of the inlined function into the surrounding code:

This is actually a very good idea, except the more Monkey-ish syntax would be to put Inline at the of the Method declaration line:
Method GetIndex:Int(x:Int, y:Int) Inline



GW_(Posted 2015) [#10]
I think I have to agree with the dissenters on this. My concern is how does this play with all the target platforms? win32/64,android etc. You can no longer assume the size of INT as defined in the M2 language.
Also, to write inline code, you would need to have expert knowledge about the monkey2 internals like 'bbString' ect. manually calling the GC to registers things ect..
I hope Monkey2 will have a slick and easy FFI like Blitzmax. but if anything outside of functions that only contain primitive types might create a lot of headaches instead.


Samah(Posted 2015) [#11]
@GW_: Also, to write inline code, you would need to have expert knowledge about the monkey2 internals like 'bbString' ect. manually calling the GC to registers things ect..

The idea is for trans to magically do this for you per platform, using pure Monkey code. The developer won't need to know any target languages. Any optimisations can be done in trans.


GW_(Posted 2015) [#12]
The idea is for trans to magically do this for you per platform

Yeah, and that's the opposite of inline code [per EdzUps post], the topic of this thread. If it's parsed and manipulated by trans then it's by nature not inline c++ code .
Ziggy brings up a lot of good points. The most important of which imo is semantic errors passing through to cpp and creating a lot of hair pulling trying to debug where it all went wrong.
Unless I'm not understanding what's being asked for. I imagine things getting very brittle very quickly.


marksibly(Posted 2015) [#13]
I am thinking of this for situations where you can't avoid having to write nasty, low level code to interface with some, erm, 'unique' c/c++. See: libjpeg (with it's own odd object/stream system) or freetype (lots of manual 'freeing' required I think) or any other number of popular C libs. Trans wont be able to magically make all these problems go away.

This code usually ends up in a blah_stub.cpp style file, along with externs in the monkey code to access the stub code, and often more stub code in the monkey code to make the cpp stub code usable in a sane way. I think it might be nice to localise all the crud in a single monkey file, instead of having it split in two. I suspect it'd minimize the number of stubs you'd need to write overall.

Trans wouldn't 'parse' c++ or anything - it would just replace @{identifier}s and that's pretty much it. This allows you to do stuff like "new @DataBuffer" from c++, something that is currently impossible/difficult. But yes, this would cause compile errors to pop up in generated code.

It doesn't tie monkey to c++, the 'native "c++"' after the function decl could be 'native "js"' or whatever. @{identifiers} would still be replaced, but monkey ignores the rest. Trans could either error out or ignore use of native "c++" in a js build. But this is really intended for 'unsafe' situations where you already know what language you're working in anyway, as you're writing stub code for libxyz.

It doesn't create any dependancy, because the dependancy was already there - the glue code has to go somewhere, eg: #Import "lib_stub.cpp".

And yes, this is definitely 'here be dragons' territory and you will need to know about gc and bbStrings and so on. But such code will be unavoidable in some situations - tucking it away in a .cpp file doesn't make it any easier to write. Perhaps this could?

Or not. Not really sure what I think of this myself and it's certainly in no danger of being actually implemented, but I do know from experience that wrapping 3rd party libs like libjpeg and freetype and any number of C libs can be very tricky - esp. if you want them to play nicely with monkey, like having libjpeg read from a monkey stream or something. And if there's a way trans can help out I think it's worth considering.


k.o.g.(Posted 2015) [#14]
"Off Topic"
@mark
is it possible to make "monkey strings" easier convertible in extern languages (c/c++ ...)


Skn3(Posted 2015) [#15]
Ok I am liking the sound of embedding the glue code into monkey, to help out in tricky situations. It is definitely a way to ease the life of module developers.

But still you may end of alienating less technical users a bit. I guess at this point, what are the real disadvantages of doing it?


GW_(Posted 2015) [#16]
Thanks for clarifying. I can see how this could help keep things manageable.


Danilo(Posted 2015) [#17]
Additionally:

Some other languages used '!' at start of line for that. If a line starts with '!', the complete line
goes directly (unmodified) to the output. In PowerBasic and PureBasic (FASM) this was used for direct ASM, and in
MX/MX2 it should go directly to the target (C++, JavaScript, whatever, ...).

ASM could be done with C++'s Inline-ASM then. Very powerful for writing low-level stuff (in the underlying language),
without the need to create and import a new file every time.

!#pragma comment(lib, "MSVCRT -VERBOSE")
!#pragma comment(linker, "<linker options>")
!#pragma comment(lib, "xxx.lib")

!#include <freeglut.h>

Function MyFunc(var:int, var2:ulong)
    ! unsigned long myvar;
    !
    ! // do something here
    ! myvar = var + var2;
    ! cout << "woohoo my var is " << myvar << endl;
    !
    ! return myvar;
End

Function SHR(var:int, value:int)
    ! return ((unsigned int)var >> value);
End

Function InlineAsm()
    !//
    !// C++ code
    !//
    !asm {
        #if COMPILER_PROCESSOR = MX2_PROCESSOR_X64
            #if COMPILER_OS = MX2_OS_MACOSX
                ! 64-bit ASM code for Mac
            #elseif COMPILER_OS = MX2_OS_LINUX
                ! 64-bit ASM code for Linux
            #elseif COMPILER_OS = MX2_OS_WINDOWS
                ! 64-bit ASM code for Windows
            #else
                CompilerError "unsupported target"
            #endif
        #elseif COMPILER_PROCESSOR = MX2_PROCESSOR_X86
            #Select COMPILER_OS
                #Case MX2_OS_MACOSX
                    ! 32-bit ASM code for MacOSX
                #Case MX2_OS_LINUX
                    ! 32-bit ASM code for Linux
                #Case MX2_OS_WINDOWS
                    ! 32-bit ASM code for Windows
                #Default
                    CompilerError "unsupported target"
            #EndSelect
        #elseif COMPILER_PROCESSOR = MX2_PROCESSOR_ARMv7
            ! ARM target code
        #else
            CompilerError "unsupported target"
        #endif
    !}
    !//
    !// more C++ code
    !//
End

CompilerError / #Error outputs a message and stops compilation.
CompilerWarning / #Warning outputs a message and continues compilation.


marksibly(Posted 2015) [#18]
> Some other languages used '!' at start of line for that.

I quite like that, as you're not in danger of mistaking native code for normal code since it's so ugly!


Samah(Posted 2015) [#19]
Seeing C++ code in a .monkey source file makes me think that Monkey 2 has lost its vision of being cross-platform. No amount of "only do this if the target language supports it" preprocessor hacks will convince me otherwise, because it still shows multiple languages in the one source file. It will turn away new programmers if they see it in an example, because they'll think "damn, I didn't want to learn C++ too, it looks scary", even if there's zero chance they'll need to use it.
I see no point in this if you can just Extern the code to a separate file anyway. Monkey's current implementation is fine.

Danilo: You could just put all that code in a separate file (including the target checks) and include it from the Monkey source. I don't understand the "not needing another file" argument if it makes the code cleaner.


DruggedBunny(Posted 2015) [#20]
I don't really think that's true... you occasionally see "!asm" instructions in PureBasic code on their forums, for example, but most people just stick to the core language where possible, since they know they're tying themselves down... but where you have to interface with non-native code, it's great to have.

I find Extern'ing a real drag in terms of setup, extra typing, translating parameters between languages, jumping back and forth between files/editors, etc, though it is of course tidier-looking in the end.

Being able to slap it into a Monkey function, though, and just call it... well, that wins, for me!

(Never liked "!" myself, would much rather it was easy to read/edit as normal code, copy and paste without having to ! and un-! each line, etc... or even decide to "do things properly" and move it to an external file to tidy up as Extern!)


Samah(Posted 2015) [#21]
@DruggedBunny: ...you occasionally see "!asm" instructions in PureBasic...

That doesn't make it right, neither is it an excuse to include it (because X language does it therefore it's ok).

@DruggedBunny: Never liked "!" myself...

Agreed, exclamation point means "not". Even though Monkey has an explicit "not" keyword, I still see ! as inversion.


DruggedBunny(Posted 2015) [#22]

neither is it an excuse to include it (because X language does it therefore it's ok).


Well, that misses my point completely:


... but most people just stick to the core language where possible, since they know they're tying themselves down... but where you have to interface with non-native code, it's great to have




Danilo(Posted 2015) [#23]
@Samah :
The binary Not operator '!' is used within expressions only, never at start of a line - so that's not a parsing problem.
You never write the following in MX:
1+2
!5

It could be any other sign, of course. For example % ^ . <

I don't have a problem with Native/EndNative or FreePascal's Asm/End blocks.
NativeCode
   std::cout << "hello monkey" << std::endl;
   asm {
      NOP
   }
End

Of course that's an advanced feature, used by ASM/C++ experts and library programmers.
If you don't need it, don't use it.

A great benefit of All-in-One codes is the ability to exchange it more easily in forum discussions.
Sometimes something isn't easy to do in MX, and somebody is helping you by providing a solution
using Externals.
Such solutions are always 2 sources, and every forum reader creates 2 new files then, paste the 2 codes
into the files, and run it - just for testing a small snippet.
With the All-in-One approach you just hit CTRL+N within your IDE to create a new (temp) file, paste the whole code and press F5 to run it.
It's at least 5 times faster and saves many people many time creating files within the right directories etc... when they just want to try a quick code from the forums.

Another benefit is that the code goes directly and unmodified to the underlying compiler/language/target. That's very powerful,
because I can use the full language directly.
With Mark's "Function xyz() native" approach, the compiler modifies the code, and I'm not sure everything of C++ will be
allowed then, or only parts of C++ that M(X)2 understands. Unless Mark includes a full C++/ASM parser of course.

With !nativeCode the M(X)2 compiler does not need to understand/parse the native code - it just gets
forwarded to the target language.

If variables / parameters / functions have a consistent naming scheme when translated to C++, the names
are not the problem. For example "Global x:int" gets translated to "int mx2_x" or something like that.
The advanced programmer then uses "mx2_x" directly in native code parts. If the naming scheme is
not documented, we compile and look up the correct native name in the generated files... ;)

Small example using PB, for better understanding:
Procedure myProc(x,y)
    CompilerIf #PB_Compiler_Processor = #PB_Processor_x86
        ! MOV EAX, [p.v_x]
        ! ADD EAX, [p.v_y]
        ProcedureReturn
    CompilerElseIf #PB_Compiler_Processor = #PB_Processor_x64
        ! MOV RAX, [p.v_x]
        ! ADD RAX, [p.v_y]
        ProcedureReturn
    CompilerElse
        CompilerError "unsupported processor"
    CompilerEndIf
EndProcedure

z = myProc( 10, 7 )
Debug z              ; outputs '17' in Debugger output window

Please note that '!' means "directASM" here. It goes unmodified to the underlying Assembler (FASM).
There is another, not-so-direct ASM method:
; x86 assembly example
;
Test = 10

EnableASM
    MOV dword [v_Test],20
DisableASM
  
Debug Test ; Will be 20

...but !directASM is much more powerful. For example: With !directASM I can use all latest language
features of the target (for example latest Opcodes that FASM supports, and things like FASM macro's).
With 'EnableASM' method this is very limited to what PB understands (and that's not all latest Opcodes,
and of course no FASM macro's).

!directASM was added after a feature request by me at the mailing list back then (over 15 years ago),
and I knew it from old PowerBasic/DOS even more years ago. After the developer said he will add it,
I bought a license for PB. It is still very powerful and used by the experts today.
I imagine the M(X)2 features we talk about today will still be used in 15 years, from now on...
...so I think it's worth it to discuss such features and make M(X)2 really powerful... ;)


DruggedBunny(Posted 2015) [#24]

I don't have a problem with Native/EndNative or FreePascal's Asm/End blocks.

NativeCode
std::cout << "hello monkey" << std::endl;
asm {
NOP
}
End



Yeah, I reckon that would be better than ! on each line, or something like:

NativeCode "cpp"
    ...
End



Samah(Posted 2015) [#25]
@Danilo: ...so that's not a parsing problem.

I never said it was a parsing problem. I just see it as meaning "not", coming from C-like languages that use it that way.

@Danilo: ...Native/EndNative...

This is much cleaner and more "Monkeyish". If inline code is supported, this would be a better way of doing it. However, it would probably be easier to have them as preprocessor directives, and you could include the target language this way.
#Native "C"
// stuff here
#EndNative


@Danilo: It's at least 5 times faster...

Where'd you get that metric from? XD

@Danilo: If variables / parameters / functions have a consistent naming scheme...

This is a terrible assumption to make, though. It means that the trans implementation could never be changed without breaking developers' code. A substitution method would be much better:
Local foo:Int
#Native "C"
${foo} = 10;
#EndNative

Note that this substitutes a reference to the Monkey variable, not the value.


ziggy(Posted 2015) [#26]
If this is finally going to be implemented (which I dislike a lot), I think the first Mark suggestion was best, because all the non monkey code into monkey code was limited to a function body. that's a bit better than allowing it to appear anywhere, without any kind of organization, just by using a ! prefix. This ! think does effectively tie monkey2 to C++ and allow for even worse code quality.


Danilo(Posted 2015) [#27]
ziggy wrote:
that's a bit better than allowing it to appear anywhere

Your suggestion makes it much more powerless. IMO there is no need to limit it to functions,
because most programming languages can have statements outside functions, too
NativeCode
    #pragma comment(linker, "<linker options>")
    #pragma comment(lib, "freeglut.lib")

    #include <freeglut.h>

    int myFunction(int x) {
        return x * x;
    }
End

Extern
    ' function declarations
    ' ...

Public

Function Test:Int(x:Int)
    Local returnVal:int
    Print "Test()"

    NativeCode
        // native code
        std::cout << "Test() - native code" << std::endl;

        MXLOCAL(returnVal) = MXARG(x) * MXARG(x);

        MXLOCAL(returnVal) = myFunction( MXARG(x) );
    End

    return returnVal
End

Function Samah:Int(x:Int)
    Local returnVal:int
    Print "Samah()"

    NativeCode
        // native code
        std::cout << "Samah() - native code" << std::endl;

        $(returnVal) = $(x) * $(x);

        $(returnVal) = myFunction( $(x) );
    End

    return returnVal
End

Function WithoutSubstitution:Int(x:Int)
    Local returnVal:int
    Print "WithoutSubstitution()"

    NativeCode
        // native code
        std::cout << "WithoutSubstitution() - native code" << std::endl;

        mx_returnVal = mx_x * mx_x;

        mx_returnVal = myFunction( mx_x );
    End

    return returnVal
End

'...

NativeCode is tied to the target language. If that is C++, NativeCode is C++. If the target is Java, NativeCode is Java.
If MX2 later decides to add an ASM target, NativeCode becomes ASM for this target.
It just gets forwarded to the underlying target language and compiler, whatever it may be.


nullterm(Posted 2015) [#28]
@ziggy, inlining lower level language into a higher is nothing new, and has been used alot. Notably assembly into C/C++.




GW_(Posted 2015) [#29]
@Danilo
That can be all done with an 'import' statement. There's no need to inline it.
Basically your either asking trans to bypass that block and dump it into the munged c++ literally or your asking trans to deal with it and which case you need start doing lexical analysis of c++ code (dragons!). Then what about other targets? java etc.
If It's just a lazy alternative to an 'Import' statement then that can create problems too. The first that comes to mind is c++ syntax errors and all the pain that can bring. The second is possibly where it gets dumped into the munged c++ file (if it's recursive descent).
I think inline code in Monkey2 is a solution that's looking for a problem


Danilo(Posted 2015) [#30]
Changed the above code using Samah's suggestion to use a "name substitution method".
In this case I used C++ Macros MXLOCAL() and MXARG() to get the MonkeyX generated name,
althought it probably requires some time to get used to it... :D


Samah(Posted 2015) [#31]
Why am I a function name now? XD

@Danilo: NativeCode is tied to the target language. If that is C++, NativeCode is C++. If the target is Java, NativeCode is Java.

But the target language isn't determined until compile-time. If I choose a Java target, it's going to assume that code is Java, which is obviously wrong.


Danilo(Posted 2015) [#32]
Samah wrote:
If I choose a Java target, it's going to assume that code is Java, which is obviously wrong.

Why should it be wrong? If you use a Java target, NativeCode is Java code.

Same as with MonkeyX today. If you choose Java target, you probably don't Import C++ or JavaScript files.
If you write NativeCode for all supported Targets, you use #IF TARGET=... - like we already do today.

That's why I am talking about "Native code", not "C++ code" and not "InlineCpp { .. }"...

I already said that:
If the target is C++, NativeCode is C++.
If the target is Java, NativeCode is Java.
If the target is JavaScript, NativeCode is JavaScript.
If the target is ASM, NativeCode is ASM.
If the target is C#, NativeCode is C#.

Errors within NativeCode are of course returned by the underlying compiler (Java, C++, ASM),
same as with imported native codes today.

If you write a small external function for 5 targets, you wouldn't need 5 files anymore.
With some #IF you could write 5 small NativeCode functions within one file, and paste
that 1 file into this forum, not 5 separate files.


Samah(Posted 2015) [#33]
@Danilo: Why should it be wrong? If you use a Java target, NativeCode is Java code.

Function Blah()
	NativeCode
		int a = 10;
		int *b = &a;
	End
End

Now build for your Java target.


EdzUp(Posted 2015) [#34]
Couldn't ya use

[Code]
NativeCode( C++ )
//my code here
End

NativeCode( Java )
//Java code here
End
[/code]

This way it would work perfectly


Danilo(Posted 2015) [#35]
@Samah:
How often do you want me to repeat the following quote?
If the target is Java, NativeCode is Java.

Of course you can't compile to a Java target and write ASM/C++/JavaScript in a NativeCode block.
It's somehow logical, so I really don't understand why you want to use native C++ code with a Java target.
Doesn't make sense to me.

Choose your style:
Function Blah()
    #IF TARGET = "JAVA"
        NativeCode
            // Java code here
        End
    #ELSEIF TARGET = "CPP"
        NativeCode
            // C++ code here
        End
    #ENDIF
End


#IF TARGET = "JAVA"

    Function Blah()
        NativeCode
            // Java code here
        End
    End

#ELSEIF TARGET = "CPP"

    Function Blah()
        NativeCode
            // C++ code here
        End
    End

#ENDIF



Samah(Posted 2015) [#36]
@EdzUp: Couldn't ya use

The point Danilo is making is that since the code will essentially be copy-pasted verbatim, trans doesn't care what language it is. Regardless of how this is implemented, you're still going to need a heap of precompiler rubbish to make sure you're choosing the right native code block for the language. This is where having multiple files and just including the right one is so much cleaner.

@ziggy: If this is finally going to be implemented (which I dislike a lot), I think the first Mark suggestion was best, because all the non monkey code into monkey code was limited to a function body. that's a bit better than allowing it to appear anywhere, without any kind of organization, just by using a ! prefix. This ! think does effectively tie monkey2 to C++ and allow for even worse code quality.

I have to agree with ziggy here. I will never wholeheartedly support inline native code, but if it really has to be there, it should be as clean as possible. If you give the developer an inch, they'll take a mile and code will look absolutely horrendous.

@Danilo: #IF TARGET = "JAVA"

This is what I was waiting for you to add to your examples. Of course I don't want to use C++ in Java. But without a way of telling trans that, it'll just get copied in and compilation will fail.


Danilo(Posted 2015) [#37]
@Samah:
That's why I was talking about C++/C#/Java/JavaScript/ASM previously. I want it flexible, and I always said that
NativeCode depends on the target platform.

While I take this into consideration (for future expandability), the only M(X)2 target currently know to me is C++.
Maybe I have missed something, but up until now I have not read about M(X)2 targeting Java/JavaScript/C#/ASM. Only C++.

That means C++ on all desktops, iOS, and Android (using NDK). Nonetheless I prefer flexibility and future compatibility,
so I don't propose the use of "InlineCPP" or something like that, because it would limit the usefulness to C++ only.


Samah(Posted 2015) [#38]
@Danilo: I want it flexible...

Copy paste is pretty flexible. :)


degac(Posted 2015) [#39]
I'm not a pro-programmer, and I don't know so much about C++ & at.
But, maybe, I'm missing the logic/goal of the MX2 'language': I want to write a program in an agnostic/independent language as Monkey. No strange syntax, compiling condition, setups etc or whatever.
If you start to put in source-code 'alien' elements, MX2 starts to be 'useless',it isn't? At this point just write in your C++ (or whatever) language...
I think MX2 should focus on language features/syntax easy-to-understand and write. Maybe something 'magic' (converted automatically etc
TRANS2 should focus on the best performance in translating to a specific target.
If I need to inject inline code for a specific procedure etc, I suppose that the MonkeyX's implemented 'command/function' is not written well, and needs to be changed/improved.
Indeed if the problem is 'inline code is useful for bridge with other .dll etc', I think a proper solution - at design level - should be find: a sort of automatic converter/etc that take care of these things (ie: if Trans find a IMPORT THIS.DLL it should create a sort of bridge between MonkeyX2 and that .dll - once converted is 'stored' in the 'module' list for future use. Maybe is not the most elegant solution, but it will resolve potential future problems I think.

As stated in other thread, MX2 needs a better (and working) debug-tool: adding 'external' things I think does this work very hard to maintain.

As I said, I'm a profane about these things, surely I wrote many stupid and silly things!, I just hope you get my idea.
Cheers


Danilo(Posted 2015) [#40]
degac wrote:
If you start to put in source-code 'alien' elements, MX2 starts to be 'useless',it isn't?

It's just one advanced feature we are talking about here. A feature that enables programmers to do many powerful things using the target languages directly.
It does in no way make the language useless to you, and if you didn't master the target languages yet, you just don't need that feature and can safely ignore it.

The rest of the language is still the same to you, and very useful. C++ programmers don't use all of the C++14 features within every source code.
It's not a problem for you, because this feature does not take away any of your possibilities using M(X)2.
It just adds some possibilities for programmers that work directly with the underlying target language(s).


itto(Posted 2015) [#41]
I second what @decac said.

It's not a problem for you


Even if in theory it doesn't constitute a problem, in practice it would mean Mark and the other developers will have to spend time implementing/mantaining/updating such features, which usually means other (perhaps more important to me) features/bugs/improvements will get delayed or not implemented altogether. And yes, it's a problem for me if the project I'm patroning starts going in a direction I don't like.


Danilo(Posted 2015) [#42]
itto wrote:
And yes, it's a problem for me if the project I'm patroning starts going in a direction I don't like.

I heard rumours the project you are patroning could just be forked for free to make it more powerful.
Rumours say it will be called "MonkeyX2 Pro". ;)


degac(Posted 2015) [#43]
@Danilo

I understood the basic idea of inline coding, and it has great potential and offers flexibility (well, sort of of...), no doubt about it.
I'm not a technician or a professional programmer, so my knowledge is of course limited.
And of course I will never use a feature I don't know, I'm not a masochist!, so it is really not a 'problem' if that feature is implemented in some way (just another complication while reading the code :P).

But if you start with a 'meta-language' (as we could define MX) and then you start to 'personalize' it with C++ code (for example), for YOU, what is the sense to program in Monkey at this point? Better write everything directly in C++...

And - at this point I'm very curious: what could be a real benefit for writing in an alien-code, I mean 'real examples'.

I can understand some specific arithmetic functions/algorithms (on BMax I read about some sort of asm-optimization using MMX extensions and so on). But again, in a world where the 'compilation/optimization' is done by other tools (Mingw at this point I think) what are real cases? And what are the advantages for the results (not for the code written!)?

As I tried to explain above, if a function/command (ie: Print() ) already present in MX default command set, is slow - and a specific solution written in C++ results faster, at this point change the MX-code to implement it. It seems more logic to me. The solution will be 'added' directly to the MX-DNA. And for everyone.

Or again, array-item or collection access I suppose could be faster (I really don't know) if written for example in ASM (and at this point you are chained to a specific ISA, as I really doubt that x86-ASM is the same for ARM). I prefer that the 'faster' solution will be implemented in MX-code.

Really, having this feature TODAY, seems quite out-of-place / out-of-time.
In BlitzMax - that is monolithic for x86/win x86/mac x86/linux - having such a feature probably was (or is) a great gun in the right hands (like I mentioned about about the MMX support).
In MX - as meta-language - it should be completely agnostic to perform the motto 'write-once, runs everywhere'.

Another example (maybe not really comparable): shaders.
It seems that writing a 'standard' shader that runs on every hardware is a very complex job. So 'high level' languages were created. They are doing the opposite road!
And Mojo2 (for what I understood of course :) ) has just only mega-shader (based on the low-common standard I believe) (with parameters I suppose) but the level of customization will be limited.

I repeat, I don't say 'MARK DON'T DO IT', just thinking if this work is necessary/needed/useful compared to the possible results (in terms of time/work/gained performances).

I believe Mark understood from MX1 that some fancy features implemented had some costs... and I suspect that re-inventing the wheel again or walking on the same roads is not very funny.

Just my opinion of course.


ziggy(Posted 2015) [#44]
@ziggy, inlining lower level language into a higher is nothing new, and has been used alot. Notably assembly into C/C++.
Yes, I know this. But most modern high level languages avoid this old idea, so they can provide a good abstraction layer that does not depend on a given architecture. That's why high level programming languages are for! hand written ASM is a lot less efficient compared to what a modern compiler can produce, so this feature has no benefits. It has been not even considered on most modern langauges designs. Take into account that C++ has lots of legacy things like the "inline" keyword that literally have no sense in the 21 century, and it's definitively not the language to take as a reference of what to do on a modern language design.

Then, regarding inlining a dinosaur like C++ on a modern language like what Monkey2 could become: I understand it can be "handy" to get things wrapped faster. But at the same time it will allow a mixture of coding paradigms (managed/unmanaged, OO/C++, etc) which won't help generate quality code, nor a focussed and consistent language "corpus". I suspect and hope, it won't be widely used and kept only to provide wrapper functionality, so in this situation, a function-based approach coupled with the Extern functionality should be enough.

It just adds some possibilities for programmers that work directly with the underlying target language(s).

But that's a bit like having breakfast while you're taking a shower. You maybe gain some time, but I prefer each thing in its place, separately, well organized and clean. At the end, doing it separately you can get much more quality breakfasts and showers.


Samah(Posted 2015) [#45]
@ziggy: But that's a bit like having breakfast while you're taking a shower.

I love your analogies. :)

@ziggy: hand written ASM is a lot less efficient compared to what a modern compiler can produce...

Exactly.

I see no reason for this feature other than to alleviate the need for multiple files. I would much rather keep each language to its own file. As soon as you put platform-specific code in your Monkey source (even if it's wrapped with precompiler directives), I feel that you may as well just write your entire game in C++. As it stands, Monkey 2 is just a wrapper on C++.


ImmutableOctet(SKNG)(Posted 2015) [#46]
Alright, it's time for me to drone on about this topic. This ended up being incredibly long, and is more about interoperability, rather than strictly "inline native code". I hope this does my stance justice. Enjoy the wall of text.

For one thing, inline/embedded assembly doesn't need to be a thing. There's no point to it, especially if you want external C++/Java/InsertLanguageHere. In the case of C++, you're already covered by that, and it's not like you'd be using assembly anywhere else in Monkey. Honestly the idea of inline external languages sounds both good and bad. On one hand, there's the hard truth; making wrappers upon wrappers for libraries is nothing but unneeded boilerplate.

On the other, you want the language to be self-contained. Here's the thing; it's not. Sure, you can have your portable code, and expect the compilers to do the work of abstraction, but you're still dealing with a modern programming language, and it even uses source-to-source. The fact is, there isn't any actual inconsistency we don't already have if we have inline native code. The only difference is, you're not writing platform-agnostic wrappers/other, you're writing specifically for a targeted language. You might be thinking to yourself that it's a perfectly good system to abstract yourself using the API you wrote to wrap a library. That may be true, but it's also not realistic. Control is the key here, if we have embedded target-native code, then we can get things done faster, and do what we need to in one place.

The fact is, a lot of people simply want to wrap libraries that exist on one or more very specific languages. Monkey abstracting this is useless under those situations. When someone uses external bindings of any type, they should expect platform dependence.

For this reason, I personally think we should have some kind of "inline" native-language feature for Monkey, but it shouldn't always be recommended. It's an explicit dependency on the native language/platform, and it should be used as such. If I want to wrap the Windows API, I shouldn't really need to abstract myself from the native language, as I'm already depending on one, as well as the platform. If I'm using something platform-portable, but still C/C++ only, like ZLib, I shouldn't have to care about abstraction. Not having this, or at least a better way to access native languages already cripples Monkey for some developers.

The arguments against better interoperability with Monkey are pretty awful thus far. I can understand not wanting to have inline native code, but the fact is, if it's going to be in the language, it should be used properly. You're not going to have problems writing portable code, just because the language can have dependencies. It should basically just be an easier way to handle what we already do.

No one should recommend using native code without a nice platform-agnostic wrapper, unless the person using the external bindings understands the situation. If I want to make a tool for desktop targets only, I should be able to use Monkey as my front-end, even if the code's not portable. There's two main reasons for this; 1. I like Monkey as a language, and I imagine I'll also like MX2, and 2. If I ever decide to overhaul my code, or write/use an in-Monkey solution if one eventually exists, I can effectively pick up my existing codebase. No need for a re-write, just some native bindings to settle. Bindings I decided to use initially, and therefore am responsible for.

Another thing to imagine as far as native code is concerned, is the way we handle implementation details. The fact is, with Monkey's current array system (For example), there's no easy way for me to optionally delegate native code as a backend, without getting into some terrible hacks. The answer of using 'DataBuffers' is fine, until you realize that Mojo already wants people using integer arrays. The end result is having to generate a brand new array to do something like this, or perhaps an "experimental" abomination like this, more specifically, this. Now, if you look at that code, you can see that the native code (C++) isn't too bad structurally. However, I end up wanting to do this, but I have no way of using a routine like that from Monkey, so I have to force converted file-paths, instead of just using 'Streams'. Now, if the native implementations of classes like 'Stream' were much better oriented around native interoperability, then I could simply take in a 'Stream', and read segments into temporary buffers. From there, I could encode as I needed to. The same goes the other way, in that example, I wouldn't need to use the language's native streams, as the API to the 'brl.stream' Monkey module (From the native language) would already be adequate. I could just take in a 'BBStream' (Or similar), and be able to use it similarly to how I would in Monkey. At least for this situation the real problem with using 'BBStreams' came down to not having better constructors/access to 'BBDataBuffer'. In cases like this, it's really not that much work to get something working, even if it's rather basic. Currently, you'd effectively have to generate a new container-buffer on the heap to get anything done. You'd also run into some problems with 'BBDataBuffers' and reference counting (Thanks to 'Object'), so having the native implementations contain extra functionality would be ideal.

What I'm really getting at here is my reasons to not use Monkey to make some tools/programs I would otherwise use it for. A few years ago, I wrote a tool called "IOSync" in BlitzMax. Now, at the time, I thought BlitzMax was fine enough for the job, and to some extent it still is. There were some problems, sure, but I could put things together quite well. For a number of reasons, I dropped the program, but it wasn't necessarily Max's fault. However, now that Monkey's moved forward with the language, and presents better performance at certain levels of abstraction, I started using Monkey.

The two big problems I had with Monkey were its interoperability with native languages, and the portable-only mindset. These aren't problems for certain types of programs. In fact, for games, Monkey hasn't been doing bad at all. But since it's easily evolved into a general-purpose language on top of being a game-language, problems creep up. For example, my 'hash' module, though pretty solid (The MD5 implementation), if I could use native code more effectively with Monkey, then I could have a C++ native backend, and use my current code-base as a platform-independent fallback. This would allow me to have better performance, and use an existing implementation on some platforms.

So, back on the topic of choosing a language, I initially really wanted to use Monkey to rewrite my "IOSync" program. The thing was, I needed to use a lot of Windows API functionality, while still abstracting myself easily. On top of this, I potentially needed threads, and access to any other APIs that would be useful. In addition to this, I wanted a small memory footprint, and I potentially wanted to revive an old networking module of mine. Pretty quickly, Monkey's way of handling external bindings became too much of a problem, and the lack of structures / scope-allocated types, and references to these types (As well as primitives) took its toll very quickly. Eventually, I caved in, and I used C++. However, assuming interoperation with native languages would have been better in Monkey 1, and ideally I could better utilize the stack, I would have used it. This is why I think Monkey's take on external bindings is flawed. For those interested, this is the program I was referring to.

Assuming you've made it this far into my post, I don't have the money to support Monkey 2 financially, but I think it's definitely starting to shape up to be what I was looking for originally. Mark, if you want to add inline native code, that's fine by me. We all just need to make sure new users don't fall into the trap of using it for normal code.

Also, @ziggy: The 'inline' keyword is still useful in C++ today, but mainly for playing nicely with the linker. And possibly when dealing with embedded systems. - Just thought I'd throw that out there.

EDIT: Attempted to fix the formatting.


therevills(Posted 2015) [#47]
ImmutableOctet(SKNG) - sorry TL;DR... maybe put a summary at the top of the post...

My 2cents = I am against inline coding, we are in the 21st century and have moved on from that.


GW_(Posted 2015) [#48]
The two big problems I had with Monkey were its interoperability with native languages, and the portable-only mindset.

We've all run into this problem. Your pain is shared.
Both of these are much better handled in Monkey2 by it's very nature (by my understanding of Marks posts and intents). In-lining native code isn't required.
Not to mention, it's not very Blitz-y to litter the syntax that way.


therevills(Posted 2015) [#49]
Not to mention, it's not very Blitz-y to litter the syntax that way.


Well... in Blitz Basic 2 you could actually inline ASM...but that was a long long time ago.


ziggy(Posted 2015) [#50]
@ImmutableOctet(SKNG): Interesting read. however, not sure how inlining would solve all your issues in a way external files could not. Monkey2 objects are probably not compatible with regular C++ STL classes because GC, different overloading, properties, etc. But the heavist thing is, basically because C++ objects need to be destructed manualy and an not managed code.


degac(Posted 2015) [#51]
Profane question:

Is it possible to create a 'special object' (in MX2, like the 'standard' supported ie: Object) used just to bind/bridge C++ objects (and nothing else), handled directly by MX' GC (so it could ignore this 'special thing' if required or act when needed, behind the curtains).
It seems that inline seems required just to bridge MX to C++ library/things.

Ok, I wrote the daily nonsense... :P


marksibly(Posted 2015) [#52]
> Is it possible to create a 'special object' (in MX2, like the 'standard' supported ie: Object) used just to bind/bridge C++ objects

Alas, there is no such thing as a 'c++ object', beyond 'objects with a virtual function table', which you'll be able to declare via Extern, ala bmx/monkey1.

The problem is, every lib has their own way of dealing with object memory management - some will require you to manually delete it, some will require you to ->release() it, some will require you to call AweSomeLib::MemoryManager::ObjectPool::Current()->ReturnToPool ( obj ) and so on.

Monkey2 can help a bit - it can converted Strings to CStrings/WStrings/UTF8Strings etc for extern function calls, but sometimes that wont even be enough!


Danilo(Posted 2015) [#53]
marksibly wrote:
beyond 'objects with a virtual function table', which you'll be able to declare via Extern, ala bmx/monkey1.

While you mention it, would it be a problem to support extern C++ classes without the requirement of being completely made with virtual functions?
Most C++ libs don't work that way. Often only few methods are virtual (if any at all), and many are not (why should everything be virtual?).
I already run into such problems with Monkey1, and eventually decided that I use my precious lifetime for better things.
The potential is there, it just should be made really nice to interface to external stuff like .lib, classes/structs/constants/macros, platform APIs (WinAPI, Mac APIs, Linux APIs, DirectX, Direct2D, OpenGL, ...).

The world needs such a nice procedural + object-oriented BASIC-like (not really, let's say bracket-less ;)), clean syntax, native, powerful, open, cross-platform language/compiler that can also (additionally) be used for application/tools programming. There is not much available to choose from in this category/area.


degac(Posted 2015) [#54]
Another - surely stupid - question/idea:
Different libs behave in different manner in dealing with memory allocation. But this must be done (by user or not) in some way.
I suppose MX2 will need to use some of these libraries. Unless you don't want to re-invent the wheel for every single feature - (networking for example) or making manually the binding for each possible lib/target combination (I clearly have no idea on this!)
So, it will be possible to create a tool to automatize the binding a lib ready for MX2?
I mean a sort of 'translator' specifically designed for this task.
And it could find (or try to find) the various release method (or delete or whatever) and register this methods/functions to be passed at MX2 code (like AddHook() for Bmax).
A new command could be called (or maybe automatically called in main() or render() loop) to handle all the 'extern' dependency. I remember Bmax had a FlushMem() command in the early version (of course linked to the GC and not with other).

I'm sure this is NOT a simple task, but maybe could be something useful for the language itself.
(I know I've a too simple idea of things works :P)


Danilo(Posted 2015) [#55]
degac wrote:
So, it will be possible to create a tool to automatize the binding a lib ready for MX2?
I mean a sort of 'translator' specifically designed for this task.

Of course you usually automatize such boring tasks, like importing hundreds or thousands functions/methods/classes/structs/enums/constants/macros,
as good as you can. ;)