Please add TDM-GCC 64 bits and VS 2015 support

Monkey Forums/Monkey Programming/Please add TDM-GCC 64 bits and VS 2015 support

JaviCervera(Posted 2015) [#1]
Right now, Monkey requires a 32 bits version of TDM-GCC to build the executables, although the 64 bits version of the compiler is perfectly capable of building 32 bits apps (just pass the flag -m32 to it). It would be nice if Monkey could detect a 64 bits TDM-GCC installation and simply pass the -m32 flag to it when compiling.

Also, it is not working for me with Visual Studio 2015 Community. It would also be nice to have support for it, I don't want to install Visual Studio Express 2013 just for Monkey.


Soap(Posted 2015) [#2]
These things probably won't happen unless you make them happen.

Why use TDM-GCC/VS over other options that work? What do you need to do in VS that can't be done without?


JaviCervera(Posted 2015) [#3]
Well, both compilers are currently supported by Monkey, but VS support is one version behind (2013) and only the 32 bits version of GCC is supported (at least in Windows).

What other options that work are you referring to?


Soap(Posted 2015) [#4]
I need to know what your specific needs are.

http://www.monkeycoder.co.nz/downloads/tdm-gcc-4.8.1-3.exe

This stand alone msbuild 2013 may work https://www.microsoft.com/en-us/download/details.aspx?id=40760

If you feel like it you could make a VS 2015 version of the project and contribute it to the community. Did you try to follow steps to upgrade the project? AFAIK if you want to use the older project in 2015 you can but you must install some extra files. Visual C++ tools in Visual Studio 2010 with SP1, Visual Studio 2012, or Visual Studio 2013.

Using -m32 flag to build 32bit versions is normal afaik. I use it every time I make Linux builds. Look up ways to detect TDM-GCC 64 bit version. Makefiles can contain conditional statements. If you create a new version of the Makefile that handles it then share it.


marksibly(Posted 2015) [#5]
Just checked, and adding the -m32 flag seems to work OK with both 32 and 64 bit mingw.

I have tried mingw64 in the path, but there have been various 'issues'...it's a little more 'bleeding edge' than mingw32 I think.

There is also a bug in mingw64-5.1.0 (which I just ran into the other day) which means it cannot link.with msvc style 'lib' files, so you must use mingw64-4.9.2. This has apparently been fixed and will likely be in the next mingw release, but for now I'm reluctant to have monkey 'default' to using mingw64.

I will add the -m32 stuff though, and perhaps add a MINGW_PATH setting to config.winnt.txt that links to mingw64-4.9.2 only, along with a suitable warning comment.

As for msvc2015, you can open existing projects with it fine, but the default project produced by monkey is for msvc2012. Probably time to upgrade...


skid(Posted 2015) [#6]
Mark, for VS2015 build automation I use:

call "%VS140COMNTOOLS%VsMSBuildCmd.bat"
MSBuild nitrosdk.sln /maxcpucount /p:Configuration=Release

and for VS2012:

call "%VS120COMNTOOLS%vsvars32.bat"
MSBuild nitrosdk.sln /maxcpucount /p:Configuration=Release

I think you need to support both, unless you want to force people onto Windows 10 which might be the right thing to do for Visual Studio use, dunno.


marksibly(Posted 2015) [#7]
Ok, I've upgraded the glfw, glfw3 and glfw3_angle targets to use vs2015 projects so you'll now need this installed if you are doing msvc builds.

Possibly a bit of a pain for some people, but it apparently works on Windows7 and up and I think it's a good idea to keep up with the latest version of VS.

I haven't dared touch the winrt stuff, but winrt projects need to be opened manually anyway and that still works fine. Also, loading these projects into vs2015 didn't cause any 'project upgrade' so I don't think there's anything I need to do here.


dawlane(Posted 2015) [#8]
It shouldn't take that much to modify trans and TED to pass the -m32 switch an an option to the Makefile, but you have to remember that the intermediate object files created will need to be removed when the switch is changed and the project rebuilt.


TheRedFox(Posted 2015) [#9]
I am coming back to Monkey.

I have been using VS2010 by replacing the 120 with 100 in the vcxproj template for the glfw3 target in MonkeyX 85e.

Works fine.

It's hard to move from 66 to 85e :-)


JaviCervera(Posted 2015) [#10]
Thanks for the -m32 flag, Mark! Hopefully that'll be enough :) VS2015 is not as important to me, but I asked because it is good to have the latest version supported.


marksibly(Posted 2015) [#11]
> It shouldn't take that much to modify trans and TED to pass the -m32 switch an an option to the Makefile

Argh, forgot people have this inexplicable (to me) desire to create 64 bit apps for windows, so I guess simply forcing -m32 is out.

Will revisit this a bit later I think...


dawlane(Posted 2015) [#12]
>Argh, forgot people have this inexplicable (to me) desire to create 64 bit apps for windows, so I guess simply forcing -m32 is out.
Not to worry Mark. I think I should have it covered for GCC on Windows and Linux. With luck, I should have something uploaded to dropbox and linked here within the hour. Currently swearing at Windows 10, BlitzMax and that bloody TDM bug. For reasons unknown, I cannot build BlitzMax application in Windows 10. Guess I will have to hack the BMK to use the MinGW cross compiler in Linux.


dawlane(Posted 2015) [#13]
As promised.
https://www.dropbox.com/s/m2yo61lgxr3lotr/monkey-86c-dawlane-mod.zip?dl=0


marksibly(Posted 2015) [#14]
Sorry, but that's a bit beyond me - haven't messed with cross compilers so wouldn't even know how to test it.

I have come up with a solution of sorts for the original windows-only request though, via the addition of some new config vars...

#GLFW_GCC_CC_OPTS="" 'common gcc opts for all hosts
#GLFW_GCC_LD_OPTS=""
#GLFW_GCC_CC_OPTS_WINNT="-m32" 'host specific opts, ie: force 32 bit apps on windows, default on others.
#GLFW_GCC_LD_OPTS_WINNT="-m32"
#GLFW_GCC_CC_OPTS_LINUX=""
#GLFW_GCC_LD_OPTS_LINUX=""
#GLFW_GCC_CC_OPTS_MACOS=""
#GLFW_GCC_LD_OPTS_MACOS=""

...so to build a 64 bit glfw app on windows with mingw64, you will need to add...

#GLFW_CC_OPTS_WINNT="-m64" 'or just use ="" as mingw64 generates 64 bit apps by default.
#GLFW_LD_OPTS_WINNT="-m64"

...to your project AND have mingw64 (but not 5.1.0!) in your config.winnt.txt.

Doesn't work with glfw3 + angle though, as the angle lib is 32 bit.

Is this 'good enough'? Will it help with your cross compiling gymnastics?


dawlane(Posted 2015) [#15]
Believe it or not Mark, it's easier to build a fair bit of stuff with the MinGW32 cross compiler in Linux than it is to use MinGW32 in Windows. For example wxWidgets, tends to be a bit of a ball ache to build on Windows with MinGW32 without a command terminal such as MSYS, but is a piece of cake under Linux with the cross compiler. I have not included a cross compiling capability from Windows to Linux or OSX for two reasons. The first is that there are no decent pre-built stand alone cross compilers for Windows to Linux/OS X and building one is not a simple task; it's the same problem between Linux and OS X. And secondly, Apples OS X SDK license. The PDF in the archive explains how to install and set up the cross compiler tools chain for a number of Linux distributions and OS X, and how to build OpenAL-soft for use with Windows or WINE.

I have noticed this morning that I messed up with the config.winnt.txt. I pasted MinGW stuff into it that was actually supposed to go into config.macos.txt. The config.winnt.txt isn't used on Linux or OS X. I will have to correct it and include the OS X toolchain when I get the iMac set up again later on today.
The binaries for TED are compiled with Qt5.5.1 (Windows 32bit VS2010), while the Linux version uses Qt5.1.2 (64bit), which should be native in the Ubuntu/Mint repositories.

#GLFW_GCC_CC_OPTS=""
#GLFW_GCC_LD_OPTS=""
I think that those should have been there from the beginning. It would have saved a lot of Makefile hacking. But just passing CPU code generating options such as -m32 will still leave you with the problem of the intermediate object files lurking around waiting to trip up the linker. It's why I chose to pass a parameter as a transcc option. Doing it this way allowed me to append either 32/64 to the end of a release/debug directory and slip in a variable in the gcc command line that can be picked up by the makefiles, hence making it much easier to keep the builds separate. As for OS X desktop 32/64 binary output, would you not have to pass parameters to xcode along with the xcode project file? And then the question would be how many Mac's out there are still using an SDK with 32bit support?

As for the cross compiler gymnastics. It wouldn't make a difference as transcc it's self has to be tricked into believing that it is compiling on a Windows OS.

Here's a general out line of how it works for MonkeyX
Install the cross compiler. Build the extra libraries or copy over actual Windows libraries. Set up WINE for testing or copy over the end results to a machine running Windows OS.
If there is a valid MinGW tool chain (MINGW_TOOLCHAIN) or in the case of OS X tool chain and path in the config.linux/macos.txt file, along with the cross compiler switch in the transcc command line. The cross compiler tools usually follow the naming convention of CPU-COMIPLER-TOOL e.g i686-w64-mingw32-g++. Hench MINGW_TOOLCHAIN is use to form part of the resulting command.
Transcc forces the ENV_HOST to winnt to trick the parser to not ignoring the winnt target directives as longs as the above checks out ok.
It's then a case of building the command line for use with a modified Makefile as in the case of glfw and forcing the destination directory to gcc_winnt.
Then once compiled, append .exe to the resulting executable and if the run option is used, then execute the result using WINE.
If there is no cross compiler then a normal host build is created.


dawlane(Posted 2015) [#16]
OK new upload to dropbox. I have trimmed it down a little by removing a few things that haven't really messed with and added the mac os tool chain. TED OS X was built with Qt5.5.1.
https://www.dropbox.com/s/c2ttafocq5b6ahl/monkey-86c-dawlane-mod-crosscomp.zip?dl=0

@Mark: If you are adding #GLFW_GCC_CC_OPTS="", #GLFW_GCC_LD_OPTS="". Then I would suggest scanning for those compiler options that deal with binary code generation. You could then append either 32 or 64 bit to the output directories before the build. But I think that passing such an options like -march=32 etc directly as an argument to transcc would allow you to trap those arguments for a bit of command line reprocessing before passing it over to the native compiler tools chain. Think along the line of being able to pass selected arguments along with a VS project to MSBuild.


marksibly(Posted 2015) [#17]
Actually, I've gone for #MINGW_GCC_MSIZE_WINNT="32", and have appended this to output Debug, Release dir name. I had tried this earlier, but it crapped out on glfw2 as the makefile was still building glfw .o's into the glfw src dir. This turned out to be fixed in the glfw3 makefile though, so I've fixed the glfw2 one based on that.

I've left #GLFW_GCC_CC_OPTS and #GLFW_GCC_LD_OPTS in there though for advanced users, although I haven't attempted to detect when they change, which means if you play with these you'll have to delete your build dir.

Anyway, I promised people an xmas monkey2 demo so I better get back to that!


JaviCervera(Posted 2015) [#18]
Looking forward to that demo, Mark. I have already ported my Vortex 3D lib to C++, and will start working on integrating it with Monkey2 soon :)


marksibly(Posted 2015) [#19]
> I have already ported my Vortex 3D lib to C++, and will start working on integrating it with Monkey2 soon :)

I'd wait a while before attempting that! Turns out interfacing with C++ will be quite tricky...


Danilo(Posted 2015) [#20]
Could you explain/elaborate on that please, Mark?

I mean, a Class and its (Private/Virtual/Protected) Method's map to a class and (Private/Virtual/Protected) methods,
simple data types (signed/unsigned) map to simple data types (signed/unsigned), and braces map to begin/end keywords.

Interfacing to other languages (C++/Java/C#/JS) may be the key, so I'm just wondering what could be the problem.


marksibly(Posted 2015) [#21]
The main problem is with c++'s funky type system.

For example, there are (at least) 3 way to pass a struct in c++:

void update( T t );
void update( T &t );
void update( const T &t );

2 of these don't map to monkey2's struct and will generate c++ errors when you build.

There are even more ways to pass pointers:

void update( T*p );
void update( const T*p );
void update( auto_ptr<T> p );
void update( shared_ptr<T> p );
void update( unique_ptr<T> p );
void update( weak_ptr<T> p );

C++ also returns 'structs' in a different way from monkey2, and monkey2 has no notion of c++'s 'const' methods, so it's impossible to override a native const method.

I think the only sane solution is to use a 'bridge' class (that extends monkey2's bbObject class) to interface with native classes (at least, those with virtual methods).

I wrote such a bridge class by hand for the litehtml lib, and it wasn't too bad. It's possible to use multiple inheritance in c++ to make it easier, as you can write a c++ class that extends both bbObject and the native object it represents. Simple methods can be directly used via Extern, while more complex methods that take 'weird types' will need to be overridden by a monkey2 compatible method.

I would definitely like to see this process automated, but I don't think it should be part of monkey. It's bound to involve lots of nasty special case code that will be constantly changing to meet the needs of new APIs with new weird pointer types. Instead, I think an external tool that generates glue classes would be a better approach.


Danilo(Posted 2015) [#22]
Thanks for the explanation, and Merry Christmas!


dawlane(Posted 2015) [#23]
Personally Mark I think that adding more preprocessor variables such as #MINGW_GCC_MSIZE_WINNT="32", #MINGW_GCC_MSIZE_LINUX="32" is over kill for such a thing and would start to get a little confusing. I think that a lot of people would prefer to just flip a combo box in an TED or any other IDE to select a CPU architecture to build, and avoid having to remember or look up a particular option as well as typing it. I will be looking at adding a combo box to TED for use with the version of transcc I have written.

As for building 32/64bit with Visual Studio and XCode. All it should need are a few more build configurations with the appropriate naming to the GLFW solution and casedConfig appended correctly in transcc. Hence one simple parameter pass as an option in the transcc command line and the appropriate modifications to the builders source code, then all should be good. I will look at doing this as well at some point over Christmas.


JaviCervera(Posted 2015) [#24]
> I'd wait a while before attempting that! Turns out interfacing with C++ will be quite tricky...

There is a C-based procedural interface, which is what I am going to bridge to Monkey2.


EdzUp MkII(Posted 2015) [#25]
Yup C++ is quite a tricky beast to be honest, MonkeyX2 is looking mighty fine :).

Compiling 64bit builds of monkeyX games and apps will be a godsend to loads and would future proof monkeyX for years to come :D


dawlane(Posted 2015) [#26]
Ok. I got to have a play at implementing a few things and I think I have got it working correctly without any problems.
What have I done.
The Visual Studio 2013 project files for GLFW have been modified to allow 32/64bit compilation (I've thrown in a few 2015 as well). These should work for 64bit compilation if you are running a 64bit version of Windows. I haven't gone into the compiler side for Window 32bit and frankly I think that Visual Studio over complicates things. A phrase containing these words springs to mind when I use VS if you are not familiar with it .... "yourself","rope","enough","hang" and "to".

I have added four GLFW configure directives and a parameter option to transcc (-msize=) for the same effect as the last one in the list, but use of f GLFW_GCC_CC/LD_OPTS and GLFW_DESKTOP_MSIZE will override it.
#GLFW_GCC_CC_OPTS
#GLFW_GCC_LD_OPTS
#GLFW_GCC_LDLIBS_OPTS
#GLFW_DESKTOP_MSIZE

The first three are Linux and Windows specific to GCC/MinGW. And are for use by advanced users, but there is a simple check for ambiguous use of -m32 and -m64 in CC and LD.
The last is specific to Linux and Windows using both MinGW and Visual Studio, using a compiler that can generate 32/64bit code.
I have not looked into OSX or into passing options to MSBuild (yet). But I think the names GLFW_XCODE_OPT and GFLW_VSTUDIO_OPT would be appropriate.

I have still left the ability to use a MinGW cross compiler intact.

https://www.dropbox.com/s/89d0l51uuu6z734/monkey-86c-dawlane-mod-beta.zip?dl=0

At some point I will have to get the sources onto Github.
EDIT: https://github.com/dawlane/dawlane-monkeyx-custom


Soap(Posted 2016) [#27]
Mark, could you update the WinRT target to VS 2015 too?