Build system and import
Community Forums/Monkey2 Talk/Build system and import
| ||
| Regarding: http://monkey2.monkey-x.com/language-reference/#the-build-system-and-import Could you please add header extension .hpp to Global and Local #Import? I just found myself renaming many .hpp to .h because .hpp didn’t work - this includes replacing all .hpp with .h within this files. .hpp is _very_ common for C++ headers. .framework files are only in the Global table. I need to #Import a local .framework package, too. The .framework comes with a lib I want to import, it’s not in the system path. Somehow I’m missing .dylib support in the tables? |
| ||
| Ok, can do. Will add ".hh" and ".cc" too as I remember these from way back. It's a bit nasty hardcoding these in, but it'll do for now. The only one I'm not clear on is .dylib - my guess is to pass to the linker AND copy it to the app exe dir if it's local. Does that sound right? |
| ||
| [edit] I guess local frameworks need to be copied too? |
| ||
| .dylib: Yes - pass to linker and copy (I think to appdir) Currently .dylib is copied to Resources. Local .framework: yes, copy too. |
| ||
| So just stuck dylibs and frameworks in the Contents/MacOS dir along with the exe? Have you got something working with these in a 'release' app? The apple docs seem to suggest an env var is needed to locate these. |
| ||
| (it's sticking them in 'resources' currently because it doesn't recognize them so assumes they're assets). |
| ||
| Looks like you have to additionally specify "-rpath" option for the ld linker: https://developer.apple.com/library/mac/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/RunpathDependentLibraries.html Probably "-rpath @loader_path/“ if the .dylib and .framework are in .app/Contents/MacOS/ Probably "-rpath @loader_path/../libs/“ if the .dylib and .framework are in .app/Contents/libs/ Renamed the .dylib to .lib to get linking working. When executing the result it says ‚image not found‘ and "otool -L myEXE“ shows: @rpath/lib1.dylib @rpath/lib2.dylib Using the „-rpath“ option for ld we add a directory to the rpath-search-path. See also http://www.dribin.org/dave/blog/archives/2009/11/15/rpath/ The .dylib path can also be changed using install_name_tool: install_name_tool -change '@rpath/lib1.dylib' '@loader_path/lib1.dylib‘ myEXE install_name_tool -change '@rpath/lib1.dylib' '@loader_path/../Libraries/lib1.dylib‘ myEXE Did this by hand and it works. EDIT: By running install_name_tool we can also add (several) rpath to the executable: install_name_tool -add_rpath @loader_path/ myEXE install_name_tool -add_rpath @loader_path/../Libraries/ myEXE install_name_tool -add_rpath @loader_path/../Frameworks/ myEXE Works fine. It is best to copy local .frameworks into TheApp.app/Contents/Frameworks/ It’s the default directory for Frameworks and most .frameworks expect to stay at this location. |
| ||
| Got SFML running by doing the following steps: 1.) Copy required .framework files into Test01.app/Contents/Frameworks/ 2.) Copy required .dylib into Test01.app/Contents/MacOS/ 3.) install_name_tool -add_rpath @loader_path/ Test01.app/Contents/MacOS/Test01 Hope we can automate this process using MX2 build system support for .dylib and .framework files. :) BTW: MX2 rocks! :D |
| ||
| Nice work! I've done a bit on this today - just pushed the results. You'll need to updatemx2cc though. Anyway, #Import now recognizes more filetypes, including local dylibs and frameworks. See language docs at monkey2 site. I also added a hook (so much nicer than 'hack' - thanks WP!) that allows you to select where dylib and framework files end up when they're copied into the app, relative to the app exe, eg: 'These relative to app exe MX2_APP_DIR_DYLIB= 'ie: dylibs go in app Contents/MacOS dir MX2_APP_DIR_FRAMEWORK=../Frameworks 'ie: frameworks go in app Contents/Frameworks dir I also tweaked the default LD_OPTS so rpath is set to find stuff in these dirs: MX2_LD_OPTS_DESKTOP=-Wl,-rpath,@executable_path -Wl,-rpath,@... This does depend on the 'rpath' inside the lib being set correctly too, but I looked at a few libs and they seemed OK. Anyway, with this stuff in I can now build/run an app this imports libclang.dylib and QtCore.framework! Doesn't actually do anything with them, but getting it linking/running was good enough for me. A quick note on local framework imports: These don't actually appear to be supported by the macos linker, eg: you can't just go -framework "some/path/wotzit.framework". So mx2cc 'fakes it' by adding a 'system framework path' and then doing a normal framework import. It still copies the framework into the app though. |
| ||
| Thanks, Mark! Will test tomorrow. :) |
| ||
| Found a small issue with this, Mark: Frameworks include often (always?) symbolic links, for example: FLAC.framework Folder |> Versions Folder | |> A Folder | | |> Resources Folder | | |> FLAC Binary Library | |> Current Symbolic Link | |> Resources -> sub-symbolic link | |> FLAC -> sub-symbolic link |> Resources Symbolic Link |> FLAC Symbolic Link MX2 does not copy the symbolic links as symbolic links, it copies the real files behind the symbolic links. The result is, in my compiled .app all .frameworks are 3 times bigger - because the .framework now contains the library "FLAC" 3 times instead 1 time only + 2 symbolic links. My SFML .app with all required frameworks is 21MB instead 7MB because of that behavior. :) Something similar happens when including .dylib: #Import "lib/libsfml-audio.2.3.2.dylib" ' real dylib #Import "lib/libsfml-graphics.2.3.2.dylib" ' real dylib #Import "lib/libsfml-network.2.3.2.dylib" ' real dylib #Import "lib/libsfml-system.2.3.2.dylib" ' real dylib #Import "lib/libsfml-window.2.3.2.dylib" ' real dylib #Import "lib/libsfml-audio.2.3.dylib" ' symbolic link #Import "lib/libsfml-graphics.2.3.dylib" ' symbolic link #Import "lib/libsfml-network.2.3.dylib" ' symbolic link #Import "lib/libsfml-system.2.3.dylib" ' symbolic link #Import "lib/libsfml-window.2.3.dylib" ' symbolic link #Import "lib/libsfml-audio.dylib" ' symbolic link #Import "lib/libsfml-graphics.dylib" ' symbolic link #Import "lib/libsfml-network.dylib" ' symbolic link #Import "lib/libsfml-system.dylib" ' symbolic link #Import "lib/libsfml-window.dylib" ' symbolic link Could you change the copy-operation for .framework and .dylib files (or probably all files that get copied), so symbolic links are still symbolic links after the copy? |
| ||
Try adding the COPYFILE_NOFOLLOW flag to line 134 of modules/std/misc/native/filesystem.cpp, eg:return copyfile( bbTString( srcPath ),bbTString( dstPath ),0,COPYFILE_DATA|COPYFILE_NOFOLLOW )>=0; ...and updatemods.sh If that doesn't work, you could try one of the other flags here: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/copyfile.3.html If all else fails, I'll have a look at it a bit later. |
| ||
| Nope. COPYFILE_NOFOLLOW: MX2CC V0.003 ***** Building app '/Users/danilo/Monkey/monkey2/modules/SFML/SFML.monkey2' ***** Parsing... Semanting... Translating... Compiling.... Linking /Users/danilo/Monkey/monkey2/modules/SFML/SFML.buildv003/desktop_debug_macos/SFML.app/Contents/MacOS/SFML Build error: Failed to copy '/Users/danilo/Monkey/monkey2/modules/SFML/lib/FLAC.framework' to '/Users/danilo/Monkey/monkey2/modules/SFML/SFML.buildv003/desktop_debug_macos/SFML.app/Contents/Frameworks/FLAC.framework' ***** Fatal mx2cc error ***** Build error. Tried some other flags and combinations: ./rebuildall.sh ***** Rebuilding modules ***** ./rebuildmods.sh: fork: Resource temporarily unavailable ***** Rebuilding mx2cc ***** ./rebuildmx2cc.sh: fork: Resource temporarily unavailable ./makedocs.sh: fork: Resource temporarily unavailable Danilos-Mac-Pro:src danilo$ ./updatemods.sh ***** Updating modules ***** MX2CC V0.003 ***** Making module 'monkey' ***** Parsing... Semanting... Translating... Compiling.... Build error: System command 'g++ -std=c++11 -g -std=c++11 -Wno-deprecated-declarations -Wno-tautological-pointer-compare -Wno-undefined-bool-conversion -Wno-int-to-void-pointer-cast -Wno-inconsistent-missing-override -Wno-logical-op-parentheses -O3 -DNDEBUG -Wno-int-to-pointer-cast -Wno-parentheses-equality -Wno-comment -I"/Users/danilo/Monkey/monkey2/modules/monkey/native" -c -o "/Users/danilo/Monkey/monkey2/modules/monkey/monkey.buildv003/build_cache/desktop_release_macos/_1_1_1native_2bbtypes.cpp.o" "/Users/danilo/Monkey/monkey2/modules/monkey/native/bbtypes.cpp"' failed. g++ -std=c++11 -g -std=c++11 -Wno-deprecated-declarations -Wno-tautological-pointer-compare -Wno-undefined-bool-conversion -Wno-int-to-void-pointer-cast -Wno-inconsistent-missing-override -Wno-logical-op-parentheses -O3 -DNDEBUG -Wno-int-to-pointer-cast -Wno-parentheses-equality -Wno-comment -I"/Users/danilo/Monkey/monkey2/modules/monkey/native" -c -o "/Users/danilo/Monkey/monkey2/modules/monkey/monkey.buildv003/build_cache/desktop_release_macos/_1_1_1native_2bbtypes.cpp.o" "/Users/danilo/Monkey/monkey2/modules/monkey/native/bbtypes.cpp" clang: error: unable to execute command: posix_spawn failed: Resource temporarily unavailable Looks like I have to re-install MX2 v0.003 now. :) |
| ||
| Mark, "COPYFILE_ALL | COPYFILE_NOFOLLOW“ works nice with files, like .dylib. Also with symbolic links to files. For directories that include symlinks and aliases, like .framework, CopyFile does not work correctly. The simplest solution for the MX2 compiler for copying .framework and other directories into the .app: If GetFileType( source ) =( FileType.File Or FileType.SymLink ) CopyFile( source, destination ) ' .dylib and other real files, including symbolic links ElseIf GetFileType( source ) = FileType.Directory System( "cp -R " + "~q" + source + "~q ~q" + destination + "~q" ) ' .framework amd other directories Else ' Error EndIf „cp -R source dest“ is the best way I found so far, for copying .framework correctly: "cp -R ./FLAC.framework ./f.framework" All symlinks/aliases inside still intact and dest has same file size as source, after copying. EDIT: Would require adding "Case S_IFLNK“ to std.filesystem.GetFileType() |
| ||
Yep, came up with something similar 'for now' - paste this into the builder.monkey2 CreateApp() method just before the 'If Not CopyAll...' line:#If __HOSTOS__="macos" If ExtractExt( src ).ToLower()=".framework" CreateDir( ExtractDir( dst ) ) If Not Exec( "rm -f -R "+dst ) Throw New BuildEx( "rm failed" ) If Not Exec( "cp -f -R "+src+" "+dst ) Throw New BuildEx( "cp failed" ) Continue Endif #Endif Had to add to 'rm' or it only worked the first time. The filesystem stuff need link support added in a few places. Shouldn't be a big deal and I'll get onto it later but this is probably the best solution for now. |
| ||
| Thanks Mark, works fine now. :) |
| ||
| No problem...I think this is shaping up to be a pretty cool little build system! Really need to cleanup builder.monkey2 at some point though... |
| ||
| Yes, it’s nice. When adding COPYFILE_NOFOLLOW to copyfile (mentioned above), it also copies symlinks to .dylib correctly. ;) |
   |