Avoid copying all data to build folder each time?
Monkey Forums/Monkey Programming/Avoid copying all data to build folder each time?
| ||
| Is there a way to stop Monkey from copying all the data folder contents to the build folder every time I hit compile? It's beginning to slow down my compile process now as I've got >1000 images (yes I know they should be in a texture atlas). Also I'm paranoid about it wearing out my SSD drive. Is there some obvious setting for only overwrite if new or something I can use? Thx. I should add that there's a big assumption on my part that this is what is causing the slow down, but it's based on some threads I read quite a while ago and something I heard from another monkey dev about it being slow as the project data grows. So I could be totally wrong about this and something else is slowing down. |
| ||
| In one way or the other you would have file access. Being it copying the files or first compare them and only copy them if they have a new time stamp. Btw. I never heard about an option to just build the code. |
| ||
| Aw shame. |
| ||
| Monkey's build mechanism is generally dumb about avoiding redoing bits that don't need it. The whole thing is clunky and needs an overhaul. I wouldn't worry too much about your SSD though. These days all SSDs are rated for tens of gigabytes of data writes a day for years. |
| ||
| The whole thing is clunky and needs an overhaul. +1. won't happen until after mojo-x comes out. i would opt for a light interpreter for build scripts, so we wouldn't need to re-build trans every time. somethign like mini-c, but with a monkey syntax. Is there a way to stop Monkey from copying all the data folder contents to the build folder every time I hit compile? there may be a way for DESKTOP targets. you can point your asset loaders to a folder outside of the .data folder. i think if you use a hard path "c:/assets/image.jpg" monkey should pick it up, and keep your .data folder empty. |
| ||
| Adam is right but not only desktop targets as LoadImage support loading from a file for the following targets: Android, iOS, Win8, Glfw. For iOS and Android you could even load images from the internet which is also supported in html5. Muddy Shoes, I won't be sure about that, monkey does some avoiding redoing which is why we sometimes need to delete the project files every now and then in order to get them to work with the latest version of monkey. |
| ||
| The fact that Monkey is incapable of detecting an old build directory and appropriately regenerating/replacing the base files is part of the clunkiness. I certainly wouldn't call it smart behaviour. Monkey doesn't appear to do any checking for changes in the game/app project files as can be seen if you run something without making any changes. A smart build process would just launch it again, Monkey rebuilds the whole thing. |
| ||
| you know, this seems like an easy fix. We don't care about the security of the implementation, so we could just accept a newer "last modified" date from the filesystem as the criteria for re-copying, no hashing or crc necessary.... |
| ||
| You could modify transcc/builder.monkey and include some sort of "ignore" blacklist of files to not copy. |
| ||
| Bumping this in case Mark hasn't seen it. I'm compiling 100s times a day a large commercial project and this is driving me a bit nuts. I've got a very fast PC and just upgraded my SSD to an even faster one but the build times are still quite slow especially compare to BlitzMax build times which are lightening fast. Would really love it if a simple last modified date check could be added for Desktop apps anyway (as that's the one I think most people use to code and test their apps with before making final builds for other targets). It could even be controlled with a compiler flag (or option in the IDE) if people were worried about it being always on. |
| ||
| This should be relatively easy to do, will have a look into it soon. |
| ||
| (as that's the one I think most people use to code and test their apps with before making final builds for other targets). I would think the HTML5 target is the main one people test/develop with as it is super quick to compile and test :) |
| ||
| I use the HTML5 target to to test as like therevills says it's super quick (FastHTML5 posted in a thread on here). However even using the "fast" target with the alpha and color changes in my game I have started to use GLFW as it runs like a dog in HTML5 now the project has grown a fair bit. |
| ||
| Yeah, I only use GLFW when I need the full debug. That and Android are the slowest compile. |
| ||
| Yahfree wrote: You could modify transcc/builder.monkey and include some sort of "ignore" blacklist of files to not copy. I have changed builder.monkey v1.60 to fix subject. At the begin of method CreateDataDir comment DeleteDir to avoid delete all data folder Method CreateDataDir:Void( dir:String ) dir=RealPath( dir ) 'DeleteDir dir,True CreateDir dir Add new function Function CopyFileUpdate:Void(p:String, t:string)
If FileTime(p) > FileTime(t) Or FileSize(p) <> FileSize(t)
CopyFile p,t
Endif
End Functionand replace in the same method calls CopyFile() function to CopyFileUpdate() Build it, move to Monkey/bin folder, rename it to transcc_winnt.exe (keep old file of course) Test it with extra jpg files in the data folder - it's fast! I use MS VC++2010. Build file is correct for test HTML5. It fails when I try build new version of transcc (GLFW target). |
| ||
| Ok, this is now in v78b Nice work nikoniko, mine's about the same! I've also added a 'delete dead data' pass, where it goes through the stuff in the target project data dir and makes sure it's meant to be there, deleting it if not. This means deleting data from your app .data dir will also remove it from the target .data dir. Ditto, changing data file type filters will update target data dir. Should this be optional? It shouldn't add any major overhead, and without it your data dir could end up with junk in it over time... |
| ||
| marksibly wrote: This means deleting data from your app .data dir will also remove it from the target .data dir. Great! I thought about synchro option when going to sleep :) Optional? I am not sure. It's real build speed up feature. Maybe as option to turn off it. |
| ||
| (havent tested it yet) but is there an argument for using FileTime( src )<>FileTime( dst ), e.g. when putting an older version of a file back in the data folder you still want it copied across. Also deals with any weird time differences between machines. Bit risky to only copy 'newer' files (?). Or did I miss something - I assume FileTime is modified time. Should be a handy feature though. |
| ||
| @therevills I use GLFW mainly because my games compiled as HTML 5 just run like crap due to Alpha blending and other effects, plus I often need Debug. Oh and I hate the sound delay too. |
| ||
| @GA - What browser do you use? With the latest Chrome version I don't often have an issue with FPS. |
| ||
| Firefox. Perhaps that's why. |
   |