building custom targets
Monkey Forums/Monkey Programming/building custom targets
| ||
| I'm sure this information is somewhere obvious and I'm just blind, but I couldn't find anything documented. What's the standard process for creating targets? I want to play a bit with doing a 3DS target, and obviously I could just load in custom modules that wrap devkitpro, tell trans to build a cpptool, and compile the main.c myself, but I'm sure that's not the "right" way to do it. Can you add targets without hacking into trans? |
| ||
| Oh boy, I've done it again. Have fun reading this mess of a response... I'll be sure to edit it to death after I've recovered from passing out. Maybe I should write a book or something about this garbage... *Grumble* *Grumble* Anyway, yeah, you can make a new target without modifying 'transcc', but you can't make a new translator or builder without rebuilding it (Which is the simple part). You'll probably need to get your hands dirty with the source, though. That doesn't mean you'll have to actually modify 'trans', but you probably need to modify the "driver". This idea is similar to my approach with my WebCC project, where the builders were the only major things changed (In Monkey) from stock 'transcc'. In that case I needed to write a bootstrapping system based on 'iframes' and JS output. Don't get me started on the whole "port 'os' to web browsers" thing. In your case, you just need to modify the "C++ Tool" target slightly. I'll get into the other details, specifically a glossary of how Monkey's compiler works, in a minute. First, let's take a step back for a moment. Monkey's core language features are portable and implemented in several native languages for each target. The key part here is that we can leverage the "C++ Tool" target and related systems, including the support for GCC. You aren't going to be making a plain C target from scratch, that's ridiculous. You can already write in C++ and build using devkitARM. Interoperability is near perfect, and works out of the box. In the case of Monkey's source, there's a few hiccups with the occasional bit of STL (Unless that's also working now?), but it's nothing you can't rewrite as standard C. To be exact, several portions of the C standard library just plain work, even 'stdout' going straight to the screen. But of course I'm talking about the core language running under 3DS homebrew. Handling graphics is a huge beast that I doubt has been solved well since the last time I looked at ctrulib. I know there was some initial work on executing shaders and handling GPU memory, but most graphics seemed to be done in software. Honestly, I didn't look well enough into it. When I tried out 3DS homebrew 6 months back, I wrote using mostly portable C++. More accurately, glorified C. With that in mind, even the POSIX/BSD socket API works (Well, most of it). Of course, this means 'brl.socket' could technically work, but there's other reasons why it wouldn't. I don't know how far the C rabbit hole goes, though. I do know about 'stdio' and at least part of 'stdlib' working. With the right supporting commands, sockets and other services should work, too. When I was messing with it, I didn't do much else, because there really wasn't a lot that could be done. The most of the rest of it seemed to be poorly documented (At the time, at least). I can say right now that making a "C++ Tool" port is theoretically doable. Graphics I'm not too sure about, but if shaders are available, or you have some kind of software renderer, then Mojo could work. Likewise, if someone gets at least fixed-function OpenGL working, the job's much simpler. As for wrapping ctrulib, you've basically hit a road block. Monkey 1 doesn't explicitly work with pointers or structures. This means you'll be writing some rather heavy wrappers. If you can manage to get OpenGL or Mojo working (Even in software), then it's definitely target material, even for hobby projects. Considering the requirements, you're dealing with a tall order, but it'd definitely be a cool project. ---------------------------------------------------------------------- Okay, so as promised, I'll get the basics out of the way with Monkey's target system. Oh, you're in for a treat this time: DISCLAIMER: This covers the basics of editing the 'transcc' frontend, and about as cleanly as you can. You'll need a supported version of GCC/MinGW to build this. Check out this page from the docs (Actually, your local version under your IDE's help may be better). You won't actually have to use GCC or MinGW directly, but Monkey needs it to build the translated output. Obviously, this is not the same compiler as devkitARM provides. ---------------------------------------------------------------------- Basically, you've got 'transcc', the "driver", which loads from the platform's configuration file. This is used for your installed SDKs and the like. After this, it enumerates a 'StringMap' of builders, the strings representing their names, of course. Each builder is implemented in Monkey and compiled into 'transcc', so you need to rebuild it to add another one. So, what is a "builder"? Essentially, they're small classes that inherit a common class which does the heavy lifting. That class uses a translator you supply in your own builder using a field of the parent class. This, along with setting the language-extension, and any other prerequisites, is done in 'Begin'. I'll be honest, it's kind of thrown together. Anyway, the point here is that the builders are the core of the tool. They start each phase; parsing, which breaks down your syntax, "semanting", which figures out what things do, and makes sure your code actually works, and finally translating. The translator basically gets all of the loaded symbols/objects/context from the first two phases, including native files via 'externs'. The source is translated, the native source is pushed in with it, and the final result is a single file holding your translated source code. Yeah, this is where the build process is the least desirable. It's not multi-file, which is annoying, but hey, that's what Monkey 2 is for. In your case, you can use C++, so you're golden on the translator front. Semantics and parsing are handled in Monkey (Implemented in the 'trans' module), and you're already using GCC. So, from a target perspective, you've got little to do. Even then, we've got some really nice global variables and other goodies that make this process more dynamic. For example, the host OS info we get from 'os', and the 'ENV_CONFIG' variable that tells us what kind of build it is. That's really the cool thing about this, you're not hacking anything, you're just extending it. This is where you realize that Mark Sibly made a pretty sweet compiler, but also that he's been doing to for years, even if the end of the factory is a bit lacking. Again, Monkey 2's around the corner soon enough, so maybe someone will give the build system a needed makeover for Monkey 1 as well. Honestly, if your make-file is anything like mine, then you can just add any extra flags you want and use 'make' like the examples do. It's the usual shtick, something like "make -f filename", or just "make" if you're clever. You'll be passing a string with this kind of command at the end of the 'MakeTarget' method. Just remember to keep that check against 'opt_run', it's kind of important. Okay, so we're probably going to make a custom builder that's basically just the "C++ Tool" one, how do we add it to the source? Before I continue, I should probably mention that you should already be copying in this case. Keep your custom files in a different directory and mimic the file structure. This way you can use git or whatever you prefer to manage the source. That's a better option than having to clean this up when you want to update. Okay, back to the fun stuff. First, you'll want to give your file a name and place it in "src/transcc/builders", name the module whatever you please. From here, open the "builders.monkey" and add an import to your new module at the top of the file. Next, go down to the 'Builders' function and add your own line, like so (Formatted for consistency, not clarity; you've been warned): Something to that effect, anyway. The class you're instantiating is the one in your module from earlier, so it's your call. Alright, so what have we got next, then? In 'transcc'? Not much, actually. Assuming all you were looking to do was tweak the builder, that's about all you need to do. Now we need to hit build and wait for the ... Finished? Great, now you need to "install" it. Move the executable you made (Found in the build folder for 'transcc') into your "bin" Monkey installation's folder. After it's been moved, rename your old "transcc_blah" binary to something else, or just move it, it doesn't really matter. (Tip: When I say "blah", I'm really just being lazy. This is your platform's name, and it'll match your new executable; winnt, macos, linux, etc.) When you're done with that, rename your new binary from "main_blah" to "transcc_blah", and you're good. Not exactly rocket science, but you'd be surprised how many people can't do this. Next, we need to take a stop at the "targets" folder, located in the root of your Monkey directory. Copy the target folder you want to base yours off of. In this case, I imagine you'll be using "cpptool". With your new folder, rename it to whatever you want. (Tip: It might be a good idea to copy this later, like you probably did for your builder's source code) Before we get down to brass tacks, OPEN UP "TARGET.MONKEY" in your favorite text editor (Ted for example) and take a look at those variables. The first one ('TARGET_NAME') should be pretty obvious, it's what we've been calling the damn thing for this novella of a post. That's both what'll pop up in your IDE, and what you can input on the command-line. Since we're The second ('TARGET_SYSTEM') is a bit of a mystery until you start thinking about it. This is what you'll refer to the target as from within the actual language. You know that thing we've barely used this whole post? That's my fault, isn't it... Anyway, name this whatever you want, but it's preferred to keep it one "word" easy to write, and ideally no spaces. Underscores could work, I guess. The third ('TARGET_BUILDER') is very familiar by now. Remember that 'StringMap' from earlier? This is how you access it. In my previous example, I used "3ds", but it just needs to be the same name. This principle applies to any target, just like the other info. So, are we done yet? Nope, now we've got the actual files to deal with. In your new target's folder, you'll find a folder called "modules", unless you're getting build errors, don't bother messing with this. That folder holds a file called "monkeytarget.monkey", and it's related to setting up an entry-point. This can also be used to add to programs built with the target. This is usually used for default imports and the like. Less ideally, you could use this for preprocessor definitions, but we'll get to those. Basically, it's free-range for any (Monkey or native) source code you want to bundle with builds; just import and go. This actually applies to files in the data folder, now that I think about it. But wait, there's more, let's go to our "template" folder. Here you'll be in familiar territory. You've first got your "CONFIG.MONKEY" file. If you've ever configured a target, you know what this is. Add any preprocessor variables you want, and they'll be defined in the built Monkey code. Finally, we've made it to the end. What you'll find here are a couple of default functions, one being 'main', which is the program entry point. There's also some weird comments, these are used back our builder to "inject" the translator output (And native files). Finally, you've also got an 'include' to "Main.h", which is in the same folder. Keep in mind the contents of this file, because it dictates the default imports of any builds made with this target. This also means you'll need to strip down If you've followed that mess, you've officially created your own target, start up your IDE, and your new 3DS target should appear. (Note: You don't have to rebuild when you want to make a target based on an existing builder) There's plenty I wasn't able to cover, but this should at least cover the initial setup. Now if you'll excuse me, I'm going to go pass out. Oh yeah, I forgot: Make sure to look into those STL dependencies. There aren't many, but I can think of at least one use of std::vector for string conversion. |
| ||
| Wow, you're the best person ever. You deserve a purple heart for that. I like how over the course of the post your sarcasm becomes less and less veiled, it makes it pretty easy to see the progression of your emotional state. Also, thanks for all the modules you make, I've used several of them for random projects (although tbh the interdependent regal system is a massive pain, dunno what else you'd do though). Shoot, I had forgotten that trans is written in Monkey, that makes it a tad more annoying. In any case, thankfully I'm not foolish (or masochistic) enough to attempt to make a complete target for every builtin module (read: this will probably never be publicly released), so the hard ones like 'os' can die for all I care. I'm just going to wrap ctrulib's graphics stuff, probably under a mojo1 interface since mojo2 exposes a lot of the underlying OpenGL implementation and so would obviously be a nightmare to get working. Even then, though, you have two screens with different res and mouse/touch coordinates only on 1, so obviously it isn't a good choice for mojo and I may just write a completely different module depending on how I feel. Luckily, the editing trans parts should be pretty easy. I'm always a bit unclear on what is and isn't open source - IIRC, all of trans is open but mojo/brl for some targets isn't? Anyway, from what you describe, I can create another builder without touching the translation process and mostly just use it to load different dependencies, add different make/etc files to the output, and run a different compile command at the end. Sweet. It's fortunate that it's designed to be so modular. Welp, in that case I guess I'll start trying to break stuff. Since you expressed some interest I will say that it's worth looking at the 3ds scene again, it's advanced a TON in the past several months. I know people are working on opengl, sfml, and other libraries but not a lot of them seem to be stable yet. Still, I'm only getting in to it now because it finally feels like you can at least get a FOOTHOLD into stuff recently. I think ctrulib has been improved a fair bit too, but I'm pretty new to it all so not 100% sure. The api feels like a terrible cross of opengl and directx paradigms, but that's why you wrap it before you actually work on anything ;) |
| ||
| Immute, I shit you not when ever I see your name in the heading of a reply to a question my first thought is always, "Right time to grab a coffe" because I know its gona be a read :) That's not a bad thing either, being Dyslexic your wall's of text take me time to get through hence the drink, may I ask that you try formatting a little more ? I know the forum sucks for it and you don't have a lot of formatting options but it does help people like me. |
| ||
| @ImmutableOctet(SKNG): I gave up after Oh boy. :-) To make it more comfortable to read and save your fingers the next time. I would suggest putting it all into a pdf file with Chapters and an index. And placing the the link in the tutorials, as there is bound to be somebody that will ask this question next time. ;-) |
| ||
| i thought it was a pretty logical flow to follow, personally. this should definitely be linked somewhere though |
| ||
| So much info, and no one gave him a star |
| ||
| I keep forgetting about the star's its so subtle I think most people don't really notice it at all which is a shame, and I rectified his lack of well overdue star. |
| ||
| Just reading up a little I second the motion to move the information Immute has thrown in this topic to some where like the tutorial section I already have a few of his posts bookmarked for later reading so having them in a good place would make them easier to find in the future, we all know how lame this forum's search functions are. |
| ||
| Immute, I shit you not when ever I see your name in the heading of a reply to a question my first thought is always, "Right time to grab a coffe" because I know its gona be a read :) Pretty much...this :). |
| ||
| I really appreciate the comments, guys. Although, I didn't realize I was coffee worthy. I don't know if I'll end up reworking this into a tutorial, though. If someone else wants to, go ahead. @Sicilica: That wasn't just my emotional state; I was up for nearly 30 hours at the time. Needless to say, I was running on fumes. Anyway, I just want to clarify how the license works. Everything the light touches is Basically, it's like this: * Mojo 2 isn't open source at all. * The HTML5 and GLFW backends for Mojo 1 are open source, as well as the Monkey portions of it. * Everything else is open source unless otherwise stated. This includes the actual targets. I wrote about this in this post (Module disclaimer), if you're interested. It's basically the same summary, though. TL;DR: If it's in here, it's open source. The license is here. It basically says to not misrepresent your modifications as the original source code. So, a modified version of Monkey can't be misrepresented as the original version. You are also not obligated to credit Mark or Blitz Research in a product based on Monkey, although it is appreciated. Just keep in mind that any product using modified software from Monkey must be marked as such. Considering it's the zlib/libpng license, you're basically free to do what you want. Although, a custom target that's dominantly based on a default target should probably have the license redistributed with it. For example, my 'jstool' target has the license with it. In general, you aren't going to have any issues if you just distribute the license with it. That goes for a lot of licenses, but I'm no lawyer. Just don't mess with Mojo, and you should be good. |
   |