Module repository categories
Monkey Forums/Monkey Programming/Module repository categories
| ||
| What do you think would be a sensible way to organize module categories on a centralized Monkey modules "assets" store? - Serialization (xml, json, etc.) - Graphics (textured polyes etc...?) - Social Media (facemonkey, twitter...) - Physics - .... ? I'm working on a module repository for Monkey, which will be web-based and free to use to all Monkey users (it won't require the usage of Jungle Ide). Also, working on a commandline tool to handle automated module updates by using GIT and/or Mercurial, so any IDE developer could write a front end for it, and make it work with the module repository. I'm now working on the "conceptualization" of the module repository site, and I think it would be nice to have a decent organization of possible module categories. |
| ||
| I think tags might be another option. That way a module can easy be organized under one or more things. But I love the idea of a module repository because there's lots of good code scattered about that never sees the light of day. gasmonso |
| ||
| Yes, I'm building it in a non exclusive way. A single module can appear in as many categories as it requires, and also, it could be done in parallel to a tags system (I'm basing it all in the wordPress apis, so it's rather easy to get this up and running reasonably fast and with a very secure system). Also, it will be "searchable". |
| ||
| Great idea :) Sounds promising :) |
| ||
| How are you planning to use version control systems here? I hope you're not going to repeat what Mark did. |
| ||
| How are you planning to use version control systems here? I hope you're not going to repeat what Mark did. My idea is to provide a website where you can sign up for free and you can register your own modules. When you register a module, you'll have to provide the checkout command used to clone the module locally (git or hg will be required). You'll be able to also register dependencies of other modules. That's the first part of the plan. Then, after this is done, I'll do an open source assets application (not sure if I'll do it on Java or directly in Monkey). My idea is that this application will integrate a browser where you can navigate through all modules in the site and install any of them with a single click. This application will check for the availability of the required version control systems on start, and allow an automated install of any missing ones. Also, this application should allow for detection of new versions and should give the option to update any installed module that has been updated in its associated repository. That's how I'm designing it, but if you have any sensible suggestion to improve this idea, feel free to make it. This is not exactly the way Mark was doing it, but close to it in the sense that most module developers are already using git or mercurial, so allowing them to distribute easily their modules by just registering them on a site, it's a nice plus for the mod developers and users. Also, the site will allow for distribution of commercial modules given they use authenticated git or mercurial repositories. Also, this site will allow the integration of donate buttons on free open source module pages. On a second "round" I would like to integrate too artist packages in the whole system, and promote the site on artists sites, so they can expose their work and we (coders) and them (artists) can get mutual benefits of this system, if it gets adopted by a sufficient users base. This app will be bundled with Jungle Ide, but will be redistributable under a MIT license, so it could be easily integrated in TED or any other Monkey IDE. |
| ||
| This all sounds good, my only real issue is that it's yet another thing I need to sign up to, I think others may feel the same way, these days I sign up to as little as possible on the internet only when there are no other options will i register on something and most of the time even then I use some crap throw away e-mail which kinda makes the whole process retarded in the first place. Would it all be possible for you to talk with Mark and see if some form of api could be setup to handle authentication ? You could for example has an API brl side that deals with remote authentication and your site checks that for login access, that way anyone with a monkey account, free or paid will instantly be able to login and use your service without any registration or signup being required. Another option would be for brl to allow the creation of api keys, with these keys anyone could then check if the key belongs to a registered user or not and then authenticate based on that result, this would be an easier solution brl could simply add a single additional field to every user and then populate it with a salted md5 of some of their account data like name + email or something, then this generated KEY would be placed in the visible section of the users account page. On your module site you could simply ask for a display name, and then the API key of an exisiting monkey user, these would be linked and stored your end for future logins, during each login your site would throw the API key at brl and say hay is this valid, and then act based on the result. just a thought. |
| ||
| This all sounds good, my only real issue is that it's yet another thing I need to sign up to, I think others may feel the same way, these days I sign up to as little as possible on the internet only when there are no other options will i register on something and most of the time even then I use some crap throw away e-mail which kinda makes the whole process retarded in the first place. Well, maybe it's not clear, but you will only require registration in order to register your own modules, but usage of the site and service will not require any kind of registration. so, if you're a module user, you won't need to register anything.That said, allowing cross-site authentication opens the door to all kind of security issues I would love to avoid. I guess that's the little price to pay. If it gets used ad its helpful, I plan on open sourcing the whole thing, so if Mark or Simonh or anyone wants to add an option to "connect" accounts, I'm all for it, but it can increase a lot the complexity of the whole system, and the maintenance. Also, it we could provide "social login" so people could create a user by using their facebook account (if they want, optionally) so they do not have to create a new password, authenticate, etc. Just log-in and it's done. But I'm just not sure if this is better or not. |
| ||
| I went through many of the issues when Mark released ModMan: http://www.monkey-x.com/Community/posts.php?topic=5399&page=1 It pretty much boils down to "package managers have no business stipulating development processes unrelated to the construction of the package". Take a look at how other languages deal with package management: http://en.wikipedia.org/wiki/List_of_software_package_management_systems#Application-level_package_managers. I think you'll find few to none demand use of specific version control methods and they pretty much all allow package construction from plain files and scripts instead. |
| ||
| @muddy_shoes: Yes, I know that and I've used Maven before and it is all great but it would be extra work to build something like this for Monkey. Ideally, I would do an application where you could define which module you want to deploy, and it detects dependencies, builds a zip-like package with version info, etc. allows you to upload it to a server, etc etc. Nice, and I'm not saying it should not come but, I think it would be a lot wiser to centralize, what we already have. Adding an additional system so people can choose git, hg or "monkeymodman" (to give it a name) is something that could be done later, in addition to a simpler hg and git based system to begin with. I'll have to do something like what you're describing for the "artists" part of it, just that I don't think it should be a priority given that there's actually lots of modules spread over the net that are already being deployed on github, googlecode, etc. |
| ||
| I think it is a great idea and I would love to see it. I agree that there is plenty of stuff that gets lost out there. Many times when someone asks about something and uses search they can't find it. Then someone links them to a > 1 year old thread that has what they need. I appreciate the efforts that you are making to make the community better. The fact that you have included the target installers in the demo version when I think many would make that a paid feature is admirable. Thanks for the things that you do for the community. |
| ||
| Thinking more and more about it. As module developer(s) do you prefer to release each version using a redistributable ZIP (or the like) that contains the module itself, or you prefer to allow people to synchronize your module using Git or Mercurial? (I would need to prioritize one approach or the other, and it'll depend on module developers preferences). |
| ||
| FWIW as mostly a module *consumer* I prefer zips. That's what I end up grabbing from the various version control sites anyway. |
| ||
| As a module Dev I hate anything that requires extra work to push new updates to people, so auto synchronization is a must. Being forced to upload zip files directly to the site would be unacceptable. Other devs should know how to get something from a repo by now, and both github and bit bucket do auto zips. Also, please consider using openID to log in. Most real Dev sites do this now, without requiring double registration. The only places that do Facebook only login are social sites and hipster startups- (I dont even have an FB account!) and double registration on openID sites only exist because they want your information but don't actually need it... |
| ||
| Prefer NOT to provide release packages as zips. It is a major PITA to keep them updated. Sync always please. Also remember maximus? https://github.com/maximos It was a similar idea done for blitzmax. You might be able to get some inspiration from the work already done there. |
| ||
| "As a module Dev I hate anything that requires extra work to push new updates to people" If you sign up for this, you need to either package ZIPs, or maintain a "latest stable and bug-free" repo separate from your WIP repo. It won't be like sharing code on the Internet with other devs. You'll have "customers" who click "update all" routinely. In fact, they'll very specifically be Ziggy's customers, at first. "I clicked update on this module and now my project won't compile" will go to him, until people learn otherwise. Especially if Monkey grows. Right now, I trust that if FontMachine says it needs an update, I am safe to say yes. Those of you who release modules... how many of you currently maintain separate WIP and release repos? My impression is that pretty much everything is actually WIP. |
| ||
| The major glitch with modman was (is?) that it just picked up the latest revision. If Ziggy requires a release to specify a revision/tag/branch/whatever for a release then that problem goes away. It still leaves the general problem of requiring devs to use particular version control mechanisms and for users to install those tools though. It also breaks Nobuyuki's "no extra work" desire but I think that's a non-starter anyway. The best you can do is minimise release overhead, not remove it. To be honest I don't get what the issue is with creating zips. If you're creating a module for release then you need to have it sitting in your module directory in the release state in order to test it (you are testing it, right?). Zipping it and uploading it wherever from there is a minute or two of effort at worst. That said don't see why this is an either/or problem. If a module package is a zip or other archive format with some sort of release and dependency specification then you can create it from files or from a repo tag/whatever if your repo is suitably organised. Ziggy seems to be shying away from creating a packager and I can understand that from an effort perspective but I think it's a necessary evil. Providing a tool eases the path to adoption and also provides a mechanism for maintaining release quality/consistency. As I've mentioned previously I had been considering building something to help address the module discovery/archival issues along with other problems like documentation (I even bought a domain). I'd be happy to help out with this as long as it's genuinely open for the community and separate from BRL and LemonBytes. |
| ||
| @Raph "As a module Dev I hate anything that requires extra work to push new updates to people" If you sign up for this, you need to either package ZIPs, or maintain a "latest stable and bug-free" repo separate from your WIP repo. It won't be like sharing code on the Internet with other devs. You'll have "customers" who click "update all" routinely. In fact, they'll very specifically be Ziggy's customers, at first. "I clicked update on this module and now my project won't compile" will go to him, until people learn otherwise. Especially if Monkey grows. Right now, I trust that if FontMachine says it needs an update, I am safe to say yes. Those of you who release modules... how many of you currently maintain separate WIP and release repos? My impression is that pretty much everything is actually WIP. Unless those "customers" are paying me money, I'm not bending over backwards for 'em. They're fellow developers and if they want a commercial experience from my framework or modules they can pay me for it. Otherwise, if they don't want updates like that then they probably shouldn't download my modules! I make every effort not to break things between updates, but I do what I have to do, because I make the modules for my own projects first and foremost. I would say that as long as we can specify a checkout branch then it doesn't matter, because people who -do- care about that can keep separate dev and release branches. I'm not super-concerned with people who break their own projects with an automatic module update, but putting update burdens on the module devs is a step backwards. We want to increase adoption of modules, but in order to do that, first you have to get module developers on board if you want any hope of a chance for a self-sustaining ecosystem. That means reducing the barriers to entry for module devs to near-nil -- you can't force everyone to play ball on your new playground. Personally I think it would be best if Ziggy had an option to "copy over" any installed module directly to a project to prevent this problem: i.e., some sort of context menu can bake a current version of an installed module to the project folder. Barring that, however, version locks in the auto-updater would be fine too. Just something to remember which modules not to update automatically unless you do so from the manager, and something to show current versions installed and latest available from the same manager. Whether to have a package manager vs. a list of modules from their own repos I think depends on the intended purpose of the feature. If it's to add value to existing Jungle customers by making updating modules more convenient for them, and this alone, then it doesn't matter how annoying it is for module developers provided it's "better" than the current situation,. However I believe this feature is meant to spur more widespread community adoption of modules and for that the barrier is getting the modules to the people who want them first. Worrying about them breaking their own projects is putting the cart before the horse -- can't happen if there's no modules that even use the system, or any devs that care to keep their modules up to date on it. No software ghettoes please. |
| ||
| I definitely agree with low barriers for module devs, and I don't want to raise them any at all! More modules is a good thing! But this feature isn't for that; it doesn't actually lower any barriers for module devs at all, right? There's nothing easier for them in this, not any easier than using git or whatever else they use today. Rather, it lowers barriers for module *users*. I think that if Monkey's ecosystem grows, it'll almost certainly follow the same pattern as most any other community: a few will contribute heavily, and far more will make use of what those contributors provide. Most of the usability issues that have been the subject of discussion here in the wake of Mark's blog post have been about lowering barriers to entry and increasing usability, and this fits in that same pattern. This feature will not serve those goals if it acts a means of distributing broken code. The single biggest value this new centralized distribution channel lies in the fact that it is in fact centralized and (hopefully) easily visible. Once it's up and running, someone should go through the snippets of code posted on these forums and turn as many of them as makes sense into modules! It would be one of the biggest helps to new arrivals. That said, I agree that specifying a checkout branch is probably plenty, as long as devs are maintaining both. |
| ||
| The fact that I'll be using wordpress (as the basis) for the repository will allow people to rank and comment on modules. After thinking a lot on how to implement this, I think I'll do it this way: In the site: 1.- Modules distribution will be based on Git and Mercurial to begin with, but a package system will come later. Just a zip-like system with a tool to automate its creation and -maybe- its upload to a server (not sure abut the latest). 2.- The site will allow people to comment on modules in each module page, and rant them. 3.- The system should allow people to notify broken modules (dead projects, etc.) We have no problems on open sourcing all this dev, in fact, we considere it would be a good idea, so we'll definitively be doing it. In the "assets manager" tool: 1.- you'll be able to navigate the site 2.- you'll be able to install the module you're viewing with a single click 3.- you'll be able to inspect which of your modules have updates, and update them with a single click 4.- git / mercurial / etc. This tools, if they're required and not present, will be installed for you (after requesting permission). 5.- Obviously, this tool will be open source too. EDIT: I'm still not sure in what language write this tool in order to develop it fast and cross-platform. My first idea was to make it in Java using Swing and JavaFX so it can be done very fast, as most of us already have the Java runtime for Android dev. But if anyone has an even faster and reliable suggestion, I'm open. Please, don't suggest BlitzMax as the web browser does not work well on Linux, and also, please don't suggest any Monkey wrapper for a foreign gui if it has not been depply tested and is known to be reliable. I want this to be done with little effort so we can concentrate in making it better, -instead of concentrating in making it work. /EDIT I think this would be enough as a starting point. Any improvement suggestion (not yet mentioned) will be welcome. Ziggy seems to be shying away from creating a packager and I can understand that from an effort perspective but I think it's a necessary evil. I'm not! I think it is a very good idea, and will do. Just that some modue devs that do already have a master branch with the proper module strcure (I do) would benefit from the git or hg approach faster. I'll be doing both, but I'll start with the git / hg system frist. You'll be able to switch from one to another in the repository site, so it shouldn't be a major issue. In fact, I think that it's better to provide ZIP when the development is very active, so all changes too much too often, but when the module matures and it's mostly on a bug-solving status, switching the distribution to git or mercurial will make it easier to release early and often quick fixes. |
| ||
| A bit late, but as a user I'd mostly like a download site with descriptions of modules (and versions available) and a search function, which then gives me the opportunity to download zips of any I want. If it's a tool with built-in git/mercurial that has the same effect as zips, e.g. lets me download a specific version into a specific folder, of course that's just as good (or better if it's more convenient). |
| ||
| A bit late, but as a user I'd mostly like a download site with descriptions of modules (and versions available) and a search function, which then gives me the opportunity to download zips of any I want. That's more or less what we're doing, but instead of getting a Zip, you'll be able to get the module installed and ready in whatever module's folder you've configured, and also be notified when any of your modules has updates availability. |
| ||
| Wow you type fast ;) We're thinking along the same lines, my edit was going in the same direction. As for module development, I *personally* would prefer zips for that too, as I'm more likely to do a small bit of code that will rarely or ever change, but I realise that developers of the big important modules, and anything that has multiple developers, will almost certainly be using source control systems. |
| ||
| @Ziggy - You may want to consider creating the "assets manager" tool as an Eclipse plugin. Much of the functionality is already there in a standardized familiar interface where you don't need to reinvent the wheel. There might also be benefits for you going this route if you ever decide to make a cross platform Jungle IDE work on Mac and Linux. |
| ||
| Love this project! To answer the initial question: Broad categories, platform support and optional tags. Here is a suggestion: - Serialization (xml, json, etc.) - Social Media (facemonkey, twitter...) - Testing & Performance (profilers, fps counters...) - Graphics (textured polyes etc...?) - Effects (particle engine, water, screen shake, explosions...) - Frameworks - Audio - Animation (spine, anim..) - Monetization (chartboost, revmob...) - Networking - Content Editing (mapeditors, map loader..) - Statistics (analytics, logging, graphs..) - Input & GUI - Algorithms (zip, encrypt) - AI (pathfinding, statemachine...) - 3rd Party (mobile service integrations, web integrations, addon libraries..) |
| ||
| I dunno, I always have thought that having all modules hosted in a single place is a far better option that having an app that attempts to connect up to tons of different repositories. Rather than taking example from modman or the other brl related asset managers, why not look at how the professionals do it, look at how linux, ubunut etc handle their packages, and modules, look at how other big named languages handle theirs and take inspiration from them, in short I think you would find that most of them are all centralized fully supported echo systems. Also not all module developers will use a repository and others will, some might not use the common systems like git, some might use subversion or dropbox, hell I know some one that only uses ftp servers, I think having a system that tries to pull stuff in from a remote repository is just a waste of time. The big issue with a centralized system is that you need to get people to actually use it, truth be told module developers need some incentive to go out of their way to upload to yet another location, the good news is that an incentive like that is actually really easy to come by, Mark simply needs to officially acknowledge and support the system easily via website links and if possible website embedding, when developers look at the site and can see a clear list of modules and a clear indication of the tool thats used to manage them then they will use the system because it will be the norm, they will still have their own repos but they will also keep their module upto date on the site because thats where 99% of people would be going to get it. |
   |