Internal/Protected Keywords
Monkey Forums/Monkey Programming/Internal/Protected Keywords
| ||
| All, I've forked trans and added two new keywords for encapsulation. Pending Mark's response, I will submit a pull request. Protected: Acts similarly to both C#'s and Java's "protected" keywords, where the field/function/method is visible to the current module and any subclasses of that class (regardless of module). Internal: Acts similarly to C#'s "internal" keyword and Java's "default" visibility, where the class/function/method/field is only visible to modules within the same "package" (directory, for Monkey). GitHub fork: https://github.com/swoolcock/monkey Download (MinGW): https://dl.dropboxusercontent.com/u/20511758/trans.zip Original post: I decided to have a go at improving Monkey's encapsulation, so I've introduced a new keyword "Internal". It works basically the same way as in C#, and similarly to Java's "default" visibility. If a field/class/method/etc. is declared as internal, it can only be referenced by monkey files in the same directory. main.monkey: Import foo.one Function Main() Local h:=New Hello Print "Main: "+h.bar ' doesn't compile, because bar is internal and "main.monkey" is not in the "foo" directory TestFunc() End foo\one.monkey: Class Hello Internal Field bar:Int = 10 End foo\two.monkey: Private Import one Public Function TestFunc() Local h:=New Hello Print "TestFunc: "+h.bar ' compiles because "two.monkey" is in the same directory as "one.monkey" End So what does this mean? It means that framework/module developers can now make things "public" within their framework without actually sharing them to the end user. Got a funky ugly utility method in one of your modules that the user should never ever see? Now you don't need to make it public. I've got it working locally in a forked version of trans, and once I'm sure it's all good I'll push it to my GitHub fork. If people are interested (especially Mark), I'll submit a pull request. And get this... I made the changes to trans in Notepad++ with no syntax highlighting and it compiled first shot. And worked first shot. I'm scared... D: |
| ||
| Does it work for files in subfolders too? |
| ||
| I am definitely for this. In fact, this is what a few people wanted as a "Protected" keyword. Basically, what C++ has for friend classes and inheriting classes, only with module-level behavior as well. |
| ||
| @ziggy: Does it work for files in subfolders too? When you try to reference anything flagged as internal, it will not compile unless the source is in the same folder. This goes for any code in any module. @ImmutableOctet: I am definitely for this. In fact, this is what a few people wanted as a "Protected" keyword. The protected keyword works slightly differently in Java. It gives you package/default visibility (like internal) but also visibility to subclasses. Personally I don't like the latter case. It seems like a cheap trick the developer can use to get access to things they shouldn't. |
| ||
| Personally I don't like the latter case. It seems like a cheap trick the developer can use to get access to things they shouldn't. What!? Like "setAccessible(true)" ;P |
| ||
| @therevills: What!? Like "setAccessible(true)" ;P You love the reflection. :) Besides, if Sunacle really cared about it they'd put in a more restrictive classloader that would prevent it. |
| ||
| This is pretty awesome, but it's not a replacement for Protected -- I'm personally not too thrilled about keywords that enforce rules about how source files should be stored on-disk, but that's just my style; I don't know why, but that seems rigid and arbitrary (actually, a lot like how Java is in general). It's a step in the right direction, though. So module developers can extend their own classes in other files and still be able to keep them away from people who aren't hacking the module itself. Woo-hoo, a win for encapsulation on the part of module developers, right? But only for ones who code in a certain way! This helps the ones who were having woes extending their own classes that had Private fields, not necessarily the ones who want to offer the ability to extend their classes to others. Even if it's basically just a 6" tall fence, Protected's main purpose is to enhance encapsulation in the same kinda way Internal does, but without the rigidity -- So, if you design a module with classes that you're encouraging people to extend in their own projects, they don't have to hack the module to do it, but still gain the benefits of having dirty-looking class members tucked away (and in the case of a proper IDE, neatly hidden from autoComplete) should they not want to extend the existing classes the module developer included. Edit: Anyway, if this goes over without a hitch, I'd love to see exactly where you needed to edit to get this working. I'd still love to see a Protected keyword available, and both of the keywords can actually work together in other languages, but until now, all of the scope directives were mutually exclusive. This is definitely something to think about seeing as how there appear to be different camps of people who are gonna prefer one style or the other of shielding class members from being accessed outside of their "proper place". Syntax-wise, you can't stack scope directives in Monkey right now, and it would be very confusing to have rules where they do, unless all scope directives which can combine with another need to be placed on the same line where any directive is called, and each new line with a directive resets everything else. |
| ||
| How is this different than Private ? Does Private only allow access within the monkey file itself? |
| ||
| in my ideal world, a Friend keyword that exposes functionality only to the modules I'm importing, and not to the modules that import me, would be the best encapsulation model to develop modules, but I guess it's quite difficult to implement. However, this is anyway a nice addition to the language. |
| ||
| @Nobuyuki: ...but it's not a replacement for Protected... It's not intended to be a replacement for protected. Protected could still be added, but increases the visibility of internal slightly. Visibility would go: Private < Internal < Protected < Public Private: Can only be seen within the same source file. Internal: Same as private, but can also be seen in source files at the same level/directory. Protected: Same as internal, but can also be seen by any subclass regardless of source file. Public: Can be seen everywhere. @Nobuyuki: I'm personally not too thrilled about keywords that enforce rules about how source files should be stored on-disk... This was the easiest way to ensure that source was at the same module "level" as others. This problem would be alleviated if Monkey source required a package/namespace definition at the top of the file, but ehhh. @Nobuyuki: Woo-hoo, a win for encapsulation on the part of module developers, right? But only for ones who code in a certain way! All module developers should code in this way. If you don't want your user to see it, you shouldn't make it public. Internal and/or Protected keywords allow a little more flexibility for those that code their current way. It actually helps those module developers! :) @Nobuyuki: Anyway, if this goes over without a hitch, I'd love to see exactly where you needed to edit to get this working. Some very minor changes to decl.monkey, parser.monkey, preprocessor.monkey, and toker.monkey. I can quite easily extend my changes to include protected too. @tiresius: How is this different than Private ? Does Private only allow access within the monkey file itself? Correct. @ziggy: ...in my ideal world, a Friend keyword that exposes functionality only to the modules I'm importing, and not to the modules that import me, would be the best encapsulation model to develop modules, but I guess it's quite difficult to implement. This is still possible, but now we get into the world of conflicting visibilities. I'd rather leave that alone for the time being. The more I change, the more testing would be required, which reduces my chance of getting a pull request accepted. Something to think about though. Edit: Pushed. https://github.com/swoolcock/monkey/commit/1f930f79cc4b2390a797cfb503de2e070ea8f6ac |
| ||
| Some very minor changes to decl.monkey, parser.monkey, preprocessor.monkey, and toker.monkey. I can quite easily extend my changes to include protected too. I would really appreciate you doing that if you're up to implementing it; I don't know about anyone else, but I'd definitely use it. There's already a bunch of lines scattered throughout my modules where I wrote "Protected" and commented it out hoping one day that Mark would include that directive :) Still, though, the thing about the syntax is something to be aware of. C# allows you to define members to be both Protected and Internal, which basically widens the restrictions to subclasses or modules, whichever one applies. I have no idea how this would be easiest to implement in Monkey, since I've never really dug around in there, but probably the most rational thing syntax-wise seems to be to limit scope directives to their own line, and allow someone to declare "Internal Protected" or "Protected Internal" on that line, but presume neither whenever a scope directive is encountered encountered unless explicitly specified. Again, not sure how this will work out, because it seems like syntax for scope directives is handled completely differently from most other keywords, but I guess the easiest thing to do would be to let any scope directive override all the others "just to be safe" at first since that's how it sounds like Monkey currently handles it. (Remember "Interfaces can't be Generic" ?) |
| ||
| Wait a minute, Samah, if all you want is to restrict files, why not just use private imports? They already work in Monkey. A private import just stops something from being auto-imported when someone imports your module. There's also "Extern Private", but that's basically a hack to make 'Private' work with external/native code. Seriously, though, couldn't you just import a module, have that module privately import its local sub-modules, then only delegate what you want to be public? To me, a "Protected" keyword makes a bit more sense. Just some food for thought, but I think we really just need an "inheritance only" keyword, which makes things private to everything but derived classes. Technically, you could just privately import / not import the base-class's module, then simply force the user to import the module with the base-class, and extend from there (Making things private as the user see fit...?). But that can get overly complicated. I honestly think "Protected" and private imports are all we need, and we all ready have the latter. MSDN's documentation is pretty inconsistent at times, but here's their page on C++'s 'protected' keyword. Basically all we need, from what I can see. There is some need for 'Internal', but I'm fine with just getting 'Protected'. Having both would still be ideal, though. I just think 'Internal' is going against Monkey's "location doesn't matter" mindset, so we might need real "packages". |
| ||
| @Nobuyuki: I would really appreciate you doing that if you're up to implementing it; Already in the process. :) @Nobuyuki: C# allows you to define members to be both Protected and Internal... This starts to get messy to implement, which is why I've opted to go for Java's "protected = package + subclasses" model. @ImmutableOctet: Wait a minute, Samah, if all you want is to restrict files, why not just use private imports? Because the developer could still manually import that file. The point of encapsulation is to give the module creator the freedom to change any and all internal implementation with little fear of breaking backward compatibility. As soon as a developer misuses part of your framework, they'll come back at you with "why doesn't this work". It's easier to deny them in the first place than have to reply "don't do that" every time. That and I'm an evil overlord. >:D Edit: Additionally, your suggestion of private imports only applies to classes and top level functions. Using the Internal keyword on methods/fields etc. is not something you can achieve through private imports. You may want to make the class public, but some of its members internal. Edit 2: The beauty of open source is that if the developer really wants to be naughty, they can just hack the framework anyway. :) |
| ||
| Ok, I've got protected working separately to internal, and they can be used separately or together. If something is "internal" it can only be used within the same directory. If it's protected, it can only be accessed by subclasses. However, you can define things as both internal and protected to get both functionalities. Once I've fully tested it I'll push the changes. |
| ||
| Does this work on methods as well as fields? As I understand it, Monkey's Private qualifier only works on fields. (Personally I prefer the C++ system.) |
| ||
| Good work Samah! I look forward to having this type of encapsulation in Monkey :) Keep up the excellent efforts! |
| ||
| Pushed. Please test it for me, because I'm lazy. XD https://dl.dropboxusercontent.com/u/20511758/trans.zip Built using MinGW because I couldn't get VC++ to work on my PC. @Gerry Quinn: Does this work on methods as well as fields? As I understand it, Monkey's Private qualifier only works on fields. Monkey's Private keyword actually works on everything. If you look in the CheckAccess method in trans, it does a check for every Decl (class/method/function/etc.) of a program. If it's private, it checks to see that it's the current module. As for the new keywords, internal works on everything. Protected only works on class members (methods/functions/globals/fields). It makes no sense to use protected outside the scope of a class. @computercoder: Good work Samah! I look forward to having this type of encapsulation in Monkey :) Keep up the excellent efforts! <3 Edit: Be aware that because protected and internal can be used together, you cannot switch between them without first resetting the visibility. You will need to set Private or Public to reset it, then choose the one you want. I may change it in the future to check if the keywords are together, or replace them with a standalone keyword "ProtectedInternal". Public Field a ' public Internal Field b ' internal Protected Field c ' protected AND internal Internal Field d ' still protected AND internal Private ' resets the protected and internal bits Internal Field e ' only internal Edit 2: Can I get opinions on this? Which way would you prefer to force both internal and protected at once? a) Keywords must be next to each other (no variable/method declarations between them) b) Need to use another keyword like ProtectedInternal. c) Something else? a) is tricky, b) is a lot easier. |
| ||
| Cool! Hopefully Mark takes a look at this. The "reset" issue definitely would benefit from having both words work together, though obviously that would be more work. ProtectedInternal seems like a good-enough solution, but of course adds to the reserved keyword soup... Crossing my fingers a pull request will spur Mark to implement the elegant solution himself....! My list of preferred implementations, in order from "most wanted" to least-wanted: 1. "Protected" and "Internal" reset each other's bits as do Public/Private, but "Protected Internal" or "Internal Protected" is its own scope level above either individually (just below Public). 2. Same as above, but with a new keyword "ProtectedInternal", which is easier to implement. 3. "Protected" and "Internal" have bits that are only reset on Public/Private directives (how it currently is with this fork) 4. Not having either of these features (current mainline) |
| ||
| @Nobuyuki: 1. "Protected" and "Internal" reset each other's bits as do Public/Private, but "Protected Internal" or "Internal Protected" is its own scope level above either individually (just below Public). The problem with that is, do I allow newlines? Should "Protected (newline) Internal" count as "protected internal" or as only internal? The tokeniser is currently designed to ignore newlines for those keywords, and I REALLY don't want to change that. @Nobuyuki: 2. Same as above, but with a new keyword "ProtectedInternal", which is easier to implement. This is my preference. @Nobuyuki: Crossing my fingers a pull request will spur Mark to implement the elegant solution himself....! The point of a pull request is to have the changes merged with the mainline, not for the mainline developer to say "hey that's a good idea" and just do it himself. :| Otherwise I wouldn't have bothered, and just spam Mark with emails instead. Edit: I've changed it to use a ProtectedInternal keyword. Using Protected or Internal by themselves will now do ONLY those visibilities. "ProtectedInternal" is the least ambiguous keyword I can think of without inventing a new term. Private, Public, Protected, Internal, ProtectedInternal are more than enough. |
| ||
| I see your point. I wasn't aware that prefixing a declaration with a scope directive on the same line was valid Monkey code. I'm assuming that this sets the directive for everything below it, regardless of the fact it's on the same line. I don't know if anyone uses this coding style, but it's a serious "gotcha" for VB coders, because member declarations inside a class or module which aren't explicitly scoped are automatically assumed to be Private in VB. It's not very pretty Monkey syntax, either, seeing as how for consistency you'd have to make sure every declare line had the directive in front of it. "Elegant" in my mind means 1., but again I wasn't aware that putting scope directives on the same line as a declare was valid syntax. In that case, "elegant" in my mind would ideally mean changing the syntax to be scoped only to the line it's on if anything else occurs on that line. That doesn't sound feasible if anyone has code that relies on "VB-as-written, Monkey as interpreted" style declarations, though, because it could break their code in subtle ways.... Alternatively, throwing a compile error if a scope directive is on the same line as a declaration would break their code in a very explicit way. Either way it could get ugly... I guess I'm just not a fan of a keyword that is the portmanteau of two other keywords that already have the same functionality in the language. Should more scoping directives be implemented in the future (someone in another thread mentioned Friend?), combining them would end up requiring even more keywords. Then again, what other access levels can exist? If the issue's really just a tempest in a teapot, then ProtectedInternal would be perfectly fine with me; just wanted to bring up what I thought could possibly be a future issue. |
| ||
| @Nobuyuki: I guess I'm just not a fan of a keyword that is the portmanteau of two other keywords that already have the same functionality in the language. I agree, but as I said in my edit, it's the least ambiguous thing I could think of. Bringing in a third visibility name that = protected+internal is just something else to remember and makes the language more confusing. |
| ||
| The more I think about it, the more I'm wondering if ProtectedInternal is even needed. I can't off the top of my head think of a situation that it would be useful. I may just drop the whole idea and leave Protected and Internal as mutually exclusive. |
| ||
| seems safer to do that to start with; if someone has a use-case scenario for it, I guess it would end up being a future feature request, huh? |
| ||
| ProtectedInternal removed, changes pushed, hosted binary updated. If you guys can do some testing I might make a pull request at some point, but I want a response from Mark first. |
| ||
| Well this post was about to fall off the page, so instead of just bumping it, I've submitted the pull request. https://github.com/blitz-research/monkey/pull/74 |
| ||
| good! Sorry that I didn't do any rigorous testing. I wouldn't be able to tell what problems were related to the IDE not supporting the modifications, and what problems were related to the patch, so I'm waiting for this to get pulled into the develop branch and for ziggy to make the Jungle inline compiler be aware of it before I start putting it through the ringer. |
| ||
| Ok, so I was naughty and made these commits on the develop branch rather than a feature branch. Now that I'm going to strip out all the Internal stuff and just leave the Protected keyword, I'm considering making a new feature branch from the original clone point. However... should I also strip the other commits? I know it's very naughty to rewrite public history, but I feel bad for putting yucky commits on a main branch. Maybe I should just delete the repo and reclone? |
| ||
| Mark has now added Protected and Friend. :) https://github.com/blitz-research/monkey/commit/4e42bae58122397697e0256060560bdeee34cb02 |
| ||
| "Friend"??? |
| ||
| Edit: I misunderstood Mark's implementation of Friend. |
| ||
| A quick rundown on friend... Friend grants 'private access' to a module. Friend statements must appear at the top of file along with imports, eg: '----- test1 ----- Import test2 Function Main() Print strange End '----- test2 ----- Friend test1 'doesn't actually import test1, just declares it as a 'friend'. If you want test1, you still need to import it. Private Global strange:Int 'private, except to this module and friend modules. Note though that since test1 hasn't declared test2 as a friend, test2 still cannot access private stuff in test1. This serves a similar purpose to 'internal' in c# (apparently, never used it), 'friend' in c++ and the idea of 'package scope' in Java. It's basically away to let external (but trusted/known) code access some of the nitty gritty bits of a module. An example of this would be in mojo.graphics, where there's a public function called SetGraphicsContext or similar. I really didn't want this to be public, but had no choice since it's basically there for mojo.app to call. With Friend, I can now make it private, add Friend mojo.app to mojo.graphics and all is groovy. As for protected, this works the same way it does in c#, c++, java (sort of) - it makes class members accessible to derived classes, eg: '----- test1 ----- Import test2 Class D Extends C Method Test:Void() Helper() End End Function Main() Local t:=New D t.Test End '----- test2 ----- Class C Protected 'can only be called by C or derived classes. Method Helper:Void() Print "Help!" End End |
| ||
| @marksibly: As for protected, this works the same way it does in c#, c++, java (sort of) - it makes class members accessible to derived classes, eg: Java's "protected" expands "package scope" to also include subclasses. private: Only the current compilation unit (package): The current compilation unit, plus all classes in the same package. protected: The current compilation unit, all classes in the same package, and any subclasses from any package. public: Everything. |
| ||
| Great news - I've always wanted a Protected access qualifier! Will this be moved into any Monkey language stuff? List and Node in particular have always seemed to me a natural fit for the use of Protected instead of Private, |
| ||
| @Gerry Quinn: +1 to that , I always wanted to be able to extend Stacks to access the raw array on occasion. Would definitely make it cleaner to pack in extensions to the native containers for things like better sorting / collating capability and etc |
| ||
| @Nobuyuki: @Gerry Quinn: +1 to that , I always wanted to be able to extend Stacks to access the raw array on occasion. Would definitely make it cleaner to pack in extensions to the native containers for things like better sorting / collating capability and etc Yeah, I will be going through the Diddy container classes to improve their sorting performance, etc. |
   |