When do circular imports fail?

Monkey Forums/Monkey Programming/When do circular imports fail?

Sicilica(Posted 2016) [#1]
So, context, I'm in the process of wrapping a C library and realized that I needed circular imports - two structs, A and B, needed to each contain an instance of the other. I was really worried for a while because, unlike C, you can't declare a function/struct/class definition early to my knowledge (ie, in C you would declare that there will be some struct B, then give struct A with it's implementation, then implement struct B with it's implementation).

It turns out that circular imports DO get resolved in Monkey, which really surprised me! I knew you could do this if the two classes were declared in the same file, but I also know that I've run into problems with circular imports failing before in Monkey.

So, my question is this: what circumstances would prevent Monkey from resolving this sort of import situation? Or even better, is there some way to resolve the dependencies in a C-style way by predeclaring definitions in a header or something and I just didn't know about it?


ImmutableOctet(SKNG)(Posted 2016) [#2]
Monkey uses a multi-pass compiler, this means the parsing, semantics and compilation (Translation) are handled one after the other, processing the source code in phases. This is in contrast to C and C++, which process the code as it appears in the source file.

Monkey is similar to Java and C# in this regard, as the compiler doesn't have to care about the order declarations appear.

In Monkey, a global variable can be assigned to another global variable declared after it. The same goes for constants, and of course, calling functions. Functions can be called when assigning a global variable or global array's content. This does not require the declaration of the function or variable to be provided beforehand, because the compiler has already processed the source code enough to know both where and what everything is.

As an example, this code structure is valid in Monkey, but not in C++:

Compile Online
The current value of X: 891
The previous value of X: 890


There's ways of getting around the limitations restricting this code structure in C++, but in Monkey it's a trait of the language.

There's some "magic" going on behind the scenes to make sure that certain details like pre-executed functions work, but that's its own topic.

With C and C++, you're dealing with a sort of "single-pass" compilation model. There's a bit more to it when it comes to optimization (Especially with C++'s templates), but that's an implementation detail. From the programmer's perspective, C and C++ compilers don't understand anything until they're declared somewhere. Usually, this is in a header.

In these languages, a header is basically how everything is declared. Including a regular source file (c, cpp) is the same as including a header file (h, hpp, etc). The only thing special about a header is that they're less confusing, and functionally, they allow two source files to reference the same declarations. (Types, functions, etc.)

When you use forward declarations, you're essentially making a contract that the type you're using will be resolved with the same symbolic name internally. However, since you're not defining the contents/implementation of the type, you're unable to give the compiler a means to allocate it, nor are you giving it enough information to access the members of the type. This is why C++ uses headers, they display how the type works to the compiler, and are symbolically the same to the linker.

This is in contrast to Monkey, which is module based. In Monkey, modules contain both an implementation, and an interface (Declarations). In order to provide the ability to perform out-of-order access, Monkey needs to first map out what can be referenced. When a module is imported, you're essentially allowing the compiler to load the symbolic information for the code in the module(s) you import. This can later be used to ensure your code is correct, as well as understand what piece of code you're talking about.

In C++, this all has to be understood using declarations and definitions ahead of time, where all of the information is compiled and passed to the linker. The linker of course makes sure all of your declarations match up, and of course, cover the other parts of the ABI.

So, does Monkey have circular dependencies? Yes, but mostly just when it comes to external code. External files have to be imported and loaded into the translator's output, meaning the order has to matter for languages like C and C++. There's ways around this, especially since Monkey 1 uses pointers for everything, but it's still a potential problem.

In pure Monkey, however, the only circular dependencies are related to the actual code structure, not the semantics of the compiler.

For example, function X can't call function Y if it calls X. Similarly, classes can't inherit themselves.

As a side note, Monkey 1 actually has a hard-coded trigger to ban references to the active type when using 'Extends'. There's actually some good reasons to write classes like this, so Monkey 2 supposedly behaves differently about this. This means regular CRTP software models would be possible in Monkey 2, but are restricted in Monkey 1.

There's a decent bit about these languages that I didn't cover, but I hope I got the idea across. C and C++ share a single-pass compilation model (With a linker step), and Monkey only has to deal with this when it hits the native side of the compilation process. There's some cool things about C's compilation model that are worth mentioning, though. For one thing, every source file is considered its own compilation unit, meaning they can be built largely in parallel.

Monkey also has some nice conceptual traits, like build optimization, but that's getting into Monkey 2 territory.


Sicilica(Posted 2016) [#3]
Dangit, SKNG, every time you post I feel bad for taking so much of your time. Don't you ever write normal size posts?

I definitely understand everything with linking in C, etc, but that's probably some really important concepts for someone. It can be odd at first - though never as bad as a big project I worked on where the other guy who had written most of the code believed in using extern's for EVERYTHING. Makes it dang near impossible to know where anything is declared, and since it was C and not C++, it also means you can't guarantee ANY encapsulation.

Does your function X calling Y calling X really not work in Monkey? I guess I've probably never tried, but I would assume that would work since you could read all the function signatures (so you can know there's an X and a Y) and then give their implementations afterwards. The situation that I was confused about was where two classes reference each other in their fields, since those are part of the class signature. In C there are several workarounds (use a void* and change it later, or declare the class without an implementation), but I wasn't sure if Monkey would be cool with that. I would never worry about that in other languages that use multi-pass compilation like say Java, because in Java everything is always in scope and you don't really "import" anything. I thought Monkey's "Import" worked more like an actual inline copy, but I guess it doesn't then?

Come to think of it, I think Mark mentioned once that that's why Import wasn't a preprocessor command, so I should have realized. Although I think it is now in Monkey2?

IN ANY CASE, I guess the tl;dr from what you wrote is, if Monkey can compile it if it's all in one file, it will still compile in multiple modules. Certainly things like a self-extending class cause circular dependency errors in Monkey already.