I still say that max's memory mangement is insane.
BlitzMax Forums/BlitzMax Programming/I still say that max's memory mangement is insane.
| ||
| While I admit that Max's memory management scheme is elegant in it's design, I am still struggling with it even after a month of using it because it is a NIGHTMARE to manage in practice. Take this comment for example, which I need to have in my code to remind me that stuff will actually get freed when I free a path in my sprite system: ' Freeing the path will cause the app to lose the reference to the point list, which will cause the point list to be freed, which sets the value for each ' link to Null, which means that nothing will be pointing at the points that were in the list, so they will be freed, which in turn will allow the links ' in the list which the points were pointing back to to be freed. See what I mean? Each point in my pointlist points back to the link which points to it. That means in order to figure out whether or not the links in the list will actually be freed, I have to recurse back through all the references in my head in order to figure out whether or not memory used by each link in the list will be freed when I forget about the pointer to my path. Comparatively, if things worked in a normal manner it would be qute simple to know what is freed, because I have to EXPLICITLY free it, or I KNOW it hasn't been freed. Ie, when I free a path, I have a list of points in that path. I need to first free all the points in the list, and then free the list. Then I'm done. I don't need to think about the fact that the list points back to the points and then go into the bowels of Max's source code and the docs to find out that Types have a special delete method, and when the garbage collector frees it, the delete method is called, and that this delete method in Max's source calls the remove method for all the links in the list, and the remove method for those links set the link's value to Null, which is actually the pointer to the object that it was pointing to, and that by setting this to Null the reference is removed, and that this is fortunate, because if Mark had simply assumed that when you lose the list reference that all references to the links in the list would also be lost, and that said pointer would automatically be set to Null when the link is freed, then we'd have a problem, because now the link would point back to the point, and the point would point back to the link, and neither would ever get freed. Because of this Mark's elegant solution is less elegant because it becomes dangerous to assume that any pointer will automatically be set to Null when the object is freed, because circular references like this will end up causing memory leaks. So in effect, Mark has simply replaced Delete Pointer with Pointer = Null. If we do not do Pointer = Null whenever we want something to be freed, just before it would be freedd normally if not for circular linkages, then because of the circular linkage issue, we will have to go through insanely complicated leaps of logic as outlined above to figure out whether stuff will be freed or not. |
| ||
| I agree with you. There are possibly just as many ways of creating a memory leak with auto garbage collection than there are by forgetting to manually destroy objects. Give me total control any day - it is a nightmare dealing with circular references - and given the shortcomings of the BM TList type, using circular references is pretty much essential if you want your code to be efficient. |
| ||
| ... Although I know where this ends: Thats a thing that depends on your implementation. If you make a list that points back to its owner, then you will need destroy methods that free up this "backlinks" to free it (its good practice to have a clean up functionality per object no mather if you program in a GC controlled environment or with trashy pointer hick-hack) Reason for that is, that BM does not seem to use root reference counters aside the regular ref counters. Those would fully solve the problem because if the cyclic structure can't be accessed from the root anymore, its simply dropped (as C#, Eiffel and other languages do it). I hope that functionality will be added as it is definitely a must have for a managed, GC driven environment as any design error creates massive problems. (leaks are no that worse problem as in C / C++ as they can't harm the rest of the system but the memory pool will grow until the system freezes). My EntitySystem already fakes this behavior to come around the various scenarios of a potential "disconnected data ring" (beside that: I don't use double linked structures as the TList implementation, that will already help you quite a lot. especially as TList is extremely unusable for depthsorting stuff) |
| ||
| It would be nice to have access to the object's ref count - is this in? I can't seem to find it in the docs (not that that means anything). Would make like easier during development for us guys used to using the trashy pointer hick-hack way for a while at least. |
| ||
| "especially as TList is extremely unusable for depthsorting stuff" Works just fine for people like me who don't make funny Compare functions. :-) |
| ||
| "especially as TList is extremely unusable for depthsorting stuff" Works just fine for people like me who don't make funny Compare functions. :-) I think what he's talking about is the fact that it uses bubble sort for sorting the list. |
| ||
| Yepp. For example. Or that is has much functionality not needed that can only be missused by users to destroy the functionality. I modified my sorting method a little. If the z value is larger than (head + tail)/2 then its entered at the back of the list and sorted up from there. (I use insertion sort) I don't mind much on sorting algorithms speed because of the amount of entities we talk of in a graphical real time environment, the sorting algo won't make a measurable difference anymore. Most time is lost through the graphical part and state settings. Brendane: I fear the ref count is not exported to BM functionality. But you could do that altering the source. I myself wouldn't do it as I think it will cause more damange then help. But one thing I'm researching is how to implement a fast root ref ... surely I would pass that along to mark then as well to become part of the official BM and solve most of the "leaks" that normally appear on the board. (it won't help with unexperienced users thought that use alloc or use memory that was created in C / C++. That will always be a thing the user has to care unless Mark puts a layer in between that enforces BM objects within BM and put the rest in a container as C# does. The actual way is very critical ... had to suffer of such stuff in eiffel already were the GC freed stuff that SDL still had alive and vice versa) |
| ||
| I don't think bubble sort is "extremely unusable" - I think he was referring to the fact that you have to override object.Compare() - and the only way to do this which works with TList.Remove() etc. is to write it incorrectly - such that it makes Sort routine swap values even when they are equal. But that's another thread ;) |
| ||
| So because your design is poor, automatic memory management sucks? I have a question for you. If you where using manual garbage collection (ie. had to free objects yourself), how would you do that without a live reference to the object? If all your sprites where in a list, and you removed the reference to that list, how would you get at the sprites to free them? Answer: You couldn't. The Automatic GC can. It ensures that nothing is freed until you're absolutely finished with it (thus no more rogue pointers, YAY). How can this possibly be a bad thing? |
| ||
| Because of missing root reference it can ... if you free a double linked list or any other cyclic structure, the GC won't free it. It will reside in the ram pool as dead memory. But thats the only point against a managed memory environment :-) |
| ||
| "how would you get at the sprites to free them? Answer: You couldn't. The Automatic GC can." No it can't. You must still set the reference to Null in order to decrement the refcount. |
| ||
| That's because he's using cyclic references = poor design. |
| ||
| It's a design I've used myself in order to get around an inefficiency in the BM TList type (not to mention because of the Sort()/Compare() issues mentioned above.) It's not necessarily a poor design - but it is over-complicated because of the reasons above - despite that, it is still infinitely more efficient than the alternative. |
| ||
| FlameDuck: I fear there are situation where cyclic structures can't be avoided. Algos often even base their break condition on that. I'm a fan of the GC driven system, you should know that :-) But the missing root ref count is a serious problem for a fully managed environment. But in most cases you can avoid it or "break the problem" by a good OO design. But it is still a serious source for leaks that should NEVER be possible or happen in a fully managed system! |
| ||
| It'd be nice if we had weak references. Those would probably help for cyclic references since an object storing a weak reference to itself will be collected even if the reference to itself isn't removed. |
| ||
| So because your design is poor, automatic memory management sucks? In what way is my design poor? If you where using manual garbage collection (ie. had to free objects yourself), how would you do that without a live reference to the object? If all your sprites where in a list, and you removed the reference to that list, how would you get at the sprites to free them? Answer: You couldn't. The Automatic GC can. That is a rather poor argument. It hinges on you, for some unknown reason, not having a reference to an object you want to free. Does this situation come up regularly for you? Because it doesn't for me. It ensures that nothing is freed until you're absolutely finished with it (thus no more rogue pointers, YAY). And in exchange, it ensures that circular references always lead to memory leaks if you fail to set pointers to null. How can this possibly be a bad thing? It's not. You set up a strawman argument. I said something was bad, and you constructed a strawman which has nothing to do with my argument and is easily knocked down. My argument is not that "nothing is freed until you're absolutely finished with it" is bad. My argument is that keeping track of what will be freed by automatic GC and when requires a lot of effort. And that you are very likely to have memory leaks due to circular references, which it is inevitable will occur on occasion. Here's one reason I keep getting circular references: If I make a list of points, and I want to iterate through them, For EachIn works fine UNTIL I need to access the previous or next point in the list while I am on the current point. At that point, the easiest way to get access to that data is to have a _Link field which I store when I create the point. Then I can simply do the following: For Point = EachIn PointList If (Distance# + Point._SegmentLength#) > Px# Then Exit Distance# = Distance# + Point._SegmentLength# Next NextPoint = PathPoint(Point._Link.NextLink().Value()) If I don't use _Link, then I have to do this everywhere I need to have a loop: Link = PointList.FirstLink() While Link <> Null Point = PathPoint(Link.Value()) If (Distance# + Point._SegmentLength#) > Px# Then Exit Distance# = Distance# + Point._SegmentLength# Link = Link.NextLink() Wend NextPoint = PathPoint(Link.NextLink().Value()) That's just an isolated example. Regardless of whether I was able to rethink how I did my iterating through lists, I'd still have to figure out these complex chains of events that lead to something being freed or not. For example, when trying to figure out what happens when Sprite.Free() calls Animate.Stop() and Animate.Stop() calls Sprite.Free(). With manual deletion, I can say with certainty immediately if the sprite is going to still be there at any particular time, but with this reference stuff I have to think backwards and step through and say "Onthat free() calls stop() and stop calls free() so when free gets called the second time will that pointer to hat animation still exist so the animation will finally be freed, because the sprite was freed by the previous call to the stop() function... It's a pain in the ass figuring out what gets freed cause I have to step through the whole process to figure out if all the links are freed or if there is a hidden circular reference in there which will prevent everything from being freed. This becomes particularly complex when you have a sprite pointing to an animation which points back to the sprite both of which are in lists and contain pointers to their own link in the list. |
| ||
| "That's because he's using cyclic references = poor design." How are cyclic references poor design? Sometimes you need to have a list of objects AND know the link that contains a particular object, and for speed reasons looping through the whole list looking for that object isn't an option. You know just making one simple change to Max would have made my life a lot simpler by rediucing the number of cycliuc references, and that is having For EachIn return the link pointer rather than returning the object the link points to. Then you could easily get the prev and next links. For Link = EachIn PointList If (Distance# + Point(Link.Value())._SegmentLength#) > Px# Then Exit Distance# = Distance# + Point(Link.Value())._SegmentLength# Next See? Much cleaner than that while loop. And you could add a third line in there to save the point pointer so you don't need to keep using Point(Link.Value()), if you needed to use that many times in the loop. |
| ||
| In what way is my design poor? You are using a data structure that is unsuitable for the problem you're trying to solve. If I make a list of points, and I want to iterate through them, For EachIn works fine UNTIL I need to access the previous or next point in the list while I am on the current point. I've already explained to you how you can avoid that. If you chose to ignore it, I'm having a hard time seeing why I should feel sorry for you. If I don't use _Link, then I have to do this everywhere I need to have a loop: Or you could override the iterator to return links, if you *need* them, rather than the actual objects. ometimes you need to have a list of objects AND know the link that contains a particular object Talk about attacking every problem as if it where a nail. If you need a collection, and you need to be able to iterate over them in linear time, and remove them in constant time (or better than linear time), then you are using the wrong collection! Try an associative array instead, like TMap (log n) or that hashMap (near-constant) that's floating around here somewhere. You know just making one simple change to Max would have made my life a lot simpler by rediucing the number of cycliuc references As I've already explained, you can make that change yourself, if the default behavior doesn't suit you. As I've also already explained, this does not nessecarilly involve changing the core BMX modules. |
| ||
| In what way is my design poor? You forget that Flameduck is still in school. Anything that isn't abstracted through 12 design patterns is still poor design to him. |
| ||
| "I've already explained to you how you can avoid that." Where? Not in this thread. "Or you could override the iterator to return links, if you *need* them, rather than the actual objects." How do I override the TList iterator without breaking other people's code when they try to use my library, and without having to make my own list type, which possibly extends the tlist type, which is retarded, because the point of my requesting that feature was to make my life EASIER, and making an extra type whenever I need a list is not making my life easier? "Try an associative array instead, like TMap (log n)" Uh, you do know that a TMap is a binary tree, and that unless you balance it when creating it, which would be a big pain in the ass to do, you could potentially have to iterate through every single element in it just as with a list. And Hashmap? I know what a hash is, and though I don't know what a hashmap is exactly I think I have a pretty good idea, and that is just as dumb as the TMap suggestion for the situation I described. TMaps are for doing stuff like figuring out which polygons are on the front side of a particular polygon. Hashmaps are... Probably for finding a particular string in a list of strings quickly. Neither of which is anywhere NEAR suited for the purpouse I described. Talk about attacking every problem as if it where a nail! "As I've also already explained, this does not nessecarilly involve changing the core BMX modules." But I bet it does involve creating your own type that extends the list type. Anyway Flameduckie, you can call my design philosophy crap all you like, but I have 200 happy customers who would disagree wirh you. :-) I suspect if we were software tools, you would be Kai's power tools. You'd have a slick interface with a few objects on the screen. Click the objects, and more objects would appear. And click those, and more would appear. Very pretty to look at... and utterly USELESS to an artist, because all the real functionality is five layers down from where you started, and what functionality there is is very rigid and inflexible, with no way to enter text in textboxes, because hey, you've got a dial, you don't need to enter numbers manually TOO. That would be bad design! I on the other hand would be Sonic Foundry's Sound Forge. You might find my interface daunting at first glance, but once you realise that everything is exactly where you expect it to be, and works exactly how you expect it to work, it's really convenient having most of the functionality right there on the face of things. And it's not like 3D Studio Max where EVERY peice of functionality is on one biug messy screen. No, my interface has order to it. It has menus, with names, and there are things in those menus that are supposed to be there. But there aren't six levels of menus because hey, we can't have VOLUME ADJUSTMENT and RESAMPLE on the same menu! That's crazy! Those are different things entirely! Well, yeah, that's true, but it sure is convenient having them on one menu. Do uou really want to have to go three menus down to get to them just so everything is object oriented? No, I didn't think so! So, no Flameduck, my design does not suck. Yours is just too damn pretty for its own good. :-) |
| ||
| You both suck. |
| ||
| Sswift kinda won the argument this time. Flameduck - you better prepare better for the next one :p I've been using cyclic references too, and I didn't even think of this as a problem. Thanks for bringing it up Sswift, now I better check my source for memory leaks. |
| ||
| I personally would never design a game that creates garbage in the first place. The speed at which objects can be recycled yourself will always be a magnitude faster than any garbage collector could offer. |
| ||
| skidracer: by "recycled" do you mean not freeing them at all, just re-initalising the values and using an existing object? OR did you mean manually freeing and recreating? |
| ||
| hmm but only if a GC is programmed as the BM one: Enforced minimalistic RAM usage instead of "pool and don't cut the pool size as long as there is remaining ram" like .NET and other managed environment handle it. There is no reason to restrict pool size to < than available ram anyway ... beside "let an app look better in numbers" ... but to me, performance counts more than low memory usage and that would allow a significant speed gain if the GC did not cut the poolsize unless its needed. (C# + managed DirectX shows that point quite good ... its not significantly slower than C++ + DX but it is worlds easier to handle, program and design) |
| ||
| Grey: I think he means the former. And I wouldn't be surprised if it involved using arrays. Skidracer's design is like Linux. So efficient it is utterly incomprehensible to anyone who isn't a computer. :-) |
| ||
| I was going to jump into thread and say, OMG sswift, why do you have to make everything so complicated, but then i realized I've had this problem myself. Or something similar. Chains of memory left stranded. I had children and parents pointing at each other. If you cut the parent lose without unlinking the child they will stay in memory forever. I had to implement a destroy method for many of my objects. There is some sample code of this happening in this old thread about a Delete method. Mark jumps in at the end. http://www.blitzbasic.com/Community/posts.php?topic=50880#567962 |
| ||
| Where? Not in this thread. No. In the one before the last one if memory serves. which possibly extends the tlist type, which is retarded, because the point of my requesting that feature was to make my life EASIER, and making an extra type whenever I need a list is not making my life easier? You make it once. It's maybe 10 lines of code, all inclusive and will not break other peoples code. If BRL change the code now, it will break more than 200 peoples code. You on the other hand could override the object iterator to return a link instead of a value without breaking any exsisting code, (including most of your own). Uh, you do know that a TMap is a binary tree, and that unless you balance it when creating it A binary tree is always balanced when you create it. The problem you describe may arise only after millions of insert / remove operations, and only on specific data sets.While I would love to dicuss the merits of datastructures with you, I feel it's somewhat a moot point. If you're interested I suggest books by Mark Allen Weiss or Robert Sedgewick. Both have written excellent and exhustive books on the matter, although I think you'll find Wiess' books more accessible. But ofcourse knowing you prefer the difficult approach, I'm sure you'll pick up Sedgewick. But I bet it does involve creating your own type that extends the list type. Isn't that the whole point of the object oriented paradigm? Sswift kinda won the argument this time. Hrm. You're entitled to your opinion. To me it seems like Sswift doesn't really understand what he's talking about. |
| ||
| See you later when you get back from your ego trip. |
| ||
Sorry, I haven't read all you posts. But I would recommend the following to avoid Memory Leaks:Type YourType Field bIsDeleted:Byte Method Remove:Byte() If bIsDeleted = False Then 'Place your One-Time-Deleting-Actions here. 'For Example: Removing Object from global List. End If bIsDeleted = True Return Tru End Method Function DeleteCheck:Byte(objVarCheck:YourType) If objVarCheck = Null Then Return False If objVarCheck.bIsDeleted = True Then objVarCheck = Null End If Return True End Function End Type Now all you have to do is checking every Link to an YourType once a Frame with Deletecheck. Simply "DeleteCheck(YourLink)". If you also want to know, whether the Object was Deleted in this Frame or not, use this: Local objTemp:YourType = objRealLink2Check If YourType.DeleteCheck(objTemp) = True And objTemp = Null Then 'Now do your Deleting-Actions you need objRealLink2Check for. objRealLink2Check = Null End If If you don't need the Link for you Final Action, the following will do: If YourType.DeleteCheck(objRealLink2Check) = True And objRealLink2Check = Null Then End If |
| ||
Why do all that when you intend to do this :-If objRealLink2Check.bIsDeleted objRealLink2Check = Null EndIf Besides, this really doesn't solve the issues of multiple references to a single object Fetze - the GC won't destroy the object until there are no references to it. |
| ||
| If you cut the parent lose without unlinking the child they will stay in memory forever. I had to implement a destroy method for many of my objects. I don't think thats true. Once you delete the child, both of their memory will be restored. |
| ||
| I think it would be nice to have the option of using a delete command - no-one ever had a problem with it in the old Blitz's and it would be nice to have the option of pointers to types back. |
| ||
| I think this topic would benefit from a little code showing the problem. Then, rather than everybodying pitching in with what they think might/should/could happen somebody might provide a solution and/or explanation. |
| ||
| You make it once. It's maybe 10 lines of code, all inclusive and will not break other peoples code. If BRL change the code now, it will break more than 200 peoples code. You on the other hand could override the object iterator to return a link instead of a value without breaking any exsisting code, (including most of your own). Oh, I'm not suggesting Mark change how EachIn itself behaves, that would, as you say, break everyone's code. It's much too late to correct that. However, it is not too late for Mark to implement a new command like Each, or EachInList that would allow us to iterate through lists properly. A binary tree is always balanced when you create it. The problem you describe may arise only after millions of insert / remove operations, and only on specific data sets. I'm not convinced... I looked at Mark's code, and considered what would happen if I inserted a mess of objects that were already sorted in order, and unless I was mistaken, what I found was that you'd get one at the top, then one to the left of that, then one to the left of that one, and one to the left of that one, and so on. If the list was using memory addresses for the compare, it seems likely that there too you would have a problem because now you'd mostly get numbers in increasing order. One Eyed Jack: I don't think thats true. Once you delete the child, both of their memory will be restored. "Delete" the child? You can't "delete" anything in Max. You have pointers to stuff, and if you lose a pointer the object is freed, unless something else points to it. In this case, you lose the pointer to the child and the parent, but both the child and the parent have pointers that point at one another, and they are never freed as a result even though you yourself now have no idea where they are in memory and think they have been freed. The only way to solve that is to set the internal pointers for one or both to Null so they do not point back to one another. If you set the child's parent pointer to Null for example, the Parent will be freed because it has nothing pointing at it anymore. Then the Child will be freed, because the Parent was the only thing pointing at it. But I bet it does involve creating your own type that extends the list type. Isn't that the whole point of the object oriented paradigm? No, the whole point of the OOP is not for everyone to have to correct flaws in a language so the creator doesn't have to fix them in a way which will then be standardized. |
| ||
| Years ago I did an experiment in C++ for DOS. I made a shoot'em up game with objects and benchmarked it. Then I converted it arrays only and benchmarked it again. The old-school arrays were MUCH faster. If you've got a lot of game objects, such as particles, just reseting the values and switching them on/off instead of creating/destroying is bound to save time... |
| ||
| Grey: Yeah but that was years ago in DOS. PC's are so fast now that while yes, it may be twice as fast to use arrays, you're talking about the difference between a hundredth of a millisecond, and TWO hundredths of a millisecond. It's just not worth optimizing certain things these days. If I was trying to animate a million particles at once, sure there might be a significant difference in speed. But a thousand? Hell no. |
| ||
| Exactly. Trying to apply an optimization from 5+ years ago isn't relevant. It's like trying to argue that fixed point math and unrolling loops are still worth it. |
| ||
| don't worry that was then, this is now. I've been writing windows software professionally for 9 years! However, in 1996 when I wrote a report generator, I DID use Integer maths to perform the scaling on the print preview to avoid loads of floating point maths and it was pretty quick :-) readability and reuse is much more important, even if your code runs slower (not that likely). |
| ||
| Removing types and using arrays instead may make your processing code faster, but in game and graphics programming, you're still limited to you video card's top speed. For example, if 10000 sprites cost you 2ms processing time and 20ms rendering time (total of 22ms), what good will it do you if you reduce the processing time to 1ms (total of 21ms vs 22ms)? There's always a sacrifice for readibility/performance, but if 1-2 milliseconds are that important to you, you may as well go back to machine code. However, there are cases where even a microsecond makes a huge difference, when processing millions of segments of data (like in MaXML 2, which is capable of parsing roughly 5,400 characters each millisecond); but in a game world with 100 - 5,000 game objects, it makes almost no difference whether you use arrays or types. |
| ||
| yes, however, I tried to make my bonus games run at the current user's refresh rate, so if it was 85Hz, my game didn't drop any frames. This only gives you 11.8ms to play with each frame and on a slow machine that 1ms time saved may actually be 4ms! I optimised the hell out of Xmas Bonus, shaving off 1ms here and there until it could run on pitiful machines. |
| ||
| Yeah. It all depends on your specific application. I always try to optimize as much as possible, unless one specific optimization would seriously compromize the structure of the code. |
| ||
Type tOne Field two:tTwo = New tTwo End Type Type tTwo Field three:tThree = New tThree End Type Type tThree Field Parent:tTwo Field LotsOfMemeory:Int[500] End Type While Not KeyHit (KEY_ESCAPE) Local one:tOne = New tOne one.two.three.Parent = one.two ' uncomment this to see gc working correclty ' one.two.three.Parent = Null Print GCMemAlloced() Wend As far as i can tell this program eats memory untill it crashes. Uncomment one.two.three.Parent = Null so that chains of memory will not be created. In fact... it crashes in both versions for some reasion. Perhaps you guys who know your stuff can tell me what im doing wrong. |
| ||
| Apparently, from what i have read on the forums, BlitzMax doesnt return the memory to the OS until shutdown. It just reuses the memory. |
| ||
| and unrolling loops are still worth it. Watch it, you will get FlameDuck started on his "unrolling loops" tangent. |
| ||
| However, it is not too late for Mark to implement a new command like Each, or EachInList that would allow us to iterate through lists properly. The thing is that EachIn is a geneic iterator. You can use it on other data structures than Lists (Arrays and TMaps in the default BlitzMAX library, I have ones that itterate through a shortest path tree, a minimum spanning tree, and a bressenham line), so you can apply it to whatever datastructure you need at any particular time.I realize it doesn't do exactly what you want in this specific case, but not all data structures have nodes that come in a meaningful sequential order, so while your Each iterator idea makes perfect sense from a list reference it doesn't work too well on a graph or a tree (what kind of traversal should it use). Should he also add EachInner and EachOuter to iterate through binary trees? What about graph traversal? EachHorizontal, EachVertical, EachShortestPath, EachMinimumSpanning, EachAStar? That's the wonderful thing about EachIn. It can work whichever way you want it to work, rather than whatever way Mark thought would be most useful. I looked at Mark's code, and considered what would happen if I inserted a mess of objects that were already sorted in order Yep. That would be a specific data set. If you already had the data sorted tho', and you never needed to change it, you could just aswell keep it in whatever structure you already have it in, rather than inserting it into the BST. Perhaps you guys who know your stuff can tell me what im doing wrong. Depends on what you're trying to do, outside of prove the BlitzMAX Garbage collector doesn't handle cyclic references. Apparently, from what i have read on the forums, BlitzMax doesnt return the memory to the OS until shutdown. It just reuses the memory. From what I understand it retains it for some time, then releases it if there hasn't been a new request for memory in the meantime. Getting and freeing memory are fairly expensive (slow) operations on most operating systems. Watch it, you will get FlameDuck started on his "unrolling loops" tangent. What do you mean? I agree with Warren on this particular issue. For the most part, unrolling loops really aren't worth it. Smaller loops will fit comfortably in the instruction cache, larger loops have bigger bottlenecks (like memory access) than jump instructions and cache misses. |
| ||
| "Perhaps you guys who know your stuff can tell me what im doing wrong." Well first, I didn't even know you could initialize a field, nevermind like this: Field two:tTwo = New tTwo I'm not sure what that does. Does it create a new instance of "tTwo" and assign that pointer to the field "two" every time you create a new instance of "tOne"? Or does it create a single instance of "tTwo" when the program starts and assign the pointer to that to the "two" field every time you create a new instance of "tOne"? I'm not sure, but I'm gonna assume it makes new instances each time. May not be important for the purpouses of figuring out the problem here though. As for why it allocates memory until it crashes: one:tOne = New tOne At this point, you have one instance of tOne, one instance of tTwo, and one instance of tThree, because instancing tOne caused an instance of tTwo to be created, which caused an instance of tThree to be created. They point to one another like so: tOne -> tTwo -> tThree Then you do this: one.two.three.Parent = one.two Which makes the tThree instance you created point BACK at the instance of tTwo. This means that you now have this:
tOne -> tTwo -> tThree --.
^--------------'
The next time through the loop, you create a new tOne instance, and you set your "one" pointer to point to the new instance. That means the pointer to the old instance is lost. So the old instance of tOne gets deleted. But the old tTwo and tThree don't get deleted. Why? Becayse tTwo points to tThree, and tThree points back at tTwo. So each thinks you have one pointer left that you can access, and the automatic garbage collector never frees them. When you do this: one.two.three.Parent = Null You remove the pointer in the old instance of tThree that pointed back to two. That means that two now has no pointers pointing to it, and it is freed. And when two is freed, the pointer it had to tThree goes with it, and that means tThree no longer has any pointers pointing at it. So it is then freed as well. So the problem with your code is the one I was complaining about. Cyclic references (two pointing to three which points back to two) causing the automatic garbage collector to screw up. |
| ||
| Yeah.. i know that stuff, thats why i posted it. I was giving tonyg an example. What i meant when asking about the crash is that this crashes even when you null the parent field in one.two.three and the garbage collector should be working correctly. Khomy, can you explain the difference between giving the memory back to the os and releasing it? In my example the memory footprint grows and grows. What happens when you are out of memory. |
| ||
| What i meant when asking about the crash is that this crashes even when you null the parent field in one.two.three and the garbage collector should be working correctly. Your program "crashes" MaxIDE (the program itself is doing just fine) for reasons unrelated to the garbage collector. Your program crashes because of a flaw in the way BRL have implemented RichEdit, and you've sent more than 64K of text to the RichEdit control. Run it from a console instead (don't forget to disable 'Build GUI App'). If you want to 'uncrash' MaxIDE, kill off your program with the Task Manager to recover RichEdit. Khomy, can you explain the difference between giving the memory back to the os and releasing it? Imagine there are three different pools of memory: we'll call them, the heap, the memory pool, the OS.Now your program wants to allocate memory for a new type. So it looks to the memory pool, which is empty. The memory pool then asks the OS to allocate it a fairly large block of memory (slow). If it has any, it does so. Now the memory pool has some memory to give to the new object, which is then placed on the heap (fast). Your program continues setting up stuff (creating new objects), and the memory pool keeps giving more memory to the heap (fast). Eventually it'll run out, so it'll ask the OS for more memory (slow). Then when the garbage collector detectects an orphaned object (or rather one with no references) the memory an object uses is moved back to the memory pool (fast). If the memory pool is using up more memory than reasonable, some of it is released back to the OS (slow). I don't know if that's the exact way BlitzMAX works, but it seems to be the prevailent theory. What happens when you are out of memory. You get an out of memory exception. |
| ||
| Well first, I didn't even know you could initialize a field, nevermind like this: Field two:tTwo = New tTwo I explored this concept here: http://www.blitzbasic.com/Community/posts.php?topic=58419 It DOES make a new instance each time (I tested this). It would only make once instance if it was a GLOBAL instead of a FIELD. |
| ||
| FlameDuck :"Imagine there are three different pools of memory: we'll call them, the heap, the memory pool, the OS." I just want to point out that the way you describe this is actually misleading. The memory pool is actually memory allocated from the process's heap. BM's memory management system then uses this pool to allocate space for BM objects, because as Flameduck points out, it's mostly quicker to manage your own pool of memory than to use OS calls to allocate new memory. |
| ||
| Which is why the GC should NOT lower the pool size unless the RAM is filled up ... At the moment, GC loses performance due to that "memory saving" behavior ... (want to have low mem usage even if there are 600MB free?!?!) |
| ||
| So does this app really eat memory then? I think this is a bit of a problem, the whole point of having automagic memory management is that a novice programer need not concern themself with freeing objects or even really think about memory. I think it would be fairly eaisy for a new programmer to create a circular refrence like this and have thier app eat memory and have no idea why. |
   |