BMax module: ChaosClone
Community Forums/Showcase/BMax module: ChaosClone
| ||
| Hi there, just wanted to present you my latest project: ChaosClone It's a module that allows for nearly universal saving, loading, cloning and combining of user defined objects. It is capable of automatic handling for arrays (also multidimensional and arrays in arrays), TBank, TList, TMap, TImage & TPixmap. All operations are managed by different metatags you add to your objects. You can combine objects and arrays and decide which fields from the objects are taken from source or target. Also, cloning of objects and their subobjects is an easy task. Repeating references are recognized and replaced to keep files small. Last not least Chaosclone offers the ability to call user defined methods for loading / saving / initializing if a data cannot be aquired with 'onboard tools' (which would be the case for pointers and arrays of arrays as reflection does not handle with them). The module is precompiled for Win, Mac and Linux and documented fairly well. If there are questions regarding how to use the module or if you detect errors feel free to contact me. ChaosClone is closed source donationware - if you like it and use it I would ask you to donate a little. Also i ask you to add credits in your app if you use it. Where to get it? http://www.chaos-interactive.de in the modules section. OR https://svn.blitzforum.de/chaos.mod ...and if you like it:  Thank you. |
| ||
| Except for the cloning part, in what way would this differ from the serialization module from Brucey (assuming his module doesn't do cloning)? Since reflection is being used, how good is it performance wise? If targeting applications it's OK if cloning a lot of objects takes some time (I'd say) but for a fast paced shooter or something it should be very fast. For the module being closed source (which is fine), do remember you're going to spend a lot of time compiling it for every BlitzMax release and keeping it up to date with BlitzMax. BlitzMax is being updated quite frequently for the last year. |
| ||
| As far as i can say it's quite some difference to Bruceys module as he uses xml to store the data where my approach does save the data directly (bytewise). I didn't for shure reinvent the wheel. I can't tell you how it will perform with real lots of objects - but I invite you to test it if it fits your needs. I must admit I didn't think about performance upon creation, but I think it should be no problem to do some 'pre-level-creation' of objects. My main target was to make it as universal as possible while trying to keep small files. The cloning and combining where sideeffects I achieved as I wrote the savin' stuff. If there should be any problems with the module because of newer Bmax-Releases I will update it for shure, in fact I'm still working at the module and trying to optimize it. If you have any problems with the Module I will try to help. As this is a complex theme I can't gurantee its free of all bugs, but the testings on all three OS were promising. I hope I could answer your questions and understood you well, because my english is not very good I'm afraid. After all, I don't want to sell broken stuff, this is why i made it donationware - if you are happy with it it would please me if you show it. If you are not happy I would be glad to know why so i can make things better. |
| ||
| Sorry for the double post. I took a look at the serialization module and what i found at the forums about it (I never used it). Ine one thread from about a year ago brucey said his module only handles one-dimensional arrays. My module handels multidimensional arrays[,,..] and arrays in arrays[[]]. Only arrays of arrays [][] are out. Also, I build in the ability to add external load and save methods if chaosclone fails on a task, so you can just extend it to fit your needs. I was not able to find a working download for bah.persistence, so i can only guess if bruceys module has this functionality too. Also, the used tags can be altered to perform multiple different tasks on the same object. Just give it a try and see if it satisfies your needs. EDIT: I read that brucey added multidim support, too. |
| ||
| I'm pretty sure you can use Brucey's persistence module to write/read data in any form you see fit. |
| ||
| As I said before, I dind't use his module before, but what I read says it uses XML. I don't want to discredit his work, I can't even judge it as i -like mentioned above- wasn't able to find a download (it is described at code.google.com, but no links). Use what you think will do it best for you, it's as easy as that. I never said my module was revolutionary better than others. I said it is there. |
| ||
| Hello BladeRunner, I did some speed test to compare bah.persistence mod and yours one. But it looks like your mod fails saving to stream. All fields marked with {clone} are empty after loading. |
| ||
| Sorry for my late answer, I must have overseen that there was a reply. I will look at your code and answer again later on. |
| ||
| Ok, there have been two different issues: First, there were usage-errors: - You declared the fields with clone only - this clones them, but it does not save them. - You did not assign the result of LoadUDObject to a type instance. After changing this, all data gets saved correctly to the stream (which i proofed with an HEX-Editor). Second, there was a bug I must have overseen as I rewrote the array-saving which caused primitive arrays no longer got loaded, whereas object ones did fine. I found the bug and fixed it and soon will upload the new version for testing. Your example jusgt did fine for me after that. Please take my apologizes that it did take so long to answer, and thanks for your bug report. I just add the 'corrected' example here: PS: The object-arg in LoadUDObject will be obsolete (optional) in the coming version of ChaosClone. It is only there for the purpose of combining of two objects. Please see the command reference on chaos-interactive.de for further info on combining. I will now get to my mates for compiling of the mac and linux- modules and then will update the downloadlink. Thanks again for testing. EDIT: Win32 and Mac Versions are up - see on www.chaos-interactive.de Linux will follow asap. |
| ||
| this could be what I need, if there is an array of 1000 elements and most of them are empty it doesnt save the empty elements? also if there is an object of 1000 fields and most of them are null it doesnt save the null fields? I tried it out... what am I doing wrong?
Import chaos.clone
Type Ttest {wholesave}
Field path:String
Field obj:Object
Field strin:String
Method save(url:Object)
Local s:TStream = WriteFile(url)
'do save stuff
s.close()
End Method
Method Load(url:Object)
Local s:TStream = ReadFile(url)
'do load stuff
s.close()
EndMethod
End Type
Local o:Ttest=New Ttest
o.strin="greetings"
SaveUDObject("hello",o)
Local obj:Ttest=Ttest(LoadUDObject("hello"))
Print obj.strin it should print 'greetings' but it doesnt |
| ||
| ok, The linux version is up now. @slenkar: Wholesave is by now fieldwise, not typewise. This means the content of the field marked with wholesave gets stored completely, no matter what tags are in it's metadata. I will however consider to implement it typewise for simplicity. Also the load() and save() methods are only needed if you have stuff that cannot be stored by the module itself (pointers, globals...). Regarding your Questions: Primitive (int, byte, float ...) Arrays are always stored as whole. Objects which are NULL get stored on a 1-byte-base, no matter if they are a field of an object or in an array. (This is necessary as the loading routine can't know if a field will be null, so it searches for a status byte.) By now the savefiles are streight forward. I am working on a compression for the save files, which also would offer some sort of encryption to the files. This may take some time as my work on this is in the beginning by now. |
| ||
| thanks for the help. is there a chance of getting an option to save all fields except those with a {nosave} tag? Thats what I used before with bruceys mod and that was very convienient my main problem is that I have some large arrays (to speed up the game) and deserialization is slow because every element of the array is deserialized even if it is zero. compression wouldnt speed up deserialization, (as you probably know) if an array element is zero, does the code put a zero in the array or does it just leave it alone? also when I change my program i dont want to make my previous saved games incompatible, so does chaos clone check for the existence of a field before trying to deserialize? When I remove an objects field the save game doesnt work unless I manually add in a bit of code to check the existence of the field in bruceys mod. thanks for your efforts |
| ||
| Hi, it should be possible to add sth. like a 'save them unmarked'- flag, will add it on my ToDO-list. As with the arrays, it depends greatly on what they consist of. Primitive arrays will be stored and loaded as whole, no matter what the contents are. This is due to the fact that the loader can't know what a field would contain in front of loading it. Also, checking a field if it is '0' before writing it would be slower than just write it. If, however, the array consists of a user defined type / object like tmap, tlist etc. then the loader will check for a null-identifier. If this is found then the loader returns null, without further cycling through the objects internals. I can't think of a way to speed this up more as the loader works recursive and steps through all elements it would expect from the original object. also when I change my program i dont want to make my previous saved games incompatible, so does chaos clone check for the existence of a field before trying to deserialize? As CC works with reflection, it assumes the Typestructure in the source that uses it. So- if you change type fields in your source, you will change what CC is looking for. BUT: You can define as many load/save-Tags as you like (if there aren't any restrictions in brl.reflection I am not aware of) and therfore alter the load/save-behaviour the way you like. You could store a version number in front of the rest of your data and use an init method to set the appropriate metadata for this version. A bit tricky but it should work without problems. This code is untested but should do the job. EDIT: One thing is to be payd attention to: The order of fields within the type DOES matter. If you change the order while reaaranging your code you will screw loading of old savestates up. This is because of the way reflection works. If you keep this single point in mind there shouldn't be problems. |
| ||
| This is due to the fact that the loader can't know what a field would contain in front of loading it when you create an array its elements are all zero or null |
| ||
| yes, they are. But the loader still can't know what would be in the single slots. If you want the array to be saved as totally empty, set it to NULL, it will be saved as a null-reference then. In every other case, even if only one element of the array is set to an other value, you will have to store it as whole. All solutions that will search for 'subarrays' of null would be very complex and resourceconsuming i think. If i store a 'is not null' in front of every array element, I would double the place it needs in the worst case, and ther would be no increase in speed. Using some sort of RLE would be possible but a pain in the ass to achieve, as it would need further recursive action in them. Also, in lot of cases it would not reduce the ammount of data. Also it would be time-consuming to to store and load this way. I suggest you try if my module can handle arrays of your needed size fast enough. I'm afraid I can't offer you a solution much faster than this. |
| ||
| ok ill try it out, thanks |
| ||
| 1.80 online, Win32 only by now. Fixed a bug which sometimes caused CClone to crash if TMaps were saved. Also added threaded builds. I will upload Linux and Mac builds asap. |
   |