Serializing objects

BlitzMax Forums/BlitzMax Programming/Serializing objects

OremLK(Posted 2008) [#1]
I've just moved over to BlitzMax from XNA, and while I'm generally finding things nice here, one thing I took for granted with XNA and .NET was its capability to automatically serialize your objects (file i/o). It was nice being able to literally plug in an array of objects--as in, of a user-defined type--and have it automatically spit out a file containing all of their data.

With the way BlitzMax OOP works, I can't really think of how I could do this myself (in a generic, one function for all object types kind of way), but admittedly I'm not that great of a programmer. So I was wondering if anyone else has come up with a way of doing this, or just a module that does it.

Thanks!


tonyg(Posted 2008) [#2]
From 1.26 Blitzmax includes reflection. You would be able to use this to write your own function to do that.


GW(Posted 2008) [#3]
This may be a start
http://www.blitzbasic.com/codearcs/codearcs.php?code=2132#comments


OremLK(Posted 2008) [#4]
Thanks, I'll see if I can figure it out from there. If I get something working, I'll post it here.


grable(Posted 2008) [#5]
I did a Serializer a while back you could look at, it isnt complete though... but it works quite well imo =)
ReflectionSerializer


OremLK(Posted 2008) [#6]
Excellent! Thanks for posting it. Looking at your source, is the incomplete part the lack of array support?


grable(Posted 2008) [#7]
Looking at your source, is the incomplete part the lack of array support?

Yeah, the JSON codec has partial array support though.


Raph(Posted 2008) [#8]
grable, this was really handy! But I ran into problems with my particle system while deserializing -- most values were read in correctly, but it broke on the first float and exited early, then after fixing that, it broke on the first negative number. So here are two minor fixes to this serializer to get it to load floats and negative numbers.

In Codecs.bmx, change line 231 to this:

		ElseIf (c >= Asc("a") And c <= Asc("z")) Or (c >= Asc("A") And c <= Asc("Z")) Or c = Asc("_") Then


which removes "c =Asc("-")" so that tokens starting with - are read as numbers once you also change line 242 to

		ElseIf (c >= Asc("0") And c <= Asc("9")) Or c = Asc("-") Then


Finally, there was code in there to handle floats, but it was in a while loop that would never parse a period. Change line 246 to

			While (c >= Asc("0") And c <= Asc("9")) Or c = Asc(".")


Works great now, thanks!


grable(Posted 2008) [#9]
Glad you found a use for it, and thanks for the fixes =)
Ive updated the archive with the latest changes as well.


Macguffin(Posted 2008) [#10]
I've had good luck so far with Otus' save code: http://blitzmax.com/codearcs/codearcs.php?code=2262

The only limitation I ran into was that, since reflection doesn't handle multi-dimensional arrays, they don't save properly.


Jim Teeuwen(Posted 2008) [#11]
I've been fiddling with this since reflection came into the language and wrote a little framework with 2 serialization implementations.

You may find them useful. Although they basically do the same as the examples posted above, my code adhere's to a somewhat more OOP compliant code model and it allows you to implement any number of output formats for your files. Included in the codearchives is an ASCII formatter and BinaryFormatter. If you care for it, I also have an XML formatter for you. This one does require a third-party XML module to work though, so I left it out of the code archives.

http://www.blitzbasic.com/codearcs/codearcs.php?code=2311


MGE(Posted 2008) [#12]
Jim T - I'm a noob to serialization, I use TList to manage particles, sprites, etc. Can your code serialize dynamic Tlists of objects as well? If so, could you please elaborate how I might save/load a current TList of particles for instance? Thanks.


Jim Teeuwen(Posted 2008) [#13]
It currently doesn't deal with Tlists or TMap's. I didn't need this functionality when I wrote this stuff.

It shouldn't be to difficult to add a tlist though, provided the elements stored in the list are all the same (No combination of int's objects, floats, strings etc). The implementation is much the same as what I used to load/save arrays.. The only difference would be that you do not have to pre-initialize a tlist to a fixed amount of elements as it does in the code.
You will have to add a separate case (TYPETKN_TLIST) for TLists though.


MGE(Posted 2008) [#14]
Ahh... ok. Thanks for the reply. This might be harder than it seems. M y Tlists all contain the same object type, but each object is mass definition of floats, ints, tlists, etc, etc. I think I would have to write something custom for this.


Brucey(Posted 2008) [#15]
Can your code serialize dynamic Tlists of objects as well?

Yes.
It doesn't care what types are in the list.
It also handles cyclic references.

Basically you get a snapshot of time. (barring current reflection inadequacies)


MGE(Posted 2008) [#16]
Brucey - This is available in one of your mods?


Grey Alien(Posted 2008) [#17]
(barring current reflection inadequacies)
such as? I guess there's a thread about it? ...


plash(Posted 2008) [#18]
such as?
Multi-dimensional arrays? (as mentioned previously)


Grey Alien(Posted 2008) [#19]
Oh sorry. OK thx. Well that does sound like a bit of a problem, considering how many games are tile-based ...


Brucey(Posted 2008) [#20]
Thought I'd have a go at fixing the reflection module.

A basic test :-)
' the type
Type TObj
	Field numbersi:Int[,] = New Int[4,4]
End Type

' example serialized
<bmo ver="2"><TObj ref="089VMG"><field name="numbersi" type="array:Int[,]">0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15</field></TObj></bmo>

Seems to have passed serializing and deserializing.


Grey Alien(Posted 2008) [#21]
With that example, how can you tell the dimension sizes in the output?


slenkar(Posted 2008) [#22]
I hope your fix goes into the official BRL module!


Brucey(Posted 2008) [#23]
With that example, how can you tell the dimension sizes in the output?


In my persistence module, it doesn't matter, since all basic type arrays store a default value (like zero) if not set.
And the way in which arrays work in BlitzMax, it doesn't matter there either, since a 2 dimensional array is stored as one long list of entries.

I've also managed to get things like this working too :
	Field arrarr:Int[][] = [[1,2], [3,4]]

...which is nice, I suppose.

..during which time I found *another* bug in the reflection code :-p


Brucey(Posted 2008) [#24]
For those interested in such nerdy things, I've posted a new version of the Reflection module (with multi-dimension array support) here, for testing.


Grey Alien(Posted 2008) [#25]
But then when loading back in the array how do you know what sizes the dimensions are? I'm confused...


slenkar(Posted 2008) [#26]
I think you can do this

local my_array[6]

print my_array.length

if thats what you mean?


MGE(Posted 2008) [#27]
We need Brucey to write a serialization tutorial for dummies. :) I'm confoosed....


Grey Alien(Posted 2008) [#28]
@Jeremy Paxman: Yeah I know that, I saying if you have a 2 dimensional array of say dimensions 4,4 and you output it via serialisation as a 16 number long array, when you read it back in, you don't want it as a 16 number long array, you need it back in a 4,4 array - but how can you do that if the serialisation doesn't store the dimensions? You'd have to output the dimensions yourself...


Brucey(Posted 2008) [#29]
You'd have to output the dimensions yourself...

We probably don't want to have to do that :-)


Grey Alien(Posted 2008) [#30]
We probably don't want to have to do that :-)
Agreed!


Brucey(Posted 2008) [#31]
Updated my BRL.Reflection module hack to enable access to array dimension values, as well as creating multi-dimensional arrays through reflection. (currently available from Module Tweaks area).


Grey Alien(Posted 2008) [#32]
Awesome. Sounds like BRL should make it official...