NewsForumsCodeLogsGallerySpecsSearch

serialization question

BlitzMax Forums/Brucey's Modules/serialization question

slenkar(Posted 2008) [#1]
Can the serialization module keep track of references to other objects within fields?

E.g.

Type box
field owner:character
endtype

or how about lists of objects within fields?

e.g.
type box
field item_list:tlist
endtype

how about 2 dimensional arrays? with a list in each cell

and where is the download?

cant find it here:
http://code.google.com/p/maxmods/downloads/list


Brucey(Posted 2008) [#2]
Yes, yes, and no.

Any references to other objects in the object that you are serializing will be serialized too. On de-serializing, you will end up with exactly the same structure as you had before you serialized - albeit the object id's will be different. (where the object id is it's memory location).

All I can say really is, try it and see ;-)

Blitz's Reflection doesn't currently support arrays of more than one dimension, alas - so I'm bound by that too.

Oh, and it also handles circular Object references :-)


slenkar(Posted 2008) [#3]
thanks,
I looked here for the download but couldnt find it
http://brucey.net/programming/blitz/
http://code.google.com/p/maxmods/downloads/list

is the google protocol buffers any better?


Brucey(Posted 2008) [#4]
Yeah... it's currently available via SVN... but here's a zip of the latest source : persistence.zip (5.9 kb).

There's only one example currently, but it should give you and idea of what it can do.

:o)


slenkar(Posted 2008) [#5]
thanks


Brucey(Posted 2008) [#6]
Oh, you will need the libxml module btw... (for the nice xml output to work) :-)


slenkar(Posted 2008) [#7]
i had a look at the example...

how would you save and load some characters and their items?

e.g.

type character
field name$
field item_list:tlist
endtype

type item
field name$
endtype


Brucey(Posted 2008) [#8]
This is a small example showing how to serialize the object to a String



Brucey(Posted 2008) [#9]
You can also create a "TStream", and have the object save into the stream instead.

The module supports all writeable streams.

To use it this way, you need to create a TPersist object, and call the SerializeToStream() method :

Local pers:TPersist = New TPersist

Local stream:TStream = WriteStream("example.bmo")
pers.SerializeToStream(the_object, stream)

stream.Close()


You can also use SerializeToFile(), which takes a filename as a param.

Hopefully that covers most options? :-)


slenkar(Posted 2008) [#10]
yeah I managed to save the game succesfully! this is a good module

not sure about loading though


EDIT-
I serialized each character in my character-list,
in the BMO file it says

<?xml version="1.0"?> every time there is a character(game character e.g. a person), (15 times)

this throws an error when I try to deserialize
"there should only be one reference to <?xml version="1.0"?> in the file"

EDIT- Im going to try serializing a list instead of individual objects

EDIT-
Hmm it seems you cant serialize a list

What now?
>


So, to sum it up, how do you save more than 1 character?


slenkar(Posted 2008) [#11]
I put my character list inside an object and it worked!

good module man


slenkar(Posted 2008) [#12]
hmm I played my game near the start saved and loaded....and it worked,

I played the game a bit further SAVED and LOADED
and it threw a lot of errors like this

Entity: line 2: parser error : Couldn't find end of Start Tag String line 2


looking at the error and the save game file, it happens when someone writes 'lines' on to a piece of paper in the game.

The words on the paper are represented by a list of arrays

every line of text on the paper is an array of words

this must be where it is messing up.

The arrays have only 1 dimension though.



I can see the errors:

lines that look like this are wrong
<field name="_value" ref="33EK1K" type="Object"/>


they should have 
</field>
on the end
here is the paper object when it is saved:



Brucey(Posted 2008) [#13]
That isn't a whole file is it?

What method are you using to save?


slenkar(Posted 2008) [#14]
no its not the whole file, just the code for an item in the game

im adding all game items to a list inside a type object

then im 'serializing to stream' the object


Brucey(Posted 2008) [#15]
I have trouble believing that it doesn't work for one item in a large list... it would either work... or not - generally.


slenkar(Posted 2008) [#16]
its every time 1d arrays get added to a list


Brucey(Posted 2008) [#17]
If you want, you can mail me a complete xml example that is broken, and I'll try to work out what might be going wrong.


slenkar(Posted 2008) [#18]
I tried to make an example program but it creates a weird error:
this is probably related to the problems I am having

'save
Import bah.persistence
Import bah.libXML

Type save_object
Field list:TList
EndType

Function savegame()
	Local s:save_object=New save_object
	Local array$[5]
	s.list=New TList
	s.list.addlast(array)
	Local pers:TPersist = New TPersist
	Local stream:TStream = WriteStream("savedgame1.bmo")
	pers.SerializeToStream(s, stream)
	stream.close()
EndFunction

Function loadgame()
	Local pers:TPersist = New TPersist
	Local stream:TStream = ReadStream("savedgame1.bmo")
	If stream<>Null
		If pers<>Null
			Local t:save_object=save_object(pers.DeSerializeFromStream(stream))
		EndIf
	EndIf
EndFunction

savegame()

loadgame()



Brucey(Posted 2008) [#19]
Hokay, I see the problem... leave it with me.
<?xml version="1.0"?>
<bmo ver="1">
  <save_object ref="037KY0">
    <field name="list" type="TList">
      <TList ref="037KZS">
        <field name="_head" type="TLink">
          <TLink ref="037L08">
            <field name="_value" ref="037L08" type="Object"/>
            <field name="_succ" type="TLink">
              <TLink ref="037L14">
                <field name="_value" type="Object">
                  <String[] ref="037KYG"/>
                </field>
                <field name="_succ" ref="037L08" type="TLink"/>
                <field name="_pred" ref="037L08" type="TLink"/>
              </TLink>
            </field>
            <field name="_pred" ref="037L14" type="TLink"/>
          </TLink>
        </field>
      </TList>
    </field>
  </save_object>
</bmo>



Difference(Posted 2008) [#20]
I don't know if this is related, but both Firefox and IE clams [] is not allowed as part of the XML tag name, so String[] should maybe replaced with something else (or encoded in some way) to allow editing/viewing/transforming with common XML tools?

[EDIT] Reading up on this, using square brackets in XML *does* seem to be allowed, so maybe it's an encoding problem, and the XML files sould be saved with <?xml version="1.0" encoding="UTF-8"?> or something?

But: I tried changing the incodeing in in Notepad++ and adding encoding="UTF-8", but that did not help.


Brucey(Posted 2008) [#21]
I've committed a new version of the persistence module, which now properly supports Array Objects (rather than simply Array Fields).

Turns out that the Reflection uses object identifiers such as "String[]" to indicate a string array. My code was (foolishly) just passing that through to the XML, which libxml refuses to parse.
It now determines whether the object is an array, and stores it properly, using the tag "_array_".


@Peter : "UTF-8" doesn't need to be specified, as it is assumed that the file is in that format unless you specify otherwise - for example "ISO-8859-1".


@Jeremy : You need to remember to close your output stream. If you don't it remains open until 1) your app closes, 2) you run out of file handles and your app crashes. ;-)


Thanks for the beta testing guys :-)

Unless there are any other major outstanding issues, I suppose this module looks about ready to go for Release.


Difference(Posted 2008) [#22]
Okay it seems to work now, Cool

[EDIT] Took out example, testing some more...


Brucey(Posted 2008) [#23]
Interesting.. This is what I get, with TPersist.format = true ...
<?xml version="1.0"?>
<bmo ver="2">
  <save_object ref="2GGQ9K">
    <field name="list" type="TList">
      <TList ref="2GGQBC">
        <field name="_head" type="TLink">
          <TLink ref="2GGQBS">
            <field name="_value" ref="2GGQBS" type="Object"/>
            <field name="_succ" type="TLink">
              <TLink ref="2GGQCO">
                <field name="_value" type="Object">
                  <_array_ ref="2GGQA0" type="String" size="5">
                    <val></val>
                    <val></val>
                    <val></val>
                    <val></val>
                    <val></val>
                  </_array_>
                </field>
                <field name="_succ" ref="2GGQBS" type="TLink"/>
                <field name="_pred" ref="2GGQBS" type="TLink"/>
              </TLink>
            </field>
            <field name="_pred" ref="2GGQCO" type="TLink"/>
          </TLink>
        </field>
      </TList>
    </field>
  </save_object>
</bmo>

...which is what I'd expect.


Difference(Posted 2008) [#24]
Sorry! It seems to be Notepad++ that is confused. Looking again it seems all good.

Should there/is there - an BOM in the beginning of the file even if it is optional? ref: http://www.opentag.com/xfaq_enc.htm?

It seems Notepad++ thinks it's an ANSI file.


Brucey(Posted 2008) [#25]
Libxml is kind enough to escape non-ascii characters, so if you were to say, include chinese text somewhere, you would would get encoding like this :

<hello>&#24744;&#22909;&#24444;&#24471;</hello>

(thanks to the forum for encoding this example for me ;-)

The BOM is only useful, I suppose, if you intend having non-ascii characters in the file somewhere.


Difference(Posted 2008) [#26]
Makes sense. Thanks for taking the time to explain.


slenkar(Posted 2008) [#27]
thanks for the error-fix