NewsForumsCodeLogsGallerySpecsSearch

BlitzXML - A XML function library

Community Forums/Showcase/BlitzXML - A XML function library

John J.(Posted 2005) [#1]
BlitzXML makes it easy to load, manipulate, and save XML files. BlitzXML is a library of functions for manipulating xml data, including a fast (parses roughly twice as fast as Microsoft Internet Explorer's XML viewer) xml parser and saver. XML is widely used anywhere from word-processors to level builders.

Included with BlitzXML_v1.71.zip is a function reference manual, and example code.

Update: Version 1.71 Released! Version 1.71 includes some misc. bug fixes (BlitzXML now loads and saves nodes in the proper order)
Update: Version 1.7 Released! Version 1.7 includes some misc. bug fixes.
Update: Version 1.6 Released! Now BlitzXML can load XML files with the standard "<?xml ... ?>" header tag. BlitzXML also saves it's XML files with a header for compatibility with other applications.
Update: Version 1.5 Released! Includes new features and minor bug fixes

Click here to download BlitzXML 1.71 (12 KB)


Dragon57(Posted 2005) [#2]
Excellent! Thanks!


Picklesworth(Posted 2005) [#3]
I can tell that this is useful...
My problem is I've got absolutely no clue what XML does :)


R0B0T0(Posted 2005) [#4]
Xml does whatever you want it to :) It's basically self-describing information, you can think of it as a human readable format for storing any kind of data (in our case for level data, positions, character stats, whatever)

It's nice if you like to tinker with your data files without having to write a custom editor. It also allows others to easily mod your game data (assuming you want that) or even code their own editors.

Just look at the example file, you'll get the idea. The trick is that xml doesn't do anything by itself, your program still has to make sense of what it is (kind of like data statements, but a lot better).

Anyways, thanks a lot John, very handy!


Ricky Smith(Posted 2005) [#5]
Thanks John - very useful indeed !


Beaker(Posted 2005) [#6]
XML rocks.


N(Posted 2005) [#7]
Xml is god. GOD!


Picklesworth(Posted 2005) [#8]
Ah, I see!
John, you're god.


HappyCat(Posted 2005) [#9]
XML is nothing and everything.

It's the simplest thing in the world - when you look at an XML file you think "is that all it is?" - it's just structured text files - nothing you couldn't do on your own (but having a standard makes them so much more useful).

And at the same time it's one of the most powerful and useful things ever. A standard, structured, human-readable, platform and application independent data format has been needed for years :-)


John J.(Posted 2005) [#10]
It's been 5 months since I've made any changes to BlitzXML, but I've finially spent a little time adding support for multiple simultaneous file support. Now you can load in as many XML files as you want at the same time!


I've tried to keep down the work for anyone using BlitzXML 1.0 who wants to switch to BlitzXML 1.5, so all you'll need to change is the xmlLoad code slightly, and remember to delete xml files before loading a new one, if you don't plan on accessing it later (just think of it as opening and closing files).

Loading an XML file now simply returns the root node of the file, allowing full access to all it's contents. You can then "close" these loaded XML files by calling the delete function on any file's root node.

Also, I fixed a bug where nodes were not being completely deleted, causing a small memory leak if you are accessing a large number of XML files.

Get the new version here (BlitzXML 1.5)



The old version is still availible here if anyone still wants it.


Bot Builder(Posted 2005) [#11]
Xml is O.K.

IMHO they made it too complicated. To create a truely compliant parser/emitter you have to write a ton of code. The stuff like dtd, namespaces, encryption, signatures, entitization, proper charachters used in names, and a very complicated schema of unicode support. Attributes are redundant with having child nodes (except in xpath where I suppose it is handy to have them). Also, all attributes, node internals are strings. No explicit typing of variables.

All of these factors makes proper xml api bloated and ugly. Which is why...

I'm going to write a markup language that looks similar to [url='http://www.yaml.org/']YAML[/url], only more of a subset of that, and include typing of node values. The idea is to allow writing a parser and generator in less than 500 lines of code.

Sure, it wont be as universal as XML but it will mirror programming objects in text beutifully.


Booticus(Posted 2005) [#12]
Awesome! I *JUST* started a few tutorials on XML. This is great! Thanks!


John J.(Posted 2005) [#13]
Booticus: I hope you get as much use out of BlitzXML as I have. Don't hesitate to post any suggestions if there's any features you would like to see in BlitzXML. Also, please let me know if you find any bugs (although hopefully there are none by now) :)

BotBuilder: I agree that the full XML specifications are a little unnecessary for most purposes for games. That's why I made BlitzXML. BlitzXML is definitely not bloated. From your post, I take it you haven't even taken a look at the BlitzXML documentation or examples. In fact, the BlitzXML "api" is very simple and easy to use. It has features for nodes, their attributes, and their data. You don't even need to know that it's using XML. Just create your data structures and save/load them as needed - All the unicode, encryption, signatures, dtd, etc. is not included since it's not really necessary for game and simple app. purposes.


John Blackledge(Posted 2005) [#14]
Err... Didn't this structure used to be called INI files?


John J.(Posted 2005) [#15]
John Blackledge: No.. INI files only allow one level of data.. a category, then a list of properties and their values. XML is much better for game purposes and apps that require saving/loading of moderately structured data (INI files are very simple, not easily implimented for levels, AI files, etc.). With an xml file, you can save data like this:
<!--Test-->
<xml>
  <info>
    <title>Test World</title>
    <author>JohnJ</author>
    <description>
      This is a test file in XML
      format used to test the
      BlitzXML loader/saver
      library.
    </description>
  </info>
  <world>
    <entity type="cube">
      <position x="3" y="0" z="0"/>
      <rotation pitch="57" yaw="35" roll="5"/>
      <color r="0" g="0" b="255"/>
    </entity>
    <entity type="sphere">
      <position x="-3" y="0" z="0"/>
      <color r="255" g="0" b="0"/>
    </entity>
    <entity type="light">
      <position x="-3" y="5" z="2"/>
      <color r="255" g="255" b="255"/>
      <source type="point" range="7"/>
    </entity>
    <entity type="camera">
      <position x="-7" y="-1" z="-4"/>
      <rotation yaw="-60"/>
      <bgcolor r="64" g="128" b="255"/>
    </entity>
    <entity type="camera">
      <position x="0" y="8" z="0"/>
      <rotation pitch="90"/>
      <bgcolor r="0" g="255" b="0"/>
      <viewport x="0" y="400" width="200" height="200"/>
    </entity>
  </world>
</xml>


What's nice about BlitzXML is that you can save/load this data easily, without even knowing how XML works. An entity might be a single xmlNode type, with all it's data easily accessible. You don't need to know a thing about what the above XML code means, although it helps if you want to manually edit your xml files.


John J.(Posted 2005) [#16]
Update: The BlitzXML parser and saver now supports the standard "<?xml ... ?>" style header.


Bot Builder(Posted 2005) [#17]
John J - oh yeah, i havent looked at the docs for yours. I was mainly referring to System.Xml in .net - bloatius maximus.

Nice work though.


VP(Posted 2005) [#18]
XML is notoriously a 'bloated' format and ISTR that there was a proposed specification for a compressed XML format.

Perhaps if I can get zlib compression figured out for Blitz, it could provide an accompaniment to BlitzXML (I promise nothing!).


LineOf7s(Posted 2005) [#19]
vinylpusher:

Here's a start.
http://www.blitzcoder.com/cgi-bin/showcase/showcase_showentry.pl?id=peter__scheutz10222002181354&comments=no
Perhaps Mr Scheutz has progressed further with his userlib version beyond the (now dead) link in the link provided above (I've not checked).

Still, the above works.


VP(Posted 2005) [#20]
Lineof7s: Hmm, interesting. Still, the project I'm working on I intend to have no external DLL usage.

I only want to implement deflate with a 32k window, no other variations and certainly not a full implementation of all zlib functions. As such, it will only really be useful for 2 things:

1) PNG exporter (my project).
2) General compression of datafiles which have an original size of 0.5k (for binary) or 1k (for text) or larger.

I could do with getting rid of this cold though, useful coding whilst ill is not easy %^/


slenkar(Posted 2005) [#21]
I tried the example and it worked.
I then loaded the example XML file into Microsoft XML editor and saved

then it would not work anymore. How do I make sure the file formats are correct?


John J.(Posted 2005) [#22]
Update: Version 1.7 Released! Version 1.7 includes some misc. bug fixes.

Slunkar: This bug-fix may solve the problem. If not, could you e-mail me the XML file that does not work?


slenkar(Posted 2005) [#23]
yes it works now thanks

do you know if you can safely encrypt and decrypt these files?


Jake L.(Posted 2005) [#24]
I just have to tell you that BlitzXML saved my day more than once. It's easy to use, fast and works like a charm.

I even wrote a little hack to store binary data (using banks) into a node. I don't know if this is the right way to go, but it works. Maybe you want to add this functionality to your official lib (hehe, I'm too lazy to rehack every new version you release).

These are the parts of your code I've changed/added (to version 1.5 or 1.6, I think):

Add a new Field "ContentType" and a function "xmlNodeDataSetBank":
Type xmlNode
	Field Name$								;The name of the node
	Field AttributeCount					;The number of attributes for the node
	Field AttributeName$[MAX_ATTRIBUTES]	;The attribute name array
	Field AttributeValue$[MAX_ATTRIBUTES]	;The attribute value array
	Field Contents$							;The data contents of the node
	Field ContentType%						; 0 = string, 1=binary (bank-data)
	Field Parent.xmlNode					;This node's parent node. If this is set to Null, then this node is the "root" node
	Field Level								;This is the node's level
	
	;These fields are manipulated by xml_RegisterChild(), xml_GetChild(), and xml_UnregisterChild()
	Field ChildCount						;The number of the node's children
	Field ChildBank							;A memory bank of handles to the node's children
End Type

;// Pass a handle to store it's data into a node
Function xmlNodeDataSetBank (node,bank%)
	this.xmlNode=Object.xmlNode(node)
	this\ContentType=1	; binary
	this\Contents=Str(bank)
End Function


Some changes in xml_NextItem to load binary-nodes:
Function xml_NextItem$(file)
	Local tag$
	While Eof(file) = False
		ch = ReadByte(file)
		If txt$ <> "" And (ch = 60 Or ch = 13) Then xml_ItemType = 2:SeekFile file,FilePos(file)-1:Return txt
		If ch=61
			If Left(txt$,7)="binary:"
				banklen=Mid(txt$,8,10)
				bank=CreateBank(banklen)
				ReadBytes (bank,file,0,banklen)
				txt$=Str(bank)
			EndIf
		EndIf
		If ch <> 13 And ch <> 15 And ch <> 10 Then txt$ = txt$ + Chr(ch)
		If ch = 13 And txt <> "" Then txt = txt + " "


and some changes in saveXML to store binary data:

Function xmlSave(FileName$, thisnode, level = 0, file = 0)
	If file = 0 Then
		file = WriteFile(FileName)
		If file = 0 Then xml_AddError("Error writing XML file (possibly, file is in use, or is the folder/drive/file is write protected).", 0):Return
	End If
	
	If level = 0 Then WriteLine file, "<?xml version="+Chr(34)+"1.0"+Chr(34)+" ?>"

	If thisnode = 0 Then this.xmlNode = First xmlNode Else this.xmlNode = Object.xmlNode(thisnode)
	While this.xmlNode <> Null
		If this\Level < level Then Exit
		If this\Level = level Then
			tag$ = this\Name
			closetag$ = "/" + this\Name
			For i = 1 To this\AttributeCount
				tag = tag + " " + this\AttributeName[i] + "=" + Chr$(34) + this\AttributeValue[i] + Chr$(34)
			Next
			indent$ = String$("  ", this\Level)
			If this\Contents = "" And this\ChildCount = 0 Then
				WriteLine file, indent + "<" + tag + " />"
			Else
				If this\ChildCount <> 0 Then
					WriteLine file, indent + "<" + tag + ">"
					If this\Contents<>""
						If this\ContentType=0
							WriteLine file, indent + "  " + this\Contents
						Else ; binary data
							bank=this\Contents
							WriteLine file, indent + "  " + "binary:"+BankSize(bank)+"="
							SeekFile (file,FilePos(file)-2)
							WriteBytes bank,file,0,BankSize(bank)
						EndIf
					EndIf
					xmlSave("", Handle(After this), this\Level + 1, file)
					WriteLine file, indent + "<" + closetag + ">"
				Else
					WriteLine file, indent + "<" + tag + ">" + this\Contents + "<" + closetag + ">"
				End If
			End If
		End If
		this = After this
	Wend
	
	If level = 0 Then CloseFile file
End Function


Pass a Bank using xmlNodeDataSetBank, xmlSave will add a node like "<mynode>binary:1024=§/%&§(%/§(%/</mynode>" - the changed xml_NextItem will scan for this and store the next 1024 bytes into a new bank. Use xmlNodeDataGet to get the bank's handle.

The only limitation of my code is the usage of "binary:" as node-data, which is now preserved for binary-data. For me that's ok.

Hope you like it


slenkar(Posted 2005) [#25]
John J. just wondered if you could help,

Im trying to use XML for a character interaction system

edit-its ok I did it


Zethrax(Posted 2005) [#26]
Looks like a very useful lib.

There were a few issues I had with it, however. It saves the nodes arse about, so that those created first in code are at the bottom of the file and those created last are at the top. This makes the file difficult to read, and may cause incompatibility with other XML readers (I don't know if node position is actully a relevant factor with XML).

Also, 'xmlSave' seems to be returning an error result (false) even when there isn't one.

Here's the code I tested with:


And here's the result:



Zethrax(Posted 2005) [#27]
So... any comments on the fact that this saves the file upside down. Seems to me like a bass ackwards way to do it.


John J.(Posted 2005) [#28]
(I don't know if node position is actully a relevant factor with XML)

As far as I know, the XML standard states that node positions are completely irrelevant. If you require node positions to be kept intact, you're probably misunderstanding the concept of XML structure. But I'll try to straighten this out for you anyway.


Scott Shaver(Posted 2005) [#29]
"When children are declared in a sequence separated by commas, the children must appear in the same sequence in the document. In a full declaration, the children must also be declared, and the children can also have children."

have a read about DTDs and XML here: http://www.xmlfiles.com/dtd/dtd_intro.asp

Esentially you could argue the point either way, if you don't use a DTD to describe and validate the XML then you probably shouldn't worry about the order. If you do use a DTD then the order matters.

hope this helps
Within the DTD for the purchase_order document, 
you could then define elements, such as:


<!DOCTYPE purchase_order [ 
<!ELEMENT purchase_order (customer)> 
<!ELEMENT customer (account_id, name)> 
<!ELEMENT account_id (#PCDATA)> 
<!ELEMENT name (first, mi, last)> 
<!ELEMENT first (#PCDATA)> 
<!ELEMENT mi (#PCDATA)> 
<!ELEMENT last (#PCDATA)>]>
The example DTD declares seven elements (purchase_order, customer, 
account_id, name, first, mi, and last) and sets the order in which 
those elements may be entered in a document. The line breaks used 
aren't relevant to the DTD, and neither is the order in which the 
elements are listed. Although the elements are entered from the 
highest level to the lowest level, you could also enter them in this order:


<!DOCTYPE purchase_order [ 
<!ELEMENT last (#PCDATA)> 
<!ELEMENT mi (#PCDATA)> 
<!ELEMENT first (#PCDATA)> 
<!ELEMENT name (first, mi, last)> 
<!ELEMENT account_id (#PCDATA)> 
<!ELEMENT customer (account_id, name)> 
<!ELEMENT purchase_order (customer)>]>
or this order:


<!DOCTYPE purchase_order [ 
<!ELEMENT account_id (#PCDATA)> 
<!ELEMENT customer (account_id, name)> 
<!ELEMENT first (#PCDATA)> 
<!ELEMENT last (#PCDATA)> 
<!ELEMENT mi (#PCDATA)> 
<!ELEMENT name (first, mi, last)> 
<!ELEMENT purchase_order (customer)>]>
As these examples show, the order of DTD declarations isn't important. 
What is important are the declaration, the declaration name, and the 
associated values. In the case of elements, the values in parentheses 
set the order in which the elements must be used. Here, customer 
elements must contain exactly one account_id element followed by exactly 
one name element. The name element must contain exactly one first 
element followed by exactly one mi ele-ment followed by exactly one 
last element. The first, mi, and last elements must contain parsed 
character data (#PCDATA), which is raw text that could contain entity 
references, such as &gt; or &lt; but don't contain other markup or 
child elements.

The spacing between the declaration name and other elements is also 
important. The following declaration is improperly formatted:


<!ELEMENTaccount_id (#PCDATA)>
as is the following declaration:


<!ELEMENT account_id(#PCDATA)>
The correct format is:


<!ELEMENT account_id (#PCDATA)>



John J.(Posted 2005) [#30]
Thanks for the info. Like I said, I'll try to fix the node order reversal. I doubt I will ever add support for DTD and other XML "features", because I think most of those features are unnecessary and just adds to the confusion. (At the local library here, there's a book titled "XML In A Nutshell", which is about 5 inches thick and has over 600 pages. If that's XML in a nutshell, I'd hate to see a complete guide. :) )

However, if there's a feature you really would like in MaXML, I'll do my best to add it.


Zethrax(Posted 2005) [#31]
Glad to hear you're going to sort this out, John :-)

The main issue I had was one of readability as I'm working on a world editor and I'm planning on using this library to generate export files with it. I'd like the end user to be able to view the exported file in a text editor to get a clear idea of how the file format is structured.

As I said above, it's a very usefull lib, and greatly appreciated.


John J.(Posted 2006) [#32]
Update: Nodes now save/load in the correct order.

P.S. Bill Stanbrook: Sorry this took so long - I got side-tracked with other projects (MaXML, BattleTanks II, etc.) and I forgot about this problem.


Roland(Posted 2006) [#33]
Awesome! Thanks's John!


Roland(Posted 2006) [#34]
Quick question -- Is there a way to indicate where in the XML a new node should be positioned (at which point in the order, not at which point in the hierarchy?)

ie:

if my existing XML was:
<rootnode>
<child1>
</child1>

<child3>
</child3>
</rootnode>

Is there any way i could put "child2" inbetween child1 and child3?

Thanks!

Roland


VP(Posted 2006) [#35]
[lack of] DTD validation and generation is one of the first things I noticed re MaXML / BlitzXML, but didn't pass comment on because I don't think we really need it, do we?

Unless you're creating some fairly high-brow application, DTD's are not needed. Your average 3D map dataset isn't going to need a DTD generating, nor will it need to be checked against one when you load it into your game.

I looked at the DTD spec when I began to grok XML. Whilst you do need one for standards compliance, stuff tends to work without it. The most use I've had from a DTD is checking that my hand-written XML was not contradicting itself. It took more work to figure out the correct DTD than it did to figure out the XML structure.

If you really do need DTD, then your project perhaps is not one that should be reliant on a free set of procedures! Altova XMLSpy is a great piece of software which, incidentally, generates a correct DTD at the click of a button.


John J.(Posted 2006) [#36]
Roland: There isn't really any way to do that in BlitzXML currently, unless you delete all nodes of a parent and re-insert them. I'll see if I can add some features to re-arrange nodes. Unfortunately, BlitzXML is ancient technology compared to MaXML 2.0, which is much faster and feature-rich (I never really considered keeping node order intact when I created BlitzXML 1.0). I may someday re-write BlitzXML with all the features of BlitzMax.

Unless you're creating some fairly high-brow application, DTD's are not needed. Your average 3D map dataset isn't going to need a DTD generating, nor will it need to be checked against one when you load it into your game.

I looked at the DTD spec when I began to grok XML. Whilst you do need one for standards compliance, stuff tends to work without it. The most use I've had from a DTD is checking that my hand-written XML was not contradicting itself. It took more work to figure out the correct DTD than it did to figure out the XML structure.


I agree. DTD support isn't really necessary in most cases.


MadMunky(Posted 2006) [#37]
is there any way to get MaXML to work with BlitzMax 1.18? or when will there be an update?


John J.(Posted 2006) [#38]
I wasn't aware of this problem. I'll try to fix it as soon as possible.


Ked(Posted 2007) [#39]
Is it possible to use this for Skinning? Like Yahoo Messenger does?


John J.(Posted 2007) [#40]
You can use BlitzXML do load/save any kind of data you want, so yes, you can save skinning configurations as XML if you want. But the actual skinning of your application is up to you (the programmer) - BlitzXML has nothing to do with graphics.

P.S. Or did you accidentally post in the wrong thread?


Axel Wheeler(Posted 2010) [#41]
Hate to bump this thread but:

A: BlitzXML is sufficiently wonderful to merit it anyway, and

B: I found a bug:

Self-closing tags without attributes seem to add a spare "/" with each save. As in <players////>. When reloaded, it then keeps all but the last slash as part of the node\Name$ string, which generally makes it unusable.

Here's my simple fix:

	xmlRootNode=xmlLoad("settings.xml")
	
	;This next loop is due to a bug in BlitzXML which passes the closing / in a tag as part of the name itself.  We remove it here.
	For n.xmlNode=Each xmlNode
		name$=n\name$
		While Right$(name$,1)="/"
			name$=Left$(name$,Len(name$)-1)
		Wend
		n\name$=name$
	Next


Thanks John J. for a great library. I'm just getting into config files and this is definitely the way to go. This website sorely needs a recommended solutions area, or rankings or something so folks don't have to browse everything to find gems like this.