How to parse/modify a (.txt) file. (Intermediate, B3D and B+)

Blitz3D Forums/Blitz3D Tutorials/How to parse/modify a (.txt) file. (Intermediate, B3D and B+)

WolRon(Posted 2003) [#1]
So, you have an .ini or .txt file that contains some user modifiable values and you want your program to have the ability to load these values and use them and/or write these values back to the file?
There are many ways that you could implement reading or writing to the file. This tutorial will discuss only a few of the methods that you could use.


***************Reading in the file***************

Let's say you have some variables in your program that need to be initialized with some values from a user editable file called gamefile.txt.
Let's also say that our test file (gamefile.txt) contains these 7 lines:

[graphics]
width=1024
height=768
colordepth=16
[sound]
sounds=on
volume=50

You could use one of the following methods to read in the file:

********Method 1:
Parse through each line looking for keywords.

First we look for bracketed sections and then load individual parameters within those sections into our program variables. The code uses the Lower$ function to eliminate the dozens of variations a parameter could be written using upper and lowercase characters. If the code finds a line that begins with another opening brace, then it assumes there are no more parameters in that section. After parsing through the graphics section of gamefile.txt it uses a SeekFile(filein, 0) to start once again at the beginning of the file searching for the sound section. Notice also in the sound section that the sounds=on line doesn't have a numerical value but it is still taken care of by the code.

Global GfxWidth=640
Global GfxHeight=480
Global GfxDepth=16
Global Snd=True
Global SndVol=50

filein = OpenFile("Gamefile.txt")
If filein <> 0
  ;load graphics mode
  foundit = 0
  Repeat
    If Lower(ReadLine$(filein))="[graphics]" Then foundit = 1
  Until Eof(filein) Or foundit = 1
  If foundit
    done = False 
    Repeat
      param$ = Lower(ReadLine$(filein))
      If Left$(param$, 6) = "width="
        val = Right$(param$, Len(param$) - 6)
        If val = 1024 or val = 800 Then GfxWidth = val
      EndIf
      If Left$(param$, 7) = "height="
        val = Right$(param$, Len(param$) - 7)
        If val = 768 or val = 600 Then GfxHeight = val
      EndIf
      If Left$(param$, 11) = "colordepth="
        val = Right$(param$, Len(param$) - 11)
        If val = 16 Or val = 24 Or val = 32 Then GfxDepth = val
      EndIf
      If Left$(param$, 1) = "[" Or Eof(filein) Then done = True 
    Until done = True 
  EndIf

  ;load sound parameters
  SeekFile(filein, 0)
  foundit = 0
  Repeat
    If Lower(ReadLine$(filein))="[sound]" Then foundit = 1
  Until Eof(filein) Or foundit = 1
  If foundit
    done = False 
    Repeat
      param$ = Lower(ReadLine$(filein))
      If Left$(param$, 7) = "sounds="
        word$ = Right$(param$, Len(param$) - 7)
        If word$ = "on"
          Snd = True
        Else
          Snd = False
        EndIf 
      EndIf
      If Left$(param$, 7) = "volume="
        val = Right$(param$, Len(param$) - 7)
        If val > -1 And val < 101 Then SndVol = val
      EndIf
      If Left$(param$, 1) = "[" Or Eof(filein) Then done = True 
    Until done = True 
  EndIf

  ;close file
  CloseFile filein
  filein = 0
Else
  DebugLog "File Not Found"
EndIf ;filein <> 0

A downfall of this specific code is that it is checking for keywords on each line with the equal sign immediately following the keyword (i.e. volume=). If the file was modified and the line read "volume =" (notice the space between volume and =) then this code would fail to load the value. This code could be modified to search for only the keyword and then parse furthur through the line to remove any extra characters before the data.

Another downfall of this code is that it's hardcoded. It's not very flexible. To load a different parameter, you have to add more code to it to handle that. Another way is to change this code into the form of a function that can write/read whatever values you pass to it.
Rob Farley has written a function that uses a similar method to both read and write data from/to the file. You can find it in the code archives here:
www.blitzbasic.com/codearcs/codearcs.php?code=776


********Method 2:
Read from a structured file.

This method allows you to load values from the file without needing to parse through the file (providing that you know the order of the parameters in the file). The file can be structured in such a way that every line is the same length. This way you can easily load any line simply by using SeekFile() to point to the desired line and read in a certain amount of characters.
You could have a structure something like this:

variablename = floatingpointvalue
AAAAAAAAAAAA=XXXXXXX.XXXXXX
width       =   1024       
height      =    768       
colordepth  =     16       
sounds      =on            
volume      =     50       
This way if you wanted to load any given value (say the colordepth), you could just skip to the third 'line' by
SeekFile'ing to (3 - 1) times 27(the linelength) plus 13(offset to value part of the line)
and ReadByte'ing in the next 14 characters (the length of the value portion)

The downfall to this method is that if a user were to modify the file and change the structure in any way, it would become corrupted.



****************Modifying the file***************

Let's say you have some variables in your program that need to be saved to a file called gamefile.txt.
For starters, let's say that gamefile.txt already contains these 5 lines:

difficulty=3
gamma=0.5
volume=75
zoffsetvalue=45903.239594
xoffsetvalue=1003478.002

Let's also say that you want to change the gamma value in the file from 0.5 to 0.8. How would you do it? It's not as simple as it sounds.
Basically there are two ways:
1. Modify specific bytes within the file
2. Overwrite the entire file.

First of all, a note about trying to modify specific bytes within the file:
Writing on a specific line is complicated because all data in the file is actually sequential. This means that the volume=75 line immediately follows the gamma=0.5 line (and the line feed character(s)). If you tried to edit the file by using OpenFile and then SeekFile'ing to the desired byte, WriteByte'ing the desired byte(s), and then Closefile the results may not be what you expect. If the number of bytes you write are shorter or longer than the number of bytes already there, you will mess up the file.

What I am getting at is that if you were to replace 0.5 with 0.8 for example, then there would be no problem. BUT, if you were to replace 0.5 with .5 or 0.83 then funny things would happen. .5 would actually become .55 and 0.83 would become 0.83 but the three would write over the first character of the next line which would make your gamma= statement equal everything on the following line as well.
In this example < will represent line feed character.
Only the first three lines of our test file are shown in this example.

difficulty=3<gamma=0.5<volume=75
;changed to 0.8 would be
difficulty=3<gamma=0.8<volume=75
;just what you want

difficulty=3<gamma=0.5<volume=75
;changed to .5 would be
difficulty=3<gamma=.55<volume=75
;extra 5 wasn't erased

difficulty=3<gamma=0.5<volume=75
;changed to 0.83 would be
difficulty=3<gamma=0.83volume=75
;line feed character erased so now gamma = 0.83volume=75

That being said, I will show you three ways you could modify the file:

********Method 1:
Write to a structured file.

One method you could implement would be to structure your file in such a way that you don't use variable length lines. Every line in your file is exactly the same length so you can easily change any line and know that you won't overwrite anything else. You could have a structure something like this:
variablename = floatingpointvalue
AAAAAAAAAAAA=XXXXXXX.XXXXXX
difficulty  =      3       
gamma       =      0.5     
volume      =     75       
zoffsetvalue=  45903.239594
xoffsetvalue=1003478.002   

This way if you wanted to change any given value (say the volume percent to 80), you could just skip to the third 'line' by
SeekFile'ing to (3 - 1) times 27(the linelength) plus 13(offset to value part of the line)
and WriteByte'ing the characters
"     80       "
(exactly 14 characters long) (NOTE that you wouldn't actually write the quotes, they are just there for clarity)

The downfall to this method is that if a user were to modify the file and change the structure in any way, it would become corrupted.
The advantage to this type of file is that you can open the file once and continually write to any given parameter within the file without writing over any other data and without having to rewrite the entire file. In other words, random access.


********Method 2:
Read the file in to an array.

Read each line of the file in to an array, modify the line you wish and then write back the entire file one line at a time.
Const MAXLINES 200
Dim dat$(MAXLINES)

; read file
filein=ReadFile("gamefile.txt")
linecount=0
While Not Eof(filein)
	linecount=linecount+1
	dat$(linecount)=ReadLine(filein)
	If linecount=MAXLINES Exit 
Wend
CloseFile filein

;modify file
dat$(2)="gamma=0.8"

; write file
fileout=WriteFile("gamefile.txt")
For i=1 To linecount
	WriteLine fileout,dat$(i)
Next
CloseFile fileout



********Method 3:
Read the entire file in to two strings. Write back the entire file as first string + your change + last string.

Using the SeekFile and FilePos commands:
-find the spot that requires changing by parsing through the file until you find the phrase your are looking for (such as "gamma")
-note the FilePos where you found the phrase
-store everything before that spot (FilePos) in a string variable (say Beginning$)
-store everything after that spot in another string variable (say Remainder$)
-overwrite the entire file with: Beginning$ + "0.8" + Remainder$


**************************************

This tutorial should be enough to get you started. Please let me know (check for my email address under WolRon) if there is any way you think this tutorial could be improved.

Ignore the signature :)


wizzlefish(Posted 2005) [#2]
Thanks for this - but is there a way to include a .txt file with BLITZ code? Just INCLUDE the Blitz code from a .txt file, without using "Include?"


WolRon(Posted 2005) [#3]
is there a way to include a .txt file with BLITZ code? Just INCLUDE the Blitz code from a .txt file, without using "Include?"
I have no idea what that means, nor do I think that it relates to the topic of this tutorial.


fall_x(Posted 2005) [#4]
He wants to open a script with blitz code, and execute it. Which indeed does not really relate to this tutorial.

No Enemies :
There's no built-in support for doing this. There are some alternatives however, like BVM, or the BlitzScript3d Frank is working on. Or parsing the script yourself, but writing a scripting engine will not be an easy task.


wizzlefish(Posted 2005) [#5]
Yeah.


wizzlefish(Posted 2005) [#6]
OK:
Function LoadLevel(filepath$)
   
	lvl.level
	file = OpenFile(filepath$)
	lvl\title$ = ReadLine(file)
	lvl\id = ReadLine(file)
	lvl\mesh = ReadLine(file)
	
	Repeat
		If Lower(ReadLine$(file))="createobject"
			obj_type$ = ReadLine$(file)
			name$ = ReadLine$(file)
			obj_mesh$ = ReadLine$(file)
			el_dest = ReadLine(file)
			x = ReadLine(file)
			y = ReadLine(file)
			z = ReadLine(file)
			alpha = ReadLine(file)
			shine = ReadLine(file)
			xv = ReadLine(file)
			yv = ReadLine(file)
			zv = ReadLine(file)
			sx = ReadLine(file)
			sy = ReadLine(file)
			sz = ReadLine(file)
			rx = ReadLine(file)
			ry = ReadLine(file)
			rz = ReadLine(file)
			CreateObject(obj_type$, name$, obj_mesh$, el_dest, x, y, z, alpha, shine, xv, yv, zv, sx, sy, sz, rx, ry, rz)
		EndIf
	Until Eof(file)

End Function


That's my "LoadLevel" function so far.

The actual file looks like this:
Loophole
0
../media/levels/lvl0.3ds
createobject
crate
crate1
../media/models/crate1.3ds
0
0
0
0
1
0
0
0
0
1
1
1
0
0
0


I haven't tested this code yet, but I'm not sure about the second "ReadLine." Would it attempt to make lvl\id a string? Or would it read an integer, as it is supposed to be?


WolRon(Posted 2005) [#7]
Would it attempt to make lvl\id a string? Or would it read an integer, as it is supposed to be?
It would read a string and convert it to an integer.

Why are you running my tutorial off-topic?


wizzlefish(Posted 2005) [#8]
Sorry.