So close to reading from embedded files...

BlitzMax Forums/BlitzMax Beginners Area/So close to reading from embedded files...

Miss Blitz

(Posted 2014) [#1]

Hi, everyone!

So, I'm Incbin-ing some graphics and sounds one fine day and I think to myself "Why don't I embed my custom-made data files, too?"

Well, I do, and I make a collection of functions for gleaning Byte, Short, Int and String values from the now incbin-ed custom files. Everything seems to go along swimmingly, but then weird bugs
start popping up in my game.

A little digging, and I find this out about my custom file for storing data about in-game items:

The first value in the file, an integer, loads perfectly. So do all the
Short and String values, but all subsequent Int values don't load right. Here's the code.:

Function ReadBinInt:Int( bf:Byte Ptr )
	Local i:Int = Int(bf[bf_e])
	bf_e :+ 4 ' an Int has 32 bits or 4 bytes. That's why the 4 point increment.
	Return i
End Function
Function ReadBinShort:Short( bf:Byte Ptr )
	Local s:Short = Short(bf[bf_e])
	bf_e :+ 2 ' a Short has 16 bits or 2 bytes. That's why the 2
                       ' point increment  
	Return s
End Function
Function ReadBinString:String( bf:Byte Ptr,stringlength:Short )
	Local str:String,str2:String
	For Local i:Short = 1 To stringlength
		str = Chr(Byte(bf[bf_e]))
		str2 :+ str
		bf_e :+ 1 ' a character in a string is 8 bits or 1 byte long,
                               ' we're moving to the next byte.
	Next 
	Return str2
End Function

Any thoughts? Thank you so much for listening. :-D

Yasha

(Posted 2014) [#2]

A Byte Ptr is, literally, a pointer to a Byte. If you deference it (`bf[bf_e]`), you are only retrieving the eight-bit value at that offset. To read a 32-bit value, you need to cast the pointer, so that the right kind of read operation happens at all:

Local i:Int = (Int Ptr(bf))[bf_e]

Casting the result of the read through a byte ptr won't make the read operation draw 32-bits; it already happened by that point. Same goes for Short, and any other data types.

Also, a character in an internal BlitzMax string is two bytes long, as BlitzMax uses UCS-2 encoding (this may not be relevant to your packed data type, of course: if you only wrote one byte then reading that much back is valid, but it does mean that you won't be able to store most non-English strings accurately).

Miss Blitz

(Posted 2014) [#3]

Thank you, Yasha!
(Reads post very carefully) Now, I know for sure I have no idea what I'm doing. Can what I'm trying to do even be done? If so, can anyone point me to a good tutorial on using pointers? i need deeply fundamental help. T_T

Yasha

(Posted 2014) [#4]

Can what I'm trying to do even be done?

Sure, just make the change indicated in the codeblock in post #2. You were very nearly right first time, you just have the parentheses in slightly the wrong place.

All a pointer is is a location in memory. You can, if you like, think of it as an index into an array where the array is your whole process space.

Since, fundamentally, everything is a sequence of bits somewhere in memory, a pointer can represent literally anything. Therefore, BlitzMax can't tell you what kind of data it will find based simply on the address. In the same way that both a float and an int are a sequence of 32 bits, and variables must be tagged to indicate which one is the valid interpretation of the bit pattern, variables holding pointers must also be tagged to indicate what kind of thing they could be pointing at.

So when you dereference a pointer as in `bf[N]`, BlitzMax uses the type of `bf` to determine what the deref operation actually entails. If `bf` has type Byte Ptr it will read a byte. If it has type Float Ptr it will read a float.

If you need to change the read operation, you can cast the pointer to another type of pointer just as you can cast a number to another type of number:

Int Ptr(bf)     'dereference this and it will read and return 32 bits
Float Ptr(bf)     'dereference this and it will read and return 32 bits as a float
Short Ptr(bf)     'dereference this and it will read and return 16 bits
Long Ptr(bf)     'dereference this and it will read and return 64 bits

The value of the pointer itself never changes. The only thing that changes is what BlitzMax is tagged as allowed to do though it. (You can also cast a pointer to Int if you want to display/examine it for debugging purposes.)

The error in the code above is that you never cast the pointer itself to tag it as returning a value larger than a byte. The code still compiled because it's legal (just unnecessary) to cast a byte - the value being returned - to a short or int.

Your original code:
Pointer tagged to read one byte -> read one byte -> convert one byte to four bytes

My code in post #2:
Pointer tagged to read one byte -> same pointer retagged to read four bytes -> read four bytes

Miss Blitz

(Posted 2014) [#5]

I think I found something to start on. If anyone else out there is stuck on pointers and stumbled onto this post, maybe you'd find this interesting?:
http://www.blitzbasic.com/Community/posts.php?topic=54348
I'll try to avoid wasting anyone else's time. I'm sorry.
Thanks again.

Miss Blitz

(Posted 2014) [#6]

Oh! You type fast! All right. I've entered that code block like you showed
me,

 (Int Ptr(bf))[bf_e]

. I used it on the other types too.
I must have done something wrong, because the program displays the first integer from the file, then seems to freeze afterward.

Function ReadBinByte:Byte( bf:Byte Ptr )
	Local b:Byte = bf[bf_e]
	bf_e :+ 1 ' increment to next byte
	Return b 
End Function 
Function ReadBinFloat:Float( bf:Byte Ptr )
	Local f:Float = (Float Ptr(bf))[bf_e]
	bf_e:+ 8 ' increase by this amount because a float uses 8 bytes (64 bits)
	Return f
End Function 
Function ReadBinInt:Int( bf:Byte Ptr )
	Local i:Int = (Int Ptr(bf))[bf_e]
	bf_e :+ 4 ' an Int has 32 bits or 4 bytes. That's why the 4 point increment.
	Return i
End Function
Function ReadBinShort:Short( bf:Byte Ptr )
	Local s:Short = (Short Ptr(bf))[bf_e]
	bf_e :+ 2 ' a Short has 16 bits or 2 bytes. That's why the 2 point increment.
	Return s
End Function
Function ReadBinString:String( bf:Byte Ptr,stringlength:Short )
	Local str:String,str2:String
	For Local i:Short = 1 To stringlength
		str = Chr(bf[bf_e])
		str2 :+ str
		bf_e :+ 1 ' a character in a string is 8 bits or 1 byte long, we're   
                               'moving to the next byte.
	Next 
	Return str2
End Function

... What did I mess up? Is it the variable I use for dereferencing the
pointers? (Thinks about it) Probably. I should probably try again in the morning. Thanks again, Yasha. You have been a huge help!

Floyd

(Posted 2014) [#7]

A Float is four bytes.

TomToad

(Posted 2014) [#8]

You can open an incbinned file as a stream and read data that way.

Code to create test data:

SuperStrict

'we will write some data to a test file with this program.

Local Stream:TStream = WriteStream("customfilename.cst")
WriteByte(Stream,1)
WriteShort(Stream,2)
WriteInt(Stream,3)
WriteLong(stream,4)
WriteFloat(stream,5.6)
WriteDouble(Stream,7.8)
WriteString(stream,"abcdefg")
CloseStream Stream

Code to read incbinned data:

SuperStrict

Incbin "customfilename.cst"

'we will read in the data and print it out
Local Stream:TStream = ReadStream("incbin::customfilename.cst")
'make sure file opened properly
If Not stream Then RuntimeError("Couldn't open incbin file")
Print "Byte = "+ReadByte(Stream)
Print "Short = "+ReadShort(Stream)
Print "Int = "+ReadInt(Stream)
Print "Long = "+ReadLong(Stream)
Print "Float = "+ReadFloat(Stream)
Print "Double = "+ReadDouble(Stream)
Print "String = "+ReadString(STream,7)

'We will read the long value again by moving the file pointer
Local p:Int = SeekStream(Stream,7)
'make sure pointer has moved
If p < 0 Then RuntimeError("Stream not seekable")
Print "Long again ="+ReadLong(STream)

'We will move the pointer back relative to its current location
p = SeekStream(Stream,StreamPos(Stream)-12) 'point it to the int
If p < 0 Then RuntimeError("Stream not seekable")
Print "Int again = "+ReadInt(Stream)

CloseStream Stream

you can also use banks if you need to access the data directly

SuperStrict

Incbin "customfilename.cst"

'we will create a bank from the incbin file
Local Bank:TBank = CreateStaticBank(IncbinPtr("customfilename.cst"),IncbinLen("customfilename.cst"))
Local bp:Int = 0 'pointer into the buffer

'now to write the values
Print "byte = "+PeekByte(Bank,bp); bp :+ 1
Print "Short = "+PeekShort(Bank,bp); bp :+ 2
Print "Int = "+PeekInt(Bank,bp); bp :+ 4
Print "Long = "+PeekLong(Bank,bp); bp :+ 8
Print "Float = "+PeekFloat(Bank,bp); bp :+ 4
Print "Double = "+PeekDouble(Bank,bp); bp :+ 8
Print "String = "+String.FromBytes(Byte Ptr(BankBuf(Bank)+bp),7)

'move the buffer pointer back to the long value
bp = 7
Print "Long again = "+PeekLong(Bank,bp); bp :+ 8
'move pointer relative back to the int
bp :- 12
Print "Int again = "+PeekInt(Bank,bp)

There is nothing wrong with the way you are doing it, just doesn't seem necessary to reinvent the wheel.

Miss Blitz

(Posted 2014) [#9]

And the day is saved! Thank you Tom Toad!

A Float is four bytes.

... It sure is! I must have gotten my Floats and Doubles mixed. Thanks for the heads up,Floyd!

Oh, this gets my game so close to done! I can't wait to show everyone the demo later. XD Thank you all again!