custom game data pack?

Blitz3D Forums/Blitz3D Programming/custom game data pack?

Caton(Posted 2015) [#1]
I packed a bunch of files in one.
how do I get it to read the bytes of one file then go to the next. this in packed into one file.


Matty(Posted 2015) [#2]
You would need to know the number of bytes in the original file to get the offset. You would most likely store these values in the header of the packed file for use when opening.

Eg the header may contain something like this.
Int number of files
For each file:
Int file offset
Int file length
Int file id

The file idwould be used by you to identify the file in another part of your application and wpuld link to a database of respurces.


steve_ancell(Posted 2015) [#3]
I take it you mean kind of like an indexing system Matty?


Caton(Posted 2015) [#4]
fileout = WriteFile("gamedata.dat")
;File Data
WriteInt( fileout,1);Number of Files
WriteString( fileout,"data\test1.txt")
WriteInt( fileout,0);Range in Pack File.
WriteInt( fileout,4);Number of Bytes To Read of Pack File
;Packed Files
WriteLine( fileout,"test");test1.txt packed file
CloseFile( fileout )
End
could you please add code to the file data? where it searches for all directory's and files then writes to file data?
and after that could you please have it write all files into packed files.


Floyd(Posted 2015) [#5]
If you are serious about getting this done then roll up your sleeves and do it yourself. Nobody is going to write your program for you. That should be obvious from the fact that you have been asking this and related questions for more than a year.

It's not that difficult if you do it in pieces.

1. Practice writing and reading files. For every command such as WriteFile, WriteLine, WriteBytes etc there is a corresponding ReadFile, ReadLine, WriteBytes. Whatever you write you will read back in exactly the same order.

2. Decide on the structure of the pack file. It should be as simple as storing the file name, file size and then the data. Do the same thing for every file. If you are using directories then that is an added complication, but not much. When reading the pack file you just keep reading until you reach the end.

could you please add code to the file data? where it searches for all directory's and files then writes to file data?

You can write a program to automate this, or you can find some existing tool to dump a list of filenames to a text file. Then manually edit the names into your packing code. It may be a tedious job but it is your job, not ours.

If this project is important to you then get to work and make it happen.


Caton(Posted 2015) [#6]
I don't know how to read all dir and files in the current dir. by code.


Matty(Posted 2015) [#7]
Then look at the sample code in the help section of blitz.


_PJ_(Posted 2015) [#8]
I don't know how to read all dir and files in the current dir. by code.

Yet you are asking how to pack and unpack entire collections of files' contents - Seems much like wanting to run before you can walk.
I suggest you start with the basics, how to read and write files incdivcidually, then look at how to determine the byteposition when reading/writing and the use of header segments to detail the size and number of packed files etc.

There are a number of filepacking routines in the Code Archives.


steve_ancell(Posted 2015) [#9]
@Caton:

The file handling commands are pretty straight forward. Just do what Matty said, read the help files then expand your coding methods.

When you started this thread I also thought you were more advanced, I thought you already knew at least the basics. If you want help then you need to put in some effort, you can't expect others to do the donkey-work for you.


steve_ancell(Posted 2015) [#10]
Caton:
I don't know how to read all dir and files in the current dir. by code.


I'll point you in the right direction, here's a link in the manual for reading dir's.

http://www.blitzbasic.com/b3ddocs/command.php?name=ReadDir&ref=2d_cat


steve_ancell(Posted 2015) [#11]
Here are links to the catagories. Have a good play around when you got enough info, you'll be surprised how far you get. ;)


This catagory for opening files:
http://www.blitzbasic.com/b3ddocs/command_list_2d_cat.php?show=File

This catagory for reading and changing what is inside of files:
http://www.blitzbasic.com/b3ddocs/command_list_2d_cat.php?show=File/Stream


Bobysait(Posted 2015) [#12]

I packed a bunch of files in one.
how do I get it to read the bytes of one file then go to the next. this in packed into one file.



I think really interesting to see that matty has a answer for that, when my first and only thought was "Wait ... what is your datapack format ???"

Then, for sure everything is already said : Read the doc, use Die and Retry method, read files, write files, test each command (and pray for the safety of your Hard Drive) That's the way programmers learn. (I'm pretty sure I'm not the only one here having a graveyard full of hard drive, graphics card, memory stick ... just because there is always a moment you need to step up and try by yourself some impredictible stuff)

My only advice : help yourself and buy a low-cost machine dedicated with low-cost material (like a 50$/100$ PC from ebay with old graphics chipset, old cpu and probably some DDr2 ram), trust me, it totally worth it, because on the first time, code is always messy and dangerous, full of memory leak and high cpu/gpu cost, but it's the way it has to be done : experience (There is no secret, a developper push its hardware on very bad conditions until he get massive experience and know exactly what he is doing).
So, be ready to change some part because, you'll kill some pieces during your tests.
So, now, it's your turn ! You want to program or you just want something to be done it's up to you
- in the first case, DIY and be ready to loose some money in hardware !
- in the second case : I'm ready to do it for you, no problem. My fees : 40$/hour.

ps :
And by the way, unless it's something personnal not expected to be published, always target some minimum hardware, because getting something to work on a high-end machine won't help to know if it works decently on the target machine.
So, if you have a low-end machine, it's full of benefits and encourage you to optimise your code every time.

And thoose advices are royalty-free, so take them according to your wishes.


steve_ancell(Posted 2015) [#13]
Although most of us aren't going to simply code it all for you, we can however help to achieve your goal in other ways.

If it is essential for everything to be in one file then I suggest using tokens to signify the start of blocks of data inside the file such as: [this_name], [that_name], [etc_name] and use a matching search string to find them, and use an escape token respectively to signify the end of that particular block of data.

In my honest opinion though, I think data packing is more hassle than it's worth.


steve_ancell(Posted 2015) [#14]
I also know another way. Have all of the block names at the start of the file with the line number and line count of each block next to each token. This way would require you knowing the postion and length of each block beforehand, but it would also be faster than using a search like I suggested in my first suggestion.


steve_ancell(Posted 2015) [#15]
Or you could use my fisrt suggestion and still have fast access if you just search each token and then store them at first run of the program.


steve_ancell(Posted 2015) [#16]
I'm sure my second suggestion is basically the same as what Matty suggested, although my explanation is a bit different. I have a weird way of explaining things due to the way my brain ticks.


steve_ancell(Posted 2015) [#17]
Have you had any luck yet Caton? ;-)


Bobysait(Posted 2015) [#18]
@steve_ancell :
You've got an "Edit" button on the top-right corner of each of your post.


steve_ancell(Posted 2015) [#19]
Edit button, post button, it still takes up the same amount of space.


steve_ancell(Posted 2015) [#20]
Plus edit doesn't show up as a new post in the catagories section, most people probably only bother going back to a topic if they see changes in the headline.


Bobysait(Posted 2015) [#21]
We don't care about "new" in the section, it's not your buisness to force people to come and see "your" message ...
As long as it's a developper forum and not a social network, you're just spamming the forum so I 'm asking you please stop and use the edit button.

ps : and, whatever you think, when I see 4 posts from the same person on a topic which is even not his, I skip his comment. This forum is not "his" place but the place of all. it's just a fact not a thought.

ps2 : Just for information : A post edited is just some text added to the same post. A "new post" is a bunch of extra data containing link to previous and next post, several dates archived (creation time, edition time, last time seen etc ...), some index for the database, owner, owner id, etc ... etc ...
It's really much more than a simple edit.


steve_ancell(Posted 2015) [#22]
We don't care about "new" in the section

And who's "we" exactly, do you speak for everyone then. I don't care what side of the bed you got out of this morning, that doesn't give you the right to start trolling.

I have no time to argue about this so I guess I will just have to walk away from it, and if you've got any sense then you'll do the same.

[edit]Now let's all get back on topic. There!, I used the edit button[/edit]


Bobysait(Posted 2015) [#23]
tsss ...

I'm not trolling you at any moment, (at least probably not as much as you "really" do)
I just gave you an information about the most commun rule all other the net for posting on a forum.
You take it personnally while I even don't care about who you are, what you are or what you think (it's not to take personnal either, it's just a fact, I don't know you and I don't need to, [and for the moment, i must admit I don't want too ... fair enough])
So I just gave you an advice warning you politely, take it for what it is.

As mentionned, it's a developpers forum, not a social network, I'm here to help, nor to make friends, neither enemies, and I'll probably continue helping you whether or not you thing i'm a troll or anything else. I'm really not here to have the role you think I have. So, think whatever you want of me, I don't care, but I assume I'm not the only one thinking editing a post is a lot better than posting again and again and again.
Just, try to trust me, go and read some forum rules, and come back and say me I was wrong, or just keep on going with the subject and let's just consider the conversation never happened. I won't blame you, I really have so much better to do than just tracking guys that don't know how a forum work. And that too, it's not to take personnal.


RemiD(Posted 2015) [#24]
Stay cool guys, these forums have only a few members discussing game programming with blitzbasic, no need to create useless conflicts...

(for your info, Caton sometimes disappears during a few weeks...)


steve_ancell(Posted 2015) [#25]
Bobysait:
I'm not trolling you at any moment, (at least probably not as much as you "really" do)

I'm no troll, I don't know where you dug that up from considering I don't "really" come here that much. I know that you're only trying to uphold the rules here but there's a bith of a margin between rules and outright state control.

Right!, with that out of the way let's both take RemiD's advice and let this go, water under the bridge and all that.


fox95871(Posted 2015) [#26]
I was messing around with Blitz media linker today for the first time in a while. I think I tried something like what you're talking about once, and it is tough. Does it absolutely have to be a one file game? I've played many games that had an exe and a folder full of files, and it didn't bug me. It might just be a programmers compulsion thing, whereas players wouldn't even care. I've had to fight stuff like that myself from time to time. Fortunately, my game's become much more linear over time, and I currently have no need to modify finished game data at all. But gloating won't help you I guess, so let's see... just to be clear, you are saying you want your game to be a one file exe that can have its data changed while the game's running, right? My first desperate attempt to do that would probably be to have a temporary file that gets created while the game's running that no one ever sees, then after the game ends, somehow have it update the exe since it's at that point not in use, then have the temporary file delete itself, but that's just a guess. I've made lots of good guesses in the past though, and I'm pretty sure I've seen files like that appear and disappear in windowed games. Keep trying :)


Matty(Posted 2015) [#27]
Theres no reason to pack all files in one that i can think of. Save headaches and use separate files. No one is going to want to steal your media based on your apparent current level of skill (not too harsh i hope) and it is easier for you to maintain your code if you can simply drop media files in and out of folders during development when needed instead of having to recompile a huge data pack.

If it bothers you that much use a zip with all media in a folder and simply unzip during install.

I dont know of many games ive played over the years from well known publishers who put all media in a single file. They may have their own formats sometimes but they still use multiple media files.


steve_ancell(Posted 2015) [#28]
You'll avoid a lot of hair loss too if you avoid the data packing thing. LOL


fox95871(Posted 2015) [#29]
Well, I once had a forty six include character animator. It took me about nine strait months to complete. All told, it fried a computer, temporarily damaged the vision part of my brain from hitting my head in frustration at times, and it turned out I ended up not even needing the thing. But it taught me, laziness can be just as valuable as hard work. Really delving into something difficult makes you progress, but taking a step back for a while helps you understand if it's really something you want.


steve_ancell(Posted 2015) [#30]
fox95871:
All told, it fried a computer, temporarily damaged the vision part of my brain from hitting my head in frustration at times


Crickey it's Rambo!. I think you may need to install extra cooling in your PC and invest in a crash helmet. :-D


Caton(Posted March) [#31]
Is there away I do this faster it's very slow when unpacking data like models,textures,sounds, and very slow with music.