Clairified topic! Comma dilimited databases in txt
BlitzMax Forums/BlitzMax Beginners Area/Clairified topic! Comma dilimited databases in txt
| ||
| Ok.. my last topic no one understood, so, after spending days thinking how exactly to phrase this.. and having my car catch fire. (wonderful week.) anyways, heres my question. This is an example of a text file for one portion of the game I'm working on. This is a unit database, telling what mechanized units are in this zone. Filename: 3488578237.(whatever) 93324, 50, 50, 4, 93324, 90, 45, 3, 93390, 30, 90, 3, Explanation of the above text and numbers: The filename, is the particular zone id that the data represents. the first number is the UNIT ID, which references another central database, listing all the units in the game. So if the game needs to know what the mechs weapons are, or how much armor it has, it looks up the other central database. The second number, is the unit’s X coordinate on the 100x100 hexagon grid, while the third number, is it’s Y coordinate.. the last number, is the facing side, starting clockwise. Things I need to know, What is the command to make blitz generate a new text file with a predetermined filename for a new zone? Each zone has its own unit database (as well as other databases), so, if the zone is colonized, the game needs to make a new database, abeit blank at the beginning, to later be filled with unit information. What is the command to search the database for a particular unit ID? How would you for instance, replace the 3rd field on unit Id 93324? Can you change just one of the entries on that particular Unit’s info? Or do you need to re-write all the info on that line to update it? What is the command and syntax to add another line and thus ‘entry’ to the text file database? For instance, we want to add a 4th mechanized unit’s information to the table.. And most importantly, what are the commands and syntax of extracting data (using) information from the database? Pulling the unit information into individual variabales(field 2 goes to viariable named X, field 3 goes to variable named Y) etc, or a single variable that contains all of the unit’s information? (all of the information on a given line goes into (a variable named Z, or the unit id’s number as the variable) I’ve bought ALL 3 blitz books I could find, game programming for teens, blitz3d and the blitzmax book.. and I cant find aNY good information on database use, or extracting data from comma delimited textfile ‘databases’ or much of anything along those lines. PLEASE HELP! This is my major hiccupping point! |
| ||
| To begin with, I'm having trouble attempting to comprehend your devotion to using a comma-separated text file "database" system. The reason you seem to be finding it hard to unearth details on how to go about writing a comma-separated text file "database" system is probably because there are few people who would consider putting in all that work to manage the files, manage the "tables", and manage the 6th field of the 7th row of the table named "894353". Of course, if, at the end of the day, you are after a completely proprietary, obfuscated file format, you are definitely heading along the right tracks. And I hope for your sake you are documenting everything in great detail. I could, at this point, go on to discuss a perfectly feasible and thoroughly tested alternative file-based database format, but it seems that 1) you don't seem interested in learning anything new/better/useful, and 2) it's already been mentioned before. Anyhoo.. for those that missed it last time. SQLite probably does all that you want, does it better, does it faster, and has examples in BlitzMax already available... However, since we are here rather, to fan the flames of your obsession with a comma-separated text file "database" system, I guess we should not try to convince you otherwise, and see about pointing in a direction where you can read/write text files and whatnot. Rather than try to edit the files in place, as you appear to want to do : How would you for instance, replace the 3rd field on unit Id 93324? I suspect you will have a much easier time of it by holding the text file "in-memory", changing it when required, and re-writing the file to disk when think it's a good time to flush the data (perhaps immediately, perhaps later). To open a file for reading, you can use "ReadStream()", along with "ReadLine()" to read a line of text. You can check for the end of a file with "EOF()". And when finished, you can close it with "CloseStream()". As for reading in the data, you can use the String "split()" method to split a line of text by "," (comma) into an array of strings. This array you could either use as is, or perhaps copy into proper fields... like X = s[0] .. Z = s[4] etc. To open a file for writing you can use "WriteStream()", along with "WriteLine()" to write a line of text. When finished, you can close it with "CloseStream()". ... |
| ||
| However, since we are here rather, to fan the flames of your obsession with a comma-separated text file "database" system, I guess we should not try to convince you otherwise, and see about pointing in a direction where you can read/write text files and whatnot. Please don't encourage him. While I realize that failure is the best teacher, you know what he has planned is never, ever going to work. The best possible solution for this problem is quadratic O(n^2) and most are worse.For the amount of data he was talking about in the other threads we're talking about an algorithm that's unlikely to finish. Ever. Like bubble sorting the bible. I'll reiterate my advice from your previous thread. Your design is fundamentally flawed. Get some experience in database design first. This would be a fairly good place to start. |
| ||
| What is this used for? Saving/loading game data from harddisk? I think you should tell us the purpose/reason why you want this system. |
| ||
| He has said. He wants a sortof vertual Ram, using a "comma-separated text file "database" system" to hold all the game data |
| ||
| whats wrong with that? I use CSV files for some of my game data so it's easily editable. Some other date is saved straight as Integers/Floats etc into TStreams like Profiles and save games, high scores etc. |
| ||
| As long as it is not huge amounts of data, there should be no problems. |
| ||
| And if you have no huge amount of data, you don't need a database system, you could use a block based binary structure, where each block has a constant size together with a TOC (table of contents) that just holds the mapping for name -> id of the head entries. All the subentries can refer to the correct index on their own. This approach would map to a simple array of TBank (or byte ptr if you need higher performance) at runtime RAM representation |
| ||
| whats wrong with that? I dont think that there is anything wrong with it either. (Just to be clear).Apparently though there was a previous thread inwhich his stated aim was concidered too difficult/not right, or what ever If I was going to do it though, I think I would not make it Text, and would go for Byte, word and Int fields |
| ||
| Ok, to give a basic idea, this is for the server end of the game. Each player, is assigned a player ID. The player then picks a section of the galaxy to go to, then planet to set up their operation on. the planet they pick, depending on size, has between 50 and several thousand zones. in each zone they can build so many buildings to enhance their 'empire'. This is a persistant universe, all players existing simultaniously. While the buildings and such would exist only for logistical reasons and are not combat units.. What I was giving an example of, was the persistant unit database. when the combat units interact, it would pull amunition off of that unit's ammo count, whlie reducing the amount of armor or shield energy on the other combat unit. I cant pre-determine what all the fields, or field size will be, because all the combat units are custom configured, and the amount of fields per field entry could differ. |
| ||
| If it is a back end server for a game, then you NEED to use a database. No doubt about it. Use a database or don't do it at all. |
| ||
| the problem ive had with sql before, is access speed with huge record amounts. which is why the txtfile database i was thinking would be superior, Smaller server footprint, quicker searches, and less problems when reaching higher count on records. The gameis likly going to have something like, 500 million entries for buildings, mines, vessles, infantry, mechanized combat units, flying units and armored units, turrets, all to be placed by that particular empire's owner. |
| ||
| I cant pre-determine what all the fields, or field size will be So, why wouldn't something like this work?
Table : AUnit
Field : unitId
Field : ...
Table : AUnitField
Field : unitId
Field : fieldId
Field : fieldType
Field : fieldValue
1 AUnit has many AUnitFields.... nothing hard coded, completely flexible. As Flameduck says, you need to learn about Database Design, because believe it or not, Relational databases are there for a reason - to manage data in a relational way. But you should probably stick with a comma-separated text file "database" system as you'll not need to learn as much. |
| ||
| Real databases also have useful things like Indexing, which if used properly can work very fast. Again, it's down to your design. Design it well, and you shouldn't have any problems. |
| ||
| Hey brucey, could you give me a more extensive example of that? also.. how would that go about saving that information to disc, and retrieving, and editing it? my original question is still valid ;) |
| ||
| quicker searches, Correct me if I'm wrong, but my understanding of searching a plain text file is it starts at the top and basically reads its way down until it finds what you're looking for? And that searching a giant text file, especially for records at the end, would be extremely slow? |
| ||
| ... unless you have lots of text files and hold an index to each in memory. In my inexperienced view a database is very much like a load of text files accessed via an index (or unique key). The advantage is that you can index different columns and use it to select and/or order your data. Using text files *might* work but its going to be re-engineering a solution that somebody has already fixed. |
| ||
| 500 million records using arrays and text files? Are you kidding me? There are many reasons you will need to use a database. Some of the most important ones are: speed - I know you think databases are slow, but reading text files with that many records will give a whole new meaning to slow. Databases are built to handle data like that quickly, using indexing and other features. data security - Databases are transactional. Each write to the database is handled and written. A power loss will lose the last few writes, no big deal. If you hold the data in memory, it is lost at a power loss or program crash. Text files needs to be randomly accessed or the whole file will need to be written at once. Meaning it could ALL be lost with a program crash or power failure. If you are keeping data on an MMO, you cannot keep important data only in memory, it must constantly be written to the hard disk in some manner. With a database, you tell the server, write this data tot he database, then your propgram goes about it's business. With a text file system, your program will have to manage opening the file, finding the correct spot to write the data, writing it, closing the file, etc... Any interruption of that process could destroy the entire file. |
| ||
| "The gameis likly going to have something like, 500 million entries for buildings, mines, vessles, infantry, mechanized combat units, flying units and armored units, turrets, all to be placed by that particular empire's owner." Waaa?! All placed by a human?? Placing half a billion units doesn't sound fun. Will there be a GUI for this, or do they have to edit the text files? ; ) |
| ||
| I think he means 500 million in total. DCI you are probably not going to like all these replies you are getting, but perhaps you have bitten off more than you can chew with this project? If you want that many entities you really need to use "normal" databases. |
| ||
| Its not really important if its the right way to go about it or not. If its a seperate distinct part of the program, either a self contained Type, or a full BlackBox, then DCI can change it latter if the need for such change seems needed. @DCI, I do agree with the others, that you would be better to use a pre-built system, so that you could get along with the meat of your game faster, but I would not fault you for trying to write you own either. However, from your reply when shown an expandable type example, I would highly recomend that you start with something simpler. Maybe write a "Model" of the "Model" of your game, not worring about that you might have 500 million of something, practice other parts of the language with maybe 50 Max. Im not sure if Bmax has a "serialise" command or not, but Im sure that part of these new commands we got last updat are for that, so maybe concider them. Also I would have to agree, that although it may seem that text from file access is faster that SQL (or other), that normaly only in arbitary examples, once you start needing randomly placed files to read, (and in you own decription you have no Idea of Field length), need to read the first half of every other file before you find what you need, the advantage of a pre-existing dbase command set will be joy. Dont forget people spend years writing good Dbases, and its only a fraction of what you need for you program. |
| ||
| Yeah, joking aside, if you want to reinvent the wheel, which is useful for learning purposes, then my advice is the same as that already given: to modify a record, read it in as a type and write the whole thing out. Trying to do text insertions would be a lot of effort for little or no speed increase. |
   |