Object storage 'framework' using SQLite

BlitzMax Forums/BlitzMax Programming/Object storage 'framework' using SQLite

Arowx(Posted 2009) [#1]
Hi I'm just working on a simple object storage framework, using Bruceys SQLLite module...

The aim is to allow any object and it's fields to be saved and loaded from a SQLite database, using reflection to obtain the data.

The project is just starting out but wondering if anyone has done something similar?

Or can think of good test cases/pitfalls that should be factored in?


Htbaa(Posted 2009) [#2]
Brucey did persistence.mod which stores the data in an XML file. Which is, I think, a lot easier to do then using a SQLite database... At least in the case of serialization.

So why not use that?


Arowx(Posted 2009) [#3]
@Htbaa Can't find a link to persistence.mod in Bruceys modules...

I liked the idea of a database, more modular, and SQLite allows you to use a single flat file...

The plan is to move all of a games media content into a single database file along with the game's object data with the aim of a more data driven approach to development! ;o)

And depending on speed I could also store hi-scores, statistics and even basic event logging!

Mainly I haven't done much with databases for a while and always found XML to be a bit overblown for most uses!


plash(Posted 2009) [#4]
Merx, the persistence module (and pretty much all the other new modules) are on the maxmods Google code project.

http://code.google.com/p/maxmods/wiki/PersistenceModule (it is currently only in the svn repository)


DavidDC(Posted 2009) [#5]
But wouldn't you need blobs for that - which were missing last I looked?


Arowx(Posted 2009) [#6]
Yes I will but I've checked out the SQLite interface for Blobs and it allows you to extract chunks of a Blob at a time, so it's probably best to use a Bank for processing in a two pass approach, get the size of the blob, create a bank of that size and pass in the bank to be filled!

It should be simple enough once, I get my head around C/Blitzmax conversions, the fun part will be integrating/hacking it into the current module.

Unless anyone has a better idea?

PS> I have only got it up to the stage of saving simple objects with int, double and string fields so far!

Next relaoding objects...!


Arowx(Posted 2009) [#7]
Yeah it now saves and loads basic objects [objects with int,double,string fields]!

Cuppa then need to get linked objects to save and load!

If you can think of any tests cases that will enhance this let me know?

I will post the code shortly, hopefully I can improve it and enhance it if the community is interested?


DavidDC(Posted 2009) [#8]
I considered doing this for my current app - so, yes seeing what you come up with would be great thanks.


Mahan(Posted 2009) [#9]
Brucey did persistence.mod which stores the data in an XML file. Which is, I think, a lot easier to do then using a SQLite database... At least in the case of serialization.


That very much depends on what you are trying to achieve. And ORM (Object Relational Mapper) like http://www.sqlobject.org/ (for python) as an example abstracts away lots of normal SQL-CRUD (Create Read Update Delete) while at the same time still having the benefit of fast searches from the database.

If you have 10 million records/objects and you need to look up and alter 100 of them for some processing an XML file serialization is clearly a worse solution than ORM functionality.

ORMs often allow you to either define classes and it will create an table-structure in the database matching these classes or even vice versa.

ORMs are also often quite smart in that if a class has an internal list of other objects (i.e. in database terms - a relation), the ORM often attempts to lazy-read the "sub-objects", i.e. they are looked up and accessed first if/when the program interacts with the SUB-list. This functionality uses so called "proxies" in several languages that supports them. A proxy is a function that is called instead of any function call of a class, to provide extended functionality. The proxy can then, in turn, evaluate the call and call internal functions in the class to get the actual job done.

Sorry for long text. TL;DR version:
Serialization <> ORM (different uses)


Arowx(Posted 2009) [#10]
Hi Mahan,

I'm going for more of a Serialization solution to database the aim is to have a games complete dataset in a flatfile sqlite database and to load all the data from that.

Hopefully this will also allow for moving to a more data driven development approach, so with a simple object editor tool you can update and amend the data, in effect altering the game without having to recode!

An ORM sounds cool but most games go through loading/saving phases and I would like to keep this simple to start with!