SQLite3 for GLFW

Monkey Targets Forums/Desktop/SQLite3 for GLFW

Raz(Posted 2013) [#1]
Hi all,

Having accepted my request in this thread ( http://monkeycoder.co.nz/Community/posts.php?topic=4385 ) Rone has written an SQLite3 wrapper for GLFW :)

It is available here - https://github.com/sascha-schmidt217/sqlite3

An example program :-

Import sql

Class TestApp

	Method New()
	
		Local _db:SQLite3DataBase = New SQLite3DataBase()
		_db.Load("database.db")
		
		_db.Query("DROP TABLE IF EXISTS Books")
		_db.Query("CREATE TABLE Books(Id INTEGER PRIMARY KEY AUTOINCREMENT, Title TEXT, Author TEXT, ISBN TEXT DEFAULT 'not available')")
		_db.Query("INSERT INTO Books(Id, Title, Author, ISBN)VALUES(1, 'War and Peace', 'Leo Tolstoy', '978-0345472403')")
		_db.Query("INSERT INTO Books(Id, Title, Author, ISBN)VALUES(2, 'diesdas', 'Leo Tolstoy', '978-0345472403')")
		_db.Query("INSERT INTO Books(Id, Title, Author, ISBN)VALUES(3, 'blibla', 'Leo Tolstoy', '978-23423')")
		_db.Query("INSERT INTO Books(Id, Title, Author, ISBN)VALUES(4, 'blubla', 'Leo Tolstoy', '978-2342342342')")
		
		Local result:= _db.Exec("SELECT * FROM BOOKS")
		While result.Read()
			For Local e:= Eachin result.StringMap
			      Print e.Key + "/" + e.Value
			End 
		Wend 

	End

End

Function Main()
	New TestApp
End



Difference(Posted 2013) [#2]
Looking good.

What would it take to make this work om html5, iOS, and Android ?


Raz(Posted 2013) [#3]
From a quick google, it appears Android has SQLite stuff built in and with HTML5 it's pretty simple to use as well (ignoring implementing a wrapper for it)!

http://cookbooks.adobe.com/post_Store_data_in_the_HTML5_SQLite_database-19115.html


Rone(Posted 2013) [#4]
added 'SQLite3Statement'


An instance of this object represents a single SQL statement. This object is variously known as a "prepared statement" or a "compiled SQL statement" or simply as a "statement".

The life of a statement object goes something like this:

1. Create the object using SQLite3DataBase.CreateStatement
2. Bind values to host parameters using the statement.Bind*() interfaces.
3. Run the SQL by calling statement.ExecuteStep() one or more times.
4. Reset the statement using statement.Reset() then go back to step 2. Do this zero or more times.


Local statement:= _db.CreateStatement("SELECT * FROM Books WHERE Id > ?")
statement.BindInt(1,3)
		
While statement.ExecuteStep()
	Print statement.GetText(0) + " / " + statement.GetText(1)
End 
	
	
statement = _db.CreateStatement("SELECT * FROM Books WHERE Id > @NNN")
statement.BindInt("NNN",0)
		
While statement.ExecuteStep()
	Print statement.GetText(0) + " / " + statement.GetText(1)
End 


'Transactions' can be used as follows

Try
	db.BeginTransaction()
			
	db.Exec("INSERT INTO test VALUES (NULL, 'test')")
			
	db.EndTransaction()
			
Catch e:SQLite3Exception
	db.RollBackTransaction()
End



Raz(Posted 2013) [#5]
Edit: Ignore below, I've submitted a pull request

----

I wanted to add it as a branch through github Rone, but it wouldn't let me.

Method LastInsertId:Int()
	Local res := Exec("SELECT last_insert_rowid() AS InsertID ")
	res.Read()
	Return Int(res.StringMap.Get("InsertID"))
End



Rone(Posted 2013) [#6]
added xna target(win32/xbox360/wp7)

based on C#-SQLite

https://code.google.com/p/csharp-sqlite/


Difference(Posted 2013) [#7]
Great.

Regarding HTML5 SqLite: After reading a little, I don't think it's the way to go:

http://sqlite.1065341.n5.nabble.com/SQLite-HTML5-and-Javascript-td65238.html
https://plus.google.com/+KevinDangoor/posts/PHqKjkcNbLU

A monkey implementation would be better of doing a native implementation and storing via local storage that is widely supported:

http://html5test.com/compare/feature/storage-localStorage.html

the IndexedDB , Web SQL implementations are totally fragmented


Rone(Posted 2013) [#8]
yes, seems that websql is not supported across all browsers
http://caniuse.com/#search=WebSQL

but converting sqlite3 to moinkey is just too much work...at least for me. sqlite.c has more than 100k lines of code...


A monkey implementation would be better of doing a native implementation...


implementing it from scratch looks like fun...and much much work.
Apart from that I have no idea of the underlying theory
http://db.grussell.org/
http://en.wikipedia.org/wiki/Relational_model
http://en.wikipedia.org/wiki/Relational_algebra


rIKmAN(Posted 2013) [#9]
Is there a possibility of this being made to work on iOS?


Rone(Posted 2013) [#10]
I have no mac, but its c/c++...therefore, it should compile.

Probably a 'FixDataPath' needs to be added and/or maybe a special sqlite3 amalgamation...

Is it already possible to add custom include directories to monkey?

#CPP_INCLUDE_DIR+="modules/sql/sqlite3/*"

#CSHARP_REFERENCE+="sqlite3.dll"
#CSHARP_USING+="sqlite3"

I would like to make it work out of the box...


Difference(Posted 2013) [#11]
How about this for HTML 5 ?

https://github.com/kripken/sql.js


Raz(Posted 2013) [#12]
I would like to make it work out of the box...

If you're able to do that, that's be brilliant for all involved :)


rIKmAN(Posted 2013) [#13]
Has there been any progress with this regarding other targets?


rIKmAN(Posted 2013) [#14]
Can anyone confirm this still works with recent versions of Monkey?
I am using v74a.


AdamRedwoods(Posted 2014) [#15]
i had to add in native/sqlite.cpp
#include <string>

and change
String GetText(int index)
	{
		const char* text = (const char*)sqlite3_column_text(stmt, index);
	    return String(text, strlen(text));
	}



rIKmAN(Posted 2014) [#16]
Thanks Adam, that got it working for me. :)


rIKmAN(Posted 2014) [#17]
Had a chance to play with this a little, and I'm getting a weird problem with the results it returns being in the wrong order.
I thought it may have been my code, but tested it with the code in the OP and got the same results.

From the code in the OP I am getting the following results:
Author: Leo Tolstoy
ISBN: 978-0345472403
Id: 1
Title: War and Peace
Author: Leo Tolstoy
ISBN: 978-0345472403
Id: 2
Title: diesdas
Author: Leo Tolstoy
ISBN: 978-23423
Id: 3
Title: blibla
Author: Leo Tolstoy
ISBN: 978-2342342342
Id: 4
Title: blubla

I was expecting the data back in the order of the SELECT query or if * then the order of the columns as defined in the table, so I expected:
ID
Title
Author
ISBN


I tried a few tests with my own code: and calling 2 TEXT columns in the SELECT worked as expected (SELECT text1, text2 FROM table) returned value1/value2, value1/value2 etc...

I then added an INT column (SELECT int1, text1, text2 FROM table) and this returned the data in a different order: text2/text1/int1, text2/text1/int1 etc
I was expecting it be as I had listed it in the SELECT query: int1/text1/text2.

I then thought it may be reversed, but the output from the OP also disproves that, so I'm now at a loss.

It doesn't seem like many people have used / use this, but any help or ideas on what is going on appreciated.


rIKmAN(Posted 2014) [#18]
Anyone?


Raz(Posted 2014) [#19]
Hi rIK, I know it's not the point but, is it necessary for you to know the order of the returned values?


rIKmAN(Posted 2014) [#20]
Hey Raz, thanks for the reply.

Well yeah it kind of is, lets say I run a query which returns 20 results which are the stats of a character in a game.
I run the query, get the results and then can iterate through this to draw the values on the stats screen at the correct place.

Time passes, things get added to the character, more stats etc, so I then go back and adjust the SELECT query so we can now show 30 stats.

What SHOULD happen: I add the 10 new columns to the end of the SELECT query, and then adjust my code to read these 10 new values from the end of the result - fine, nice and easy and what I thought would happen/

What DOES happen: I add 10 new columns to the SELECT query, and adjust my code to read these 10 new values from the result.
However, the result has now been jumbled up in comparison to the order of the SELECT statement, and even the existing order is changed and everything just seems to be placed in a completely random order.

My stats screen is now showing completely wrong data and I have to go back through and see what order results are are returned, adjust the display code to make sure it is reading the data in the new order, and do this each time I decide to add another column into a search query.

Now, I might have got this completely wrong, but this seems completely unworkable and there must surely be a way to define the order of results like you do in normal SQL, otherwise how do you know what order to expect the data back, and thus what to do with the results?

Trial and error each time I add to the database isn't really an option, time wise, but if I have cocked up somewhere and am completely misunderstanding something, please enlighten me! :)

Thanks.

PS If I'm not explaining very well let me know and I will try and give a better example.


Raz(Posted 2014) [#21]
Ok, been messing with it some more and I don't know why but, if you change ISBN to Isbn, you will find the columns are returned in alphabetical order when using SELECT * FROM

Something to do with StringMap maybe?

Import sql

Class TestApp

	Method New()
	
		Local _db:SQLite3DataBase = New SQLite3DataBase()
		_db.Load("database.db")
		
		_db.Query("DROP TABLE IF EXISTS Books")
		_db.Query("CREATE TABLE Books(Id INTEGER PRIMARY KEY AUTOINCREMENT, Dog INTEGER, Cat TEXT, Title TEXT, Author TEXT, Isbn TEXT DEFAULT 'not available')")
		_db.Query("INSERT INTO Books(Id, Dog, Cat, Title, Author, Isbn)VALUES(1, 5, 'cat', 'War and Peace', 'Leo Tolstoy', '978-0345472403')")
		
		Print "Alphabetical order"
		Print "----------"
		Local result:= _db.Exec("SELECT * FROM BOOKS")
		While result.Read()
			For Local e:= Eachin result.StringMap
			      Print e.Key + "/" + e.Value
			End 
		Wend 
		
		Print "--------------------------------"
		Print "Alphabetical order"
		Print "----------"
		result= _db.Exec("SELECT Id, Dog, Cat, Title, Author, Isbn FROM BOOKS")
		While result.Read()
			For Local e:= Eachin result.StringMap
			      Print e.Key + "/" + e.Value
			End 
		Wend 
		
		Print "--------------------------------"
		Print "Uppercase S goes before lowercase d"
		Print "----------"
		result= _db.Exec("SELECT Id, Dog, Cat, Title, Author, Isbn as ISBN FROM BOOKS")
		While result.Read()
			For Local e:= Eachin result.StringMap
			      Print e.Key + "/" + e.Value
			End 
		Wend 

	End

End

Function Main()
	New TestApp
End



zardon(Posted 2014) [#22]
The lack of a good SQLite / Active Record / ORM library is perhaps the one reason I haven't picked up Monkey yet and am hoping a good solution very soon


Rone(Posted 2014) [#23]
Hey,
here is the fix...
Sqlite3Result.GetString is also fixed...

** added SQLEntry

change Sqlite3Column in sql.monkey as follows:
Class SQLEntry
	Field Key:String
	Field Value:String
	Method New(key:String, value:String)
		Key = key
		Value = value
	End 
End 

Class SQLite3Column

	Field items:StringMap<String>
	Field arr:SQLEntry[]
	
	Method New(name_arr:String[], value_array:String[])
		items = New StringMap<String>
		Local length = name_arr.Length
		For Local i = 0 Until length
			items.Set(name_arr[i], value_array[i])
		End 

		arr = New SQLEntry[length]
		For Local i = 0 Until length
			arr[i] = New SQLEntry(name_arr[i], value_array[i])
		End 
	End 
	
End 


Sqlite3Result must also be changed...

Class SQLite3Result

	Field columns:= New Stack<SQLite3Column>
	Field length:Int = 0
	Field index:Int = 0
	
	Method Add(name_arr:String[], value_array:String[])
		columns.Push(New SQLite3Column(name_arr,value_array))
		length+=1
	End 
	
	Method Read?()
		index+=1
		Return length - index >= 0
	End 
	
	Method Reset()
		index = 0
	End 
	
	Method GetString$(_index)
		Return columns.Get(index-1).arr[_index].Value
	End 
	
	Method GetString$(name$)
		Return columns.Get(index-1).items.Get(name)
	End 
	
	Method StringMap:SQLEntry[]() 
		Return columns.Get(index-1).arr
	End 
	
	Method Columns%()
		Return length
	End 
	
End



rIKmAN(Posted 2014) [#24]
Raz and Rone - thank you both for your time and the fix! :)

I'll be able to test this out tomorrow, thanks again.


Kaltern(Posted 2014) [#25]
Getting errors when compiling using this - have followed the installation instructions.



What am I not doing right? :|


Kaltern(Posted 2014) [#26]
BTW I managed to compile in VC2010, so I guess I'll just use that instead :)