DataBuffers and how to read/write a part of a file

Monkey Forums/Monkey Programming/DataBuffers and how to read/write a part of a file

Raul	(Posted 2013) [#1]

I have a big file (60-70 mb) and I want to make a "copy-paste" operation using bytes.

Until now, wit little files I used:

data = DataBuffer.Load("file_here.xml")
If data
     Local file:= FileStream.Open("copyHere.xml", "w")
     If file
          file.Write data, 0, data.Length
	  file.Close
     EndIf
EndIf

The problem is that if the file is bigger, the app will crash.

I need to read parts of it and then write them to the new file.

Raul	(Posted 2013) [#2]

I tried a very simple example:

Local data2:DataBuffer = New DataBuffer(512)
     Local file:= FileStream.Open("menu/music.mp3", "r")
     file.Read(data2, 0, data2.Length)
     file.Close()

and it's crashing on my Android tablet.

Markus

(Posted 2013) [#3]

maybe try at glfw target, there is a debug mode.

+

Local file:= FileStream.Open("menu/music.mp3", "r")
if file
     file.Read(data2, 0, data2.Length)
     file.Close()
endif

Raul	(Posted 2013) [#4]

Tried on GLFW:

Memory access violation
file:FileStream is null

rIKmAN

(Posted 2013) [#5]

Sounds silly, but do you have the path/file correct, including case?

Markus

(Posted 2013) [#6]

is the file in projectname.data/menu/ path?

Raul	(Posted 2013) [#7]

yes, the path is correct. double checked several times.

Raul	(Posted 2013) [#8]

Just started a new project, to be sure I am working ONLY with these functions:

Local data2:DataBuffer = New DataBuffer(512)
Local file:= FileStream.Open("menu/music1.mp3", "r")
file.Read(data2, 0, data2.Length)
file.Close()

It is crashing.. The file path is ok :)

Markus

(Posted 2013) [#9]

it is a real mp3 or maybe the wrong extension, maybe not copied to
.apk file archiv.
did you try a other mp3, is your file from web with wrong contents or
header format?

try also
Function FileSize : Int ( path:String )
Return the length of a file, in bytes, or 0 if the file does not exist.

Raul	(Posted 2013) [#10]

FileSize returns 0 :|

I tried also the cpp target. Same result. I cannot understand how is that possible?

It does not matter if the mp3 is valid.. I tried several files type..Simply does not work. I cannot understand how is this possible.

For example, take this one:


Import mojo
Import brl
Import brl.filesystem

Class Game Extends App
		
	Field myImg:Image
	
	Method OnCreate:Int()

		SetUpdateRate 60
		
		
		Local data2:DataBuffer = New DataBuffer(512)
		
		Print "Filezise: " + FileSize("menu/back.png")
		
		myImg = LoadImage("menu/back.png")
		
		Local file:= FileStream.Open("menu/music1.mp3", "r")
		If file
     		file.Read(data2, 0, data2.Length)
     		file.Close()
		EndIf
						
		'EndApp()		
		
		
		Return 0
	End

	Method OnUpdate:Int()
	
		Return 0
	End

	Method OnRender:Int ()				
		DrawImage(myImg, 0, 0)
		Return 0
	End

	
End

it will print myImage is "0" but it will also draw it :)

Raul	(Posted 2013) [#11]

I will commit suicide soon.

I was able to play with the FileStream on glfw target, but I still cannot make it work for Android.

So, PLEASE, is there someone here who knows how to:

1. Open a big file from a web location: "http://rvlgames.com/myfile.mp3"
2. Write the content in "monkey://internal", piece by piece.

I really need to finish this little project soon and I have to make this happens.. :|

AdamRedwoods

(Posted 2013) [#12]

1. a big file? be careful, remember Android's heap size is limited, and varies per device.
2. opening files from web location means it's buffering, so make sure you wait until it is done loading before performing operations.
3. if something is crashing, try to get the debug log and see where it is crashing.

Raul	(Posted 2013) [#13]

well, something between 2 and 3 is killing me.

you said: "so make sure you wait until it is done loading before performing operations."

I have 1 thread only:

Data = DataBuffer.Load("http://www.rvlgames.com/music0.mp3") '	38 mb file						

If data
     Local file:= FileStream.Open("monkey://internal/music.mp3", "w")
     If file
          file.Write data, 0, data.Length
          file.Close
     EndIf
			
EndIf

:)

regarding the debug... any thougts what should I try to search for??

AdamRedwoods

(Posted 2013) [#14]

Ok, so looking at the android source for DataBuffer.Load() it seems it uses an array, which AFAIK, is using heap memory. therefore, a 38mb file will probably hit the heap limit and crash.

You may want to try using a TCPSTREAM instead, which allows you to read-write the mp3 file in chunks, much more manageable for Android memory limits. (assuming memory is the reason it's crashing).

EDIT:
try to post an error log like this guy did here. it really helps!
http://monkeycoder.co.nz/Community/posts.php?topic=5721

Markus

(Posted 2013) [#15]

re:FileSize returns 0 :|
did you use the last monkey version or maybe a old one?

Raul	(Posted 2013) [#16]

@AdamRedwoods:
I see. So that's why is crashing.

I tried to have a look over the TCP example from the bananas/mak folder but honestly cant understand how to "download" a file. there was some example on send messages and receive..

@Markus:
I am using: 71c

Raul	(Posted 2013) [#17]

I tried to run the echoserver_TCP example and I receive

" Error connecting "