Simple finger drawing app question

Monkey Targets Forums/Android/Simple finger drawing app question

Artem Kuchin(Posted 2014) [#1]
I am new to monkey-x and started with something simple to learn. Like multitouch figer drawing app.

And within and house i realised that the way i approached it is totally wrong. I just draw thick rounded lines (rectables+circle) as finger moves. All points are put into an array of point object. Each onrender clear screen and redraws all lines.
As number of lines grow this becomes painfully slow. So, basically it is impossible to fully draw the screen. Even the idea is also wrong because it will becomes slower and slower if i draw above the already drawn lines.
It is bad, because i wanted color animation on timer, but i now i see that's impossible

So, I thought that a good way will be to draw somewhere else then create an image from that and on onrender just blit the image onto the screen. So, the overhead will be always fixed and will not depend on number of lines, points.

However, i don't see any way to draw into some other buffer except the frame buffer.
So, as i see it i have only one option: create my own drawing functions (for line and circle), draw everything in an array and then writepixels into an images. OnRender just show the image.

I am correct here or there is some other way?


Gerry Quinn(Posted 2014) [#2]
I think you have to do a mixture of both.

Periodically (probably when the user finishes a painting motion) you can load the contents of the back buffer into an image.

Then, when the user is doing an actual painting action, you draw the saved image to the screen, and then overlay the circles, lines etc. from the new painting action (as you are doing now). Then you can forget about all the stored individual items (except when making an Undo system).

When he lifts his finger, again you copy from the back buffer to the current image and clear the stored actions, and so on,


Artem Kuchin(Posted 2014) [#3]
Oh, yes, that can do it do.
But i wonder why i cannot use an image as a buffer and draw there. OR use some OTHER buffer, draw there using standard tools and then convert it into image and blit into the current frame buffer.
Is it mojo limitation or underlying graphics system (is it gles?) limitation?


Artem Kuchin(Posted 2014) [#4]
I was thinking further but i don't have deep knowledge how things work. So, maybe someone else will help me.
The question: Do i really need to redraw everything on every frame? Do CLS and then draw everything? Maybe i can leave it as it is and update it only when application is was stopped and resumed?


Midimaster(Posted 2014) [#5]
In my Ballerburg game (GooglePlay) you can see a manipulation of the castle image. The user can "repaint" destroyed castle with his fingers.

At the beginning I create an empty INT-array with screen size.
Field Original%[] = New Int[240*400]


A new image will be created from it once.
Field CastleImage:Image = CreateImage(240,400)
CastleImage.WritePixels(Original, 0, 0, 240,400)


In OnRender() only this image will be displayed.

Later all "finger paintings" cause manipulations im this array.
Method Manipulate:Void(X%,Y%,B%,H%)
		For Local j%=0 To H
			For Local i%=0 To B
					Original[(j+Y)*240+(i+X)]=$FFFF0000
			Next
		Next
		CastleImage = CreateImage(240,400)
		CastleImage.WritePixels(BurgArray, 0, 0, 240,400)
	End

If the array has been changed the image will be created new from it.
In OnRender() the image will display the new state.


Gerry Quinn(Posted 2014) [#6]
"But i wonder why i cannot use an image as a buffer and draw there". I suspect on a lot of Android hardware this may not be practical because the hardware is designed to draw to the frame buffer. Of course it could be done in software (using your own drawing functions that target an Int array), but then you'd have two different drawing systems, and a slower speed.

For your application, I see no benefit, because you will be drawing into the frame buffer anyway.

"Do i really need to redraw everything on every frame?" I think on some platforms you probably do, but I'm not certain. You could try it out and see. Though with Android it wouldn't surprise me if it varied between devices.


Artem Kuchin(Posted 2014) [#7]
Midimaster:
Yes, i was thinking about it, but that would mean creating my own drawing function for all cases: lines, circles, filles and outlines, blending, blitting and other stuff. Too much work

Gerry:
I am new to android and gles, but i though that mojo creates a surface where i draw. Why not:
1) create another surface and let me choose where i want to draw and what surface to show?
2) just redraw the surface with all my drawings on it without clearing?


Midimaster(Posted 2014) [#8]
It was not so much work....

As a first step you try to create a ARRAY-DRAWING of one pixel:
Method Manipulate:Void(X%,Y%)
	Original[Y*240+X]=$FFFF0000
End

Then you can develop more and more functions based on this. It is terrible fast on android...


Artem Kuchin(Posted 2014) [#9]
Maybe developing is terribly fast but manipulating array data in such way IMHO is terribly slow.
When you draw anything through GL you use hardware or native driver. Hardware is the faster way, probably 1000 times faster than working with arrays in java.
Software driver emulation is still very fast because C has pointer arithmetics, so you don't have to do
[Y*240+X], you just make a pointer and increment it and write into memory like *p=x and you also have memory block copy and fill instructions.
Basically it is as fast as machine code can be. Any drawing in an array in java is 2-3 orders of magnitude slower.
But i admit that is is a way to go some cases.

I think texture drawing is the way to go without inventing a bicycle.
I am need to learn the way gles works. Maybe i am missing something.


Artem Kuchin(Posted 2014) [#10]
I've read some stuff by now and i see that as of GLES 2 it is possible to render anything into a texture then then use that texture for anything.
So, as i see it i can have a number of allocated frame buffers where i can render anything and then i choose any of them to texture a surface shown
on a screen. So, onrender will only retexture thesurface if needed and all drawing is done other places only when needed.
What do you think?


Artem Kuchin(Posted 2014) [#11]
I have been experimenting a little with this app. So, i used mixed algorithm. I draw lines, when they reach just 20 i draw them, readpixels, write into image, reset pixel count.
On render i just draw the image and lines up to 19. Don't even have to do Cls.

Issues
1) On HTML5/GLFW target ReadPixels blinks screen - very annoying, but everything works very fast
2) On ANDROID readpixels or writepiexls takes over 1 second on my Xperia Ion (screen 800x480) - UNUSABLE

So, must try render to texture and maybe some other low level android specific way to go (i have no clue).

This is funny, how a simple task appears to be unimplementable in simple ways.


Gerry Quinn(Posted 2014) [#12]
Hmm, I've used ReadPixels/WritePixels on Android with no problem, but I just use them for creating graphics at the start of a level, so I never paid any attention to the speed. A second is a long time, though, I find it very surprising it should take so long. Are you certain you aren't doing something repeatedly that eats up all the time?


Artem Kuchin(Posted 2014) [#13]
Yes, i am sure. I timed ReadPixels with Millisecs
When the points are copied into an image ReadPixels is timed and then
showed in the upper left corner. My number are between 250-340 - TOO LONG. Write pixels takes 30ms - TOO long too actually. But total time somehow feels close to 0.5-1s


Artem Kuchin(Posted 2014) [#14]
Digging it further
I have rewritten it to draw into an array and writepixels then. The code

Import mojo

Class Game Extends App
    Field image:Image
    Field points:Point[]
    Field _start_r:Int=255
	Field _start_g:Int=0
	Field _start_b:Int=0
	Field _start_status:Int=1
	Field _frame_counter:Int=0
	Field _touch_state:Int=0
	Field pixelbuffer:Int[]
	Field bufferimage:Image
	Field pn=-1
	Field f_first=1
	Field last_x=-1000
	Field last_y=-1000
	Field lp_x=-1000
	Field lp_y=-1000
	Field t_x
	Field t_y
	Field tt
	Field tt2	
	Field k
	Field f
	Field dw
	Field dh

    Method OnCreate()
        SetUpdateRate 15
        bufferimage=CreateImage(DeviceWidth(), DeviceHeight )      
       	pixelbuffer=New Int[DeviceWidth()*DeviceHeight()]
       	Self.MyCls(pixelbuffer,$FF000000)
		bufferimage.WritePixels(pixelbuffer,0,0,DeviceWidth(),DeviceHeight());
		dw=DeviceWidth()
		dh=DeviceHeight()
    End
    

    Method OnUpdate()
    	If TouchDown(0) Then
    		_touch_state=1
    		t_x=TouchX(0)
    		t_y=TouchY(0)
	    	If lp_x<>t_x And lp_y<>t_y Then
	    		k=k+1
	    		tt=Millisecs()
#rem
				If lp_x>-1000 And lp_y>-1000 Then
					MyLine(pixelbuffer,lp_x,lp_y,t_x,t_y,DeviceWidth(),8,$FFFFFF00)
				Else
					MyCircle(pixelbuffer,t_x,t_y,8,DeviceWidth(),$FFFFFF00)
				Endif
#end
	    		bufferimage.WritePixels(pixelbuffer,0,0,dw,dh);

	    		lp_x=t_x;
	    		lp_y=t_y;
	    		tt=Millisecs()-tt
    		End
    	Else 
    		If _touch_state Then
    			Print "break!"
    			_touch_state=0
	    		lp_x=-1000
	    		lp_y=-1000
	    	End
    	End
    End
    

    Method OnRender()
	    tt2=Millisecs()
		DrawImage(bufferimage,0,0);
		tt2=Millisecs()-tt2
		f=f+1

			DrawText(tt+ "    "+tt2+ "    "+k+ "   "+f,50,50)

    End



	Method MyCls(buffer:Int[],color#)
    	Local l=buffer.Length()
    	For Local i=0 Until l
    		buffer[i]=color
    	Next
	End

End



Function Main()
    New Game()
End


    



As you see i commented out actual drawing. So, it only writepixels.
It works fine on HTML5 and windows, but terribly slow on android.

Funny that tt (time for WritePixels) shows only 30-50ms, BUT when i touch a point on the screen the frame counter freezes for a half of a second. I do not understand how it happens. WritePixels returns after 30-50ms but app is frozen until bitmap is reloaded into graphics RAM?

So, drawing into and array is also not a way to go.

I looked at the write piexls, it uses setPixels on a bitmap attached to a surface and then invalidates it. So, basically, as i understand, it reload the full bitmap
into graphics memory from RAM - this take long time.

So, currently there is no way to implement finger drawing app in using current monkey modules. Am i right?


Gerry Quinn(Posted 2014) [#15]
You're doing a lot in one render when you reset. You read the pixels. Then you write the pixels to the image. Then you re-draw the image (which means it has to get loaded into graphics memory). Then you draw the new points again.

You could have an 'else' statement after the read/write, as the image is already drawn. However there will be some delay on the next render as it goes into graphics memory.

If it's slow it's slow, though I expect it is device dependent. There are ways the issue could be ameliorated. For a start, you clearly should never reset except when the user finishes a drawing action. Maybe even use some clever algorithm to monitor his movements and make sure he has stopped for a certain fraction of a second. There's also the possibility of using 'dirty rectangles' if the drawing is typically restricted to part of the image.

That said, I am only throwing out suggestions here. Have you checked the speed of other finger-painting apps on your device? You could also google for discussions of how to maximise the speed of such apps in any language - that might at least give a hint as to what has to be done.

EDIT: re-reading your post I see you have identified the problem as the loading the image into graphics memory. I guess the only way to avoid that would be if you can use external code to get the data there more directly. Don't know if that's possible or not, i.e. I don't know if there is a hardware pipeline from the frame buffer to loaded textures, or whether there's no option but to go through RAM.


Artem Kuchin(Posted 2014) [#16]
Other finger drawing apps work fine. I have like 5 of them.

Check out my message with rewritten code: no reset there at all.

EDIT
I've seen some example. Some use the same basic way store points, redraw them
in onDraw - that's wrong way.

Now look at this:
http://www.java2s.com/Code/Android/User-Event/Usingyourfingertodraw.htm

It uses canvas. It work like this
bitmap is attached to canvas
canvas is attached to view

then i can draw on canvas which means drawing in bitmap directly

I don't know how compile the example just because have never done it before, so, cannot check the speed. But i guess it will be very good.
However, mojo does not provide canvas because it work on open gl.
And in open gl, as i understand it, to do the same thing i need to render into texture.

I am stuck. Need to create a canvas alternative to mojo? Sick.


Gerry Quinn(Posted 2014) [#17]
Well, if the code on that page is actually fast, I guess you could make extern functions to access bitmaps the same way. I would test it first, though. But if other apps can do it, it obviously must be possible.


Artem Kuchin(Posted 2014) [#18]
Today I've learned how to build android SDK projects using command line tools and since i am familiar with java, good at C++ and very good at plain C it took me 10 minutes to modify the example to draw the way i want.

Result: amazingly fast and clean finger drawing app.

Alas, i hoped to use monkey as rapid development tool for simple things like this but it seems like i will waste more time struggling with it. Hope it will be useful in other apps, this one i'll make in java.


Raph(Posted 2014) [#19]
Is the code you ended up with something that should go into Monkey's core?


Gerry Quinn(Posted 2014) [#20]
Certainly it seems like it would be worth having a module to access bitmaps. It's worth noting that the whole ReadPixels/WritePixels thing was kind of an afterthought, and mainly put in to cater for people who wanted to procedurally generate level backgrounds etc. In most such cases it's not time-sensitive.


Artem Kuchin(Posted 2014) [#21]
You don't get it. Currently mojo uses GLES and attached surface to one buffer and that's what you see. All drawing is done on that buffer.
No way you can attach canvas to the same view. It is either GLES or canvas. So, to use the same canvas code i found and used in java one need to write a totally
different mojo.graphics module.
OR we can finally do the render to texture feature in the mojo. And it has been done by someone of the forum for GLFW (only) target, but it was a quick and dirty hack and requires minor mojo change.
I am really not in the mood to do it all in java for android. As i read the forum the problem is long time outstanding and nothing was done about it. But i guess the problem i met on this app is relativelly rare. Usually you can get away by usual rendering.
For now i figured that it pay off to use the original tools and languages for a platform.


Gerry Quinn(Posted 2014) [#22]
Indeed, this seems to be a case where Mojo as designed is going to be limited and you need to switch to something else. I do think it's fairly unusual, though, generally things you want to do work pretty well and of course it's multi-platform.


Artem Kuchin(Posted 2014) [#23]
just a note: libgdx is multiplatform too, but it support FBOs and renders to texture.