Plotting Pixels in an Image
Monkey Forums/Monkey Beginners/Plotting Pixels in an Image
| ||
| Is there anyway I can plot pixels inside an image ? like an old school plot x,y routine, there is no "lockimage" or anything like that as in BlitzMax? I need to be able to plot pixels inside a 320 x 200 pixel image and cant do it at speed using DrawRect Thanks. [edit] found Createimage and WritePixels but no examples and instructions very sparse, any one have any examples for a newbie like me ? |
| ||
| Do you need to save the image later on, or have any reason to actually change it's contents? can't you draw on top instead?, what's the target anyway? |
| ||
Example of CreateImage and WritePixelsImport mojo Function Main:Int() New Example() Return 0 End Class Example Extends App Method OnCreate:Int() SetUpdateRate(15) image = CreateImage(320, 200) Local pixels:Int[320 * 200] For Local i:Int = 0 Until 320 * 200 Local x:Int = i Mod 320 ' <-- unnecessary, but this is how you find x Local y:Int = i / 320 '<-- unnecessary, but this is how you find y ' if you need to plot points from x and y ' y * 320 + x = i pixels[i] = GenerateColor(i) Next image.WritePixels(pixels, 0, 0, 320, 200, 0) Return 0 End Method OnRender:Int() Cls() DrawImage(image, 10, 10) Return 0 End Method GenerateColor:Int( i:Int ) Local rgb:Int = 255 * (Abs((i / 255.0) - 0.5) * 2.0) Return rgb | (rgb Shl 8) | (rgb Shl 16) End Field image:Image End |
| ||
| I also found an example last night for : LoadImageData() then computer shutdown and cant find it now ? I was wondering if LoadImageData() is fastest or your example (I need high performance) Tried your example on a Droid, 2 FPS Tried in on a PC, 60 FPS but near 100% CPU load, so nothing left for rest of code. I have done this in X86 assembler and CPU load is 1% I suspect there is a bottleneck around plotting one pixel at a time, I need to move all the pixels to the image at one time, something to do with LoadImageData() etc, cant find examples |
| ||
| Switch off the function GenerateColor() to check the speed of the WritePixels() function. The sample above sets all 640.000 pixels of the image. In reallity you will have less calls when f.e. drawing a function. If you plot a linear graph you will normally have to draw not more than 320 pixels. The speed will be 200 times faster! Your result of 2FPS may be a mistake... Because the first chance to check FPSs is after OnCreate(). But the test is already finished niside OnCreate()! A second way (in cases of a lot pixels) is to draw the current image onto the screen, then plot the pixels on the screen too, then grab the screen with ReadPixels() into an array then (re-)create the image from the array with WritePixels() |
| ||
| . |
| ||
| . |
| ||
| grab the screen with ReadPixels() into an array then (re-)create the image from the array with WritePixels() Unfortunately WritePixels() is impossible slow on Android (Nexus 7 2013) with 1920x1200 screen. 960x600 is still very slow, 480x300 becomes useable. |
| ||
| Test code to show the problem with Android. Code runs fine with targets: iOS, HTML5, Flash, GLFW On Android, "time between OnRender" when trying to draw something: test_mode = 1 (WritePixels() with whole image data) 18 ms with 128x128 image (55 FPS) 50 ms with 256x256 image (20 FPS) 100 ms with 512x512 image (10 FPS) 400 ms with 1024x1024 image ( 2.5 FPS) test_mode = 2 (WritePixels() with 3x3 size only) 18 ms with 128x128 image (55 FPS) 50 ms with 256x256 image (20 FPS) 100 ms with 512x512 image (10 FPS) 320 ms with 1024x1024 image (3 FPS) Frame rate goes down rapidly with WritePixels() and images bigger than 128x128. Only with Android! All other targets are fine, 1024x1024 image and always under 20ms (50+ FPS) with iOS, HTML5, Flash, GLFW. Looks like something is wrong with the Android stuff. '#ANDROID_NATIVE_GL_ENABLED = True
#MOJO_IMAGE_FILTERING_ENABLED=False
Import mojo
Function Main:Int()
New Example()
Return 0
End
Class Example Extends App
Global test_mode:Int = 2 ' 1 = copy whole image
' 2 = copy only 3x3 rect
Const ScreenW = 512 ' 128 256 512 1024
Const ScreenH = 512 ' 128 256 512 1024
Method OnCreate:Int()
SetUpdateRate(60)
image = CreateImage(ScreenW, ScreenH)
For Local i:Int = 0 Until ScreenW * ScreenH
pixels[i] = $FF404040
Next
image.WritePixels(pixels, 0, 0, ScreenW, ScreenH, 0)
If test_mode = 2
For Local i:Int = 0 Until ScreenW * ScreenH
pixels[i] = $FFFFFFFF
Next
Endif
Return 0
End
Method OnUpdate:Int()
counter += 1
Local time := Millisecs()
timeBetweenOnUpdate = time - oldUpdateTime
oldUpdateTime = time
If TouchDown(0)
Local x:Float = TouchX(0)
Local y:Float = TouchY(0)
'Print "DOWN " + x + ":" + y
If x < ScreenW-4 And y < ScreenH-4 Then
If test_mode = 1
For Local i:=0 To 2
pixels[Int(y) *ScreenH + Int(x)+i] = $FFFFFFFF
pixels[(Int(y)+1)*ScreenH + Int(x)+i] = $FFFFFFFF
pixels[(Int(y)+2)*ScreenH + Int(x)+i] = $FFFFFFFF
next
image.WritePixels(pixels, 0, 0, ScreenW, ScreenH, 0)
Elseif test_mode = 2
image.WritePixels(pixels, Int(x), Int(y), 3, 3, 0)
Endif
Endif
Endif
updateEnd = Millisecs()
onUpdateTime = updateEnd-time
Return 0
End
Method OnRender:Int()
Local time := Millisecs()
timeBetweenOnRender = time - timeBetweenOnRender
Cls()
DrawImage(image, 0, 0)
DrawText("time required for OnUpdate: "+onUpdateTime,10,10)
DrawText("time between OnUpdate and OnRender: "+(time-updateEnd),10,30)
DrawText("time between OnRender: "+timeBetweenOnRender,10,50)
DrawText("Counter: "+counter,10,70)
timeBetweenOnRender = time
Return 0
End
Field timeBetweenOnUpdate:Int, oldUpdateTime:int
Field timeBetweenOnRender:Int
Field onUpdateTime:Int, updateEnd:Int, counter:Int
Field image:Image
Field pixels:Int[ScreenW * ScreenH]
End |
| ||
| A 1024x1024 sized integer array is about 4 million bytes (4 MB). That's quite a lot for a device with limited memory and to pack all into one space. It'd probably be faster to use a 1024 sized integer array and do each line separately 1024 times. Or whatever optimal size is optimal without causing slow downs. |
| ||
| 4MB is nothing. I tested with Google Nexus 7 2013 and the device has 2GB RAM. The GPU is an Adreno 320, which seems to be a quite powerful thing. The allocation of the 4MB array does not cause any slowdown, as it runs at 55 FPS when not touching the screen. As soon as I touch the screen, and WritePixels is called, the FPS goes down rapidly with images at sizes 256x256, 512x512, 1024x1024. |
| ||
| I've seen an array that size cause slowdowns before. Aside from that though, the issue could be mitigated by not trying to write all of the pixels on one update. |
| ||
| Aside from that though, the issue could be mitigated by not trying to write all of the pixels on one update. It is what test_mode 2 is for. It copies only a 3x3 rect (9 pixels), but looking at the resulting output numbers... they are quite the same. I have images at 256x256, 512x512, 1024x1024. Using WritePixels() on those images with 3x3 rect or with full size does not make a difference. What's the results on your Android device(s)? Several guys here have written that Android WritePixels() is very slow for them. Midimaster is the only exception, he says everything is fine for him. That's the reason I would like to resolve the issue and discuss what may be the problem with Android WritePixels(). It would be nice to use GrabImage(), manipulate that small image only, and draw the manipulated image back onto the original. Unfortunately I can't find a command to draw an image onto another image. Using WritePixels() does not help, because it slows down to 3 FPS, even if i write only a 3x3 rect onto a 1024x1024 image. Here is the code with only an 3x3 array and WritePixels() only using the 3x3 array, so we can see the 4MB array size thing is not the issue: '#ANDROID_NATIVE_GL_ENABLED = True
#MOJO_IMAGE_FILTERING_ENABLED=False
Import mojo
Function Main:Int()
New Example()
Return 0
End
Class Example Extends App
Const ScreenW = 1024 ' 128 256 512 1024
Const ScreenH = 1024 ' 128 256 512 1024
Method OnCreate:Int()
SetUpdateRate(60)
image = CreateImage(ScreenW, ScreenH)
pixelbuffer = New Int[ ScreenW * ScreenH ]
For Local i:Int = 0 Until ScreenW * ScreenH
pixelbuffer[i] = $FF404040
Next
image.WritePixels(pixelbuffer, 0, 0, ScreenW, ScreenH, 0)
pixelbuffer = pixelbuffer.Resize(1)
Print( pixelbuffer.Length() )
For Local i:Int = 0 To 8
pixels[i] = $FFFFFFFF
Next
Return 0
End
Method OnUpdate:Int()
counter += 1
Local time := Millisecs()
timeBetweenOnUpdate = time - oldUpdateTime
oldUpdateTime = time
If TouchDown(0)
Local x:Float = TouchX(0)
Local y:Float = TouchY(0)
'Print "DOWN " + x + ":" + y
If x < ScreenW-4 And y < ScreenH-4 Then
image.WritePixels(pixels, Int(x), Int(y), 3, 3, 0)
Endif
Endif
updateEnd = Millisecs()
onUpdateTime = updateEnd-time
Return 0
End
Method OnRender:Int()
Local time := Millisecs()
timeBetweenOnRender = time - timeBetweenOnRender
Cls()
DrawImage(image, 0, 0)
DrawText("time required for OnUpdate: "+onUpdateTime,10,10)
DrawText("time between OnUpdate and OnRender: "+(time-updateEnd),10,30)
DrawText("time between OnRender: "+timeBetweenOnRender,10,50)
DrawText("Counter: "+counter,10,70)
timeBetweenOnRender = time
Return 0
End
Field timeBetweenOnUpdate:Int, oldUpdateTime:int
Field timeBetweenOnRender:Int
Field onUpdateTime:Int, updateEnd:Int, counter:Int
Field image:Image
Field pixels:Int[3 * 3]
Field pixelbuffer:Int[]
End |
| ||
| I'm not at my workstation, but I'll look more into the slowdowns on Android tomorrow. Maybe I can resolve it. It wouldn't be the first thing I've resolved on the targets. Edit: I just grabbed the source code on my computer here to take a look. WritePixels2 (which is WritePixels in this case), is doing some pretty standard bitmap modifications. From a first glance everything seems like it's Android's fault. Hmm... |
| ||
| Looking at mojo.android.java, I see many operations like converting RGBA to ARGB, changing buffer order to BIG_ENDIAN and LITTLE_ENDIAN, etc. Re-arranging 4 million bytes for sure takes some time. And all this on byte basis, using many shifts, binary And's, and multiplications - for every single pixel. |
| ||
| I didn't see that in WritePixels. Maybe I was looking in the wrong area. But yeah, re-arranging 4 million bytes would almost certainly cause slowdowns. Edit: Read pixels does that. Write Pixels clears the GL buffer context, uses what appears to be a built-in pixel setting function and then invalidates the surface to force it to redraw. Nothing out of the ordinary really. Hmm: https://stackoverflow.com/questions/4715840/improving-speed-of-getpixel-and-setpixel-on-android-bitmap |
| ||
| Indeed, you should probably do it every frame. On some platforms, doing stuff in OnCreate() may give bad results simply because there is some startup stuff going on around that time which slows every program. But if the target is Android, see the thread on a 'finger painting app' in the Android forum. There are faster ways to manipulate images on Android compared to what Monkey does. |
| ||
| Indeed, you should probably do it every frame. On some platforms, doing stuff in OnCreate() may give bad results simply because there is some startup stuff going on around that time which slows every program. But if the target is Android, see the thread on a 'finger painting app' in the Android forum. There are faster ways to manipulate images on Android compared to what Monkey does. |
| ||
I became all jazzed up to fix this problem. Testing a 1024x1024 image it took an astonishing 1176 ms from my test. Ouch. I looked at the actual source code for the Android SDK. Here's the function for setPixels.public void setPixels(int[] pixels, int offset, int stride,
int x, int y, int width, int height) {
checkRecycled("Can't call setPixels() on a recycled bitmap");
if (!isMutable()) {
throw new IllegalStateException();
}
if (width == 0 || height == 0) {
return; // nothing to do
}
checkPixelsAccess(x, y, width, height, offset, stride, pixels);
nativeSetPixels(mNativeBitmap, pixels, offset, stride,
x, y, width, height, mIsPremultiplied);
}It uses whatever the device's native function call is for setting pixels. From what I was reading, the only real way to speed it up is the use the NDK. How lame, I was hoping for something more easily fixable. Edit: Well, I'm going to sleep. Meant to go to sleep hours ago but got distracted by this. |
| ||
| On my new Droid. it takes around half a second per frame @320 x 240 - wow |
| ||
| Just commented out the line "surface.Invalidate();" in WritePixels2() and tested again. 1024x1024 image: mode 1 (full image): 80 ms = 12.5 FPS (was 400ms = 2.5 FPS) mode 2 (3x3 rect): 18 ms = 55.5 FPS (was 320ms = 3 FPS) Android's setPixels() may not be the fastest. But even more time is spend in Monkey X functions, because surface.Invalidate() adds the texture to discarded textures. This leads to glDeleteTextures() in function FlushDiscarded(). The texture gets deleted and later re-created with a call to Bind(), and Bind() contains all that slow buffer re-arranging stuff I was talking about. OK so far. I re-enabled "surface.Invalidate();" in WritePixels2(), and now I changed line 118 in function gxtkGraphics.Flush() from "surf.Bind();" to "surf.Bind2();" to test the the new ***experimental*** Bind2(). With 1024x1024 image and the 3x3 rect mode I get now 58 FPS. Yes, changing the magic line 118 in mojo.android.java from Bind() to Bind2() made the FPS jump from 3 to 58 FPS! :D |
| ||
| Yes, changing the magic line 118 in mojo.android.java from Bind() to Bind2() made the FPS jump from 3 to 58 FPS! ***** v79c ***** Removed 'Flush' calls from mojo native WritePixels2 functions - shouldn't be necessary and causes issues outside OnRender. Well, the problem was resolved officially in another manner. I never even tried to disable the call to flush. |
| ||
| Well, the problem was resolved officially in another manner. I never even tried to disable the call to flush. That's a different thing. I tried latest experimental v79c this morning and it did not speed-up the Android WritePixels() issue. v79c contains a bug fix for http://www.monkey-x.com/Community/posts.php?topic=8512 Bind() and ***experimental*** Bind2() are another thing. After changing the Android bitmap pixels, the OpenGL texture must get re-created/updated. Without surface.Invalidate() the OpenGL texture gets not updated. The experimental Bind2() function shows they are aware of the issue and they are working on it. |
| ||
| Flush calls Bind, which is what I assumed to be causing the slowdown. I misunderstood that it's for something else. |
   |