Issue reading/drawing pixels in a "virtual" canvas

Monkey Forums/Monkey Programming/Issue reading/drawing pixels in a "virtual" canvas

MonoDesire(Posted 2016) [#1]
Hi all,

Using mojo2, I have an issue reading pixels from a canvas and then drawing the read pixels back on the same canvas. The canvas I want to read from has a "virtual resolution", i.e. I am using SetViewport and SetProjection2d to accomplish a smaller resolution than the real device has.

In the code below there is a canvasA in the background, having the same size and resolution as the real device. On top of that, is a centered canvasB, which has the smaller virtual resolution. Inside canvasB, I load and draw a .png image. That works fine.

Next I want to read pixels from canvasB, and I want to read at the exact same area as where the .png image was drawn. And then I want to draw the read pixels somewhere else on canvasB.

But it does not work... What happens is that when I draw the read pixels, they have the same content as the background color of canvasA. And also, the size of the read pixles is a bit off.

Here is the code I have a trouble with (it's based on a fork from code by @ImmutableOctet): https://github.com/monodesire/mojo2_multi_canvas-monkey

The same code here (but obviously without the .png file that is drawn):



Any help and hints are very welcome! :-)


ImmutableOctet(SKNG)(Posted 2016) [#2]
Hey, just as a heads up, I tried messing with this the other day and had some mixed results when testing. Making a dedicated image for the "virtual screen" fixed the problem, though. I also messed with manually calling 'Flush' and changing operation orders. I remember having some success, but for some reason it wouldn't work on the first render. Eventually I just gave up because I've had other projects to work on.

To be honest, the whole using 'ReadPixels' on a shared canvas thing sounds pretty bad in its own right, though. Although, I should bring up that 'ReadPixels' doesn't care about the viewport you're using in Mojo 2, only the underlying texture. So, all coordinates are global, meaning the outer area will be read from no matter what. Fixing this isn't particularly hard, though. You just need to offset by the X and Y positions of the inner screen, and make sure you don't overstep your boundaries.

I should bring up that for static uses, you can basically cut out 'ReadPixels' altogether. What I mean by this is that you can render to a 'Texture' or 'Image' object, then use it. So you could, for example, generate dynamic graphics using a shader, or specific Mojo 2 calls, then use it as a normal graphic. This sort of thing is pretty common for generating sprite atlases, along with calls to 'WritePixels'.

If you wanted the graphic back on the CPU, you could dedicate a pass beforehand to rendering to the screen, grabbing the pixels, then clearing and doing the regular work. Or, assuming it works (I haven't tested it, but it should) you could skip the actual backbuffer altogether and use a dedicated 'Image' or 'Texture' as a framebuffer and read back from that.

There's quite a few ways to go about this, so I hope my suggestions help.


MonoDesire(Posted 2016) [#3]
Thanks a lot ImmutableOctet for all this info! It is not the first time you save me here... ;-) I appreciate it a lot.

You are providing a lot of good info. I need to digest it a bit, and do some elaborating.

You write this: "that 'ReadPixels' doesn't care about the viewport you're using in Mojo 2, only the underlying texture"

Do you here mean that I should use x, y, width and height during ReadPixels that correspond to device canvas (i.e. the "real" screen's size/resolution)? That makes sense, and explains why I got pixels from the device canvas (canvasA in the above code). I need to try this out.

You also write: "render to a 'Texture' or 'Image' object"

This is very good information. I haven't realized this is the way to go. I am probably a bit off trying to use use ReadPixels in the way I am trying to. I need to explore this. Maybe skip the ReadPixels thing altogether and try this first.

Maybe I should explain why I want to WritePixels in the first place: The reason is because when I render the "normal" game area, I use maybe 20-40 images (small ones, like 0.5-2 kB). I do a lot of calls between different class objects to fetch data of various kind, and also some math operations (although simple ones). All in all, some things are going on in each render. Not any heavy stuff, but anyway. Then I want to draw a pop-up dialog window on top of the "nomal" game area. So some parts of the game area will still be visible around the dialog window, but they will be static (since the game is paused as long as the dialog is displayed). So, I was thinking that I take a snapshot/screenshot of the game area and statically show it, and then draw my dialog window on top of that. When the dialog is closed, I go back to normal rendering.

Does this concept makes sense? Is this a good approach? (But instead of doing ReadPixels, I should definitely investigate how to render to a Texture or Image object instead.)


ImmutableOctet(SKNG)(Posted 2016) [#4]
Yeah, you've got the right idea if you don't want to do any heavy lifting while paused. In this case you don't need 'ReadPixels' at all, just another texture to swap in when you need it.

Basically, you would render the scene the final time before the game's paused, but rather than passing around the primary/screen canvas, you would use a 'Canvas' object made with a dedicated target 'Image'. From there you could just render the image before the menu graphics.

Alternatively, you could do a decent bit of the work ahead of time by using a 'DrawList' once, then rendering it before the menu via 'RenderDrawList'. This particular option is a bit more taxing, and requires an appropriate code structure.

The nice thing about the second approach is that you don't use as much graphics memory, but instead use more bandwidth, and a bit of processing power; that's certainly better than resorting to 'ReadPixels' and 'WritePixels'.

I'd only recommend the second option if you're looking to reduce your GPU memory footprint, but that doesn't matter too much if you're not using a lot of textures as it is.


MonoDesire(Posted 2016) [#5]
I am glad to have my idea confirmed.

This sounds exactly like the strategy that I will start working on: "...you would render the scene the final time before the game's paused, but rather than passing around the primary/screen canvas, you would use a 'Canvas' object made with a dedicated target 'Image'. From there you could just render the image before the menu graphics."

Again, big thank you ImmutableOctet. Your help has been invaluable.