ReadPixels() picking up Cls bits?

Monkey Forums/Monkey Programming/ReadPixels() picking up Cls bits?

Nobuyuki(Posted 2013) [#1]
Hi,

Is there any way short of hacking Mojo to get ReadPixels() to not pick up the background pixels left when Cls is called? I did try writing my own Cls() which set glClearColor and an alpha value, but this seems to have little effect where I tested it (on glfw) -- the code works for the purposes of ReadPixels() insomuch that it doesn't pick up the bits cleared by Cls(), but it doesn't read any partial alpha blitting otherwise.

Here's where it gets interesting -- I attempted to hack mojo in order to make Cls() allow specification of an alpha argument, but the same thing happened. The GLFW build was no longer picking up the pixels left behind by Cls(), even when alpha was set to 1. It seems that merely allowing the argument to be specified somehow changes its behavior. Is this a bug?

Anyway, the reason I want this is to be able to create alpha composite surfaces at any time arbitrarily at runtime, and not just on the first render loop before Cls() (as is currently necessary). The current method doesn't allow for multiple "screen-based" systems (like diddy) to do this work correctly -- with a transparent clear, objects which need to redraw themselves can be flagged to do so on the next renderloop, call Cls(0,0,0,0), redraw themselves at the beginning of the loop, then be overwritten by a more traditional Cls(0,0,0,1).

Here's an example of what I mean (on GLFW, v69):


Further question for consideration: Are there any platforms where reading from a surface with a transparent background is simply not possible? Is this the reason for the limitation in the first place?

Edit: I've discovered something else with the second method, as well. Values with alpha that are blitted to the surface with the "transparent" color actually matte to the Cls color. Maybe this has something to do with premultiplied alpha? I don't know much about it.


therevills(Posted 2013) [#2]
If you want to do it after cls you might want to look at supplying a mask:

		ReadPixels(pixels, 0, 0, image.Width(), image.Height())
		' convert the mask colour (black) to alpha
		For Local i:Int=0 Until image.Width() * image.Height()
			Local argb:Int = pixels[i]
			Local a:Int = (argb Shr 24) & $ff
			Local r:Int = (argb Shr 16) & $ff
			Local g:Int = (argb Shr 8) & $ff
			Local b:Int = argb & $ff

			If a = 255 And r = 0 And g = 0 And b = 0 Then
				a = 0
				argb = (a Shl 24) | (r Shl 16) | (g Shl 8) | b
				pixels[i] = argb
			End
		Next
		
		image.WritePixels(pixels, 0, 0, image.Width(), image.Height())



Skn3(Posted 2013) [#3]
I was wondering this too earlier.

Is this a limitation that could be rectified easily enough?


Nobuyuki(Posted 2013) [#4]
therevills: this only provides 1-bit alpha, doesn't it? That's probably only useful for a collision mask or retro graphics... There's also ways to "knock out" a channel with 8-bit alpha by specifying a color threshold, but we'd still be trying to work around not being able to actually have compositing with 8-bit alpha transparency on new image surfaces. What I'm hoping to figure out is why we can't/don't have a way to explicitly "clear" the main surface with transparency, since apparently it's the only place we're allowed to read pixels from.


Skn3(Posted 2013) [#5]
edit, I see what you mean about the alpha.

It is blending with backbuffer color. If you change your cls2 to clear 255,255,255 before building your image, you will notice that your alpha is correct. But only for the white portions.

perhaps glBlendFuncSeparate+glBlendEquationSeparate could be used to apply different blending equations for rgb and alpha?

Not sure if its supported though?


Gerry Quinn(Posted 2013) [#6]
Do all render surfaces even support transparency? I could see it being used for web page tricks, but on a mobile device what would be the point?


Nobuyuki(Posted 2013) [#7]
skn3: I have determined that the most neutral color for blending with the backbuffer color is either 127 gray, or 128 gray. That seems to provide a (mostly) neutral blend, with only a small amount of desaturation around the fringes on most backgrounds instead of a straight-up halo. Still, without a true transparent color behind the pixels we want to pick up, I'm having trouble coming up with a mostly-managed workaround that can "restore" the alpha channel short of providing a separate 8-bit mask with every asset :\

Gerry: The purpose is for runtime pixel manipulation. I personally have many uses for it, one of which is very important for my proposed AngelFont replacement -- alpha compositing full color images onto a separate surface (ie: pre-rendering) before blitting increases the speed of displaying large text blocks immensely. Images that are mostly-static, like text, don't need their textures to be invalidated often, and are well-suited to being pre-calculated. The same applies to any sort of procedurally-generated texture which is applied at runtime, such as AutoTiles. It's even useful for image obfuscation purposes (ie: re-assembling textures from "garbled" image files at runtime). Most of these features are heavily limited because Monkey's inability to read pixels directly from an arbitrary surface or to clear the backbuffer with "true" transparency makes it impossible to retrieve the original alpha data from the components that are supposed to make up the final composite texture.


Gerry Quinn(Posted 2013) [#8]
My point was not about whether it would be useful, but whether it is even supported in hardware!

Say you have an Android tablet, what would be the use in the back-buffer supporting transparency? There's nothing behind to show through!


Nobuyuki(Posted 2013) [#9]
Gerry: The main purpose in my mind is a workaround; to be able to use ReadPixels to grab the main surface's pixels with alpha preserved, using it as a temporary scratchboard, and then apply the result to a new surface for use in-game. What we really need is reading pixels from an arbitrary surface; we don't have that, though, because that's not even close to being supported on certain targets. Instead, we're only able to read from the main surface. This would be just fine, if it weren't for the fact that calling Cls() clears the main surface with opaque pixels. I've demonstrated that it doesn't have to do this, though, and is available and mostly-consistent on two very important targets -- HTML5 (the original and "most limited" target of Monkey), and GLFW (the quintessential OpenGL target, whose capabilities are very similar to all of the other OpenGL targets).


Skn3(Posted 2013) [#10]
Just a quick thing to look at:

mojo.glfw.cpp
bool gxtkSurface::OnUnsafeLoadComplete(){

It looks like the image is being pre-multiplied. Dunno if that will help you?


Nobuyuki(Posted 2013) [#11]
if I knew more about pre-multiplication, I could say with better certainty. It may be easier to hack mojo on the openGL targets to get an array from a pixel buffer object or something on an arbitrary texture or image in memory, and feed it back to monkey in a format that WritePixels understands. It will require some investigating, and is certainly a less than optimal solution (unless Mark accepts it into mainline, or does it himself!)


Skn3(Posted 2013) [#12]
Does this help?
http://monkeycoder.co.nz/Community/posts.php?topic=5285


Nobuyuki(Posted 2013) [#13]
skn3: That looks interesting! Even if it isn't a clean "untainted monkey" solution, it's definitely a step in the right direction. I always love seeing OpenGL hacks that add draw functionality; I just wish more of them would start being incorporated into mainline!


chrisc2977(Posted 2013) [#14]
I Hope this is the right place to post it, I have taken the idea of the above '1 bit mask code' and made it smoother.

The function below returns an 'argb' array.
(essentially copying an image into an array and retaining a smooth alpha)
(increase the margin local if a bit of alpha appears where its not wanted)







Left is Original Image, Right is image created from the above function and writepixel.
The lovely pink BG is my cls color :)
As you can see its quite smooth, not perfect, but certainly usable :)
(The image is NOT scaled, so you can see even the edges of the small font are somewhat preserved)


Nobuyuki(Posted 2013) [#15]
Took it a step further. This will grab and convert the image data directly from the texture:

http://www.monkeycoder.co.nz/Community/posts.php?topic=6087