gah! How to clip rotated rectangles.

BlitzMax Forums/BlitzMax Programming/gah! How to clip rotated rectangles.

Grey Alien(Posted 2007) [#1]
It's probably not possible, but I'll post anyway incase any of you clever boffins have any ideas.

Basically I'm NOT using the BMax Viewport command because it fails on some videocards (as in, it does nothing) - I've seen it with my own eyes.

I'm using this very useful code by Ian Duff:

' -----------------------------------------------------------------------------
' ccDrawImageArea: Draws part of an image.
' (by Ian Duff)
' -----------------------------------------------------------------------------
Function ccDrawImageArea(image:TImage, x#, y#, rx#, ry#, rw#, rh#, theframe=0)
 'Note that this code works fine in DirectX or OpenGL on PCs - it autodetects (Grey Alien).
  Local origin_x#, origin_y# ; GetOrigin (origin_x, origin_y)
  Local tw = ccDrawImageAreaPow2Size(image.width)
  Local th = ccDrawImageAreaPow2Size(image.height)
  Local rw1#  = rx + rw
  Local rh1#  = ry + rh
  Local x0# = -image.handle_x, x1# = x0 + rw
  Local y0# = -image.handle_y, y1# = y0 + rh
  
  If rw1 > image.width
    x1 = x0 + rw + image.width - rw1
    rw1 = image.width
  EndIf
   
  If rh1 > image.height
    y1 = y0 + rh + image.height - rh1
    rh1 = image.height
  EndIf
?Win32
  If TD3D7ImageFrame(image.frame(theframe))
    Local frame:TD3D7ImageFrame = TD3D7ImageFrame(image.frame(theframe))
    
    frame.setUV(rx / tw, ry / th, rw1 / tw, rh1 / th)
    frame.Draw x0, y0, x1, y1, x + origin_x, y + origin_y
    frame.setUV(0, 0, image.width / Float(tw), image.height / Float(th))
  Else
?
    Local frameA:TGLImageFrame = TGLImageFrame (image.frame(theframe))
                
    frameA.u0 = rx / tw
    frameA.v0 = ry / th
    frameA.u1 = rw1 / tw
    frameA.v1 = rh1 / th
    
    frameA.Draw x0, y0, x1, y1, x + origin_x, y + origin_y
    
    frameA.u0 = 0
    frameA.v0 = 0
    frameA.u1 = image.width / Float(tw)
    frameA.v1 = image.height / Float(th)
?Win32
  EndIf
?
  
  Function ccDrawImageAreaPow2Size%(n)
    Local ry = 1
    
    While ry < n
      ry :* 2
    Wend
    
    Return ry
  End Function
End Function


This works great and I can pass in parameters to basically draw a clipped rectangle to my view port, like this:

' -----------------------------------------------------------------------------
' ccClipImageToViewport: Clips an image into a "safe" viewport (doesn't use BMax viewport)
' -----------------------------------------------------------------------------
Function ccClipImageToViewport(image:TImage, imagex#, imagey#, ViewportX#, ViewPortY#, ViewPortW#, ViewPortH#, offsetx=0, offsety=0)
	'Perform basic clipping first by checking to see if the image is completely
	'outside of the viewport.
	'Note that images are drawn from the top left, not midhandled or anything else.
	Local w = ImageWidth(image)
	Local h = ImageHeight(image)
	
	If imagex+w>=ViewportX And imagex-w<ViewportX+ViewportW and..
		imagey+h>=ViewportY And imagey-h<ViewportY+ViewportH Then
		'Clip left and top
		Local startx#=ViewportX-imagex
		Local starty#=ViewportY-imagey
		If startx<0 Then startx=0 'clamp normal values
		If starty<0 Then starty=0 'clamp normal values
		'Clip right and bottom
		Local endx#=(imagex+w)-(ViewportX+ViewportW)
		Local endy#=(imagey+h)-(ViewportY+ViewportH)
		If endx<0 Then endx=0 'clamp normal values
		If endy<0 Then endy=0 'clamp normal values
		ccDrawImageArea(Image, imageX+startX+offsetx, imagey+starty+offsety, startx, starty, w-startx-endx, h-starty-endy)
	EndIf
End Function

Probably the above function can be optimised, please let me know if you can think of a smarter way to do it.

Anyway ... If I use SetRotation and call the above code, the image is drawn rotated fine but of course it's clipped BEFORE the rotation and thus weird stuff happens to it like bits appear missing inside the viewport and other bits poke out the viewport :-(

I want to rotate it and then draw it clipped AFTER the rotation so that the clipping rectangle is at Rotation 0, not the same angle as the rotated image. Make sense? If not I'll post an image.

Oh and it has to be fast enough to use at runtime, not precalculated (e.g. using pixmaps etc).

Anyone got any ideas how to resolve this as I'm stuck because my DX/GL knowledge is minimal. Many thanks in advance.


Dreamora(Posted 2007) [#2]
The only way to get this done in realtime is using Render2Texture and to draw onto a different surface than your background.
Probleme here is that this restricts the viewport to power of 2 sizes

there are not many other usefull possibilities for graphic cards that are that bad that they do not even support DX6 level stuff like scissortests and its DX pendant which in most cases means: pure 2D cards that will fallback to software emulation and better would use OpenGL than DX7

The only other would defeat its use itself in several cases: Draw black areas after the "to be clipped" stuff has been drawn to just hide the parts you didn't want to draw ie the parts outside the clip area.


tonyg(Posted 2007) [#3]
Might need that image.Could you achieve what you're after another way? Could you use Textured Poly?


Grey Alien(Posted 2007) [#4]
I've never use Textured Poly is that another Indiepath mod?


Bremer(Posted 2007) [#5]
It should be possible using glScissor:

http://www.opengl.org/documentation/specs/man_pages/hardcopy/GL/html/gl/scissor.html

I do not know enough about DX to know what works in a similar way.


Dreamora(Posted 2007) [#6]
thats exactly what BM Viewport does under gl driver. problem is how it is done in DX


kfprimm(Posted 2007) [#7]
IDirect3DDevice9::SetScissorRect


Dreamora(Posted 2007) [#8]
BM is DX7 not DX9


tonyg(Posted 2007) [#9]
Indiepath does have one but there are a few more listed in the forums and this in the Code Archives.


MGE(Posted 2007) [#10]
If you're redrawing the background every frame anyway, perhaps just slice the redraw into chuncks and redraw over the areas that need to be clipped. And even if you have to draw the entire scene, draw your rotated images, and then redraw the clipped areas, I can't see that being a speed killer??


Grey Alien(Posted 2007) [#11]
Thanks all, not sure I can put those to use effectively.

JGOware: Thanks but regretably it's not that simple.


Bremer(Posted 2007) [#12]
Does it need to be realtime calculated, or can it be slower than that?


Grey Alien(Posted 2007) [#13]
At first I thought I'd like realtime, but then I realised I need to pre-rotate an image (with alpha channel), and then draw it in a small manually managed viewport that I do my own clipping in by drawing only a portion of the final image. Why can you help?


Bremer(Posted 2007) [#14]
I am fairly certain that I have some code from back when I made graphics demos that rotate an image with clipping and writing directly to the image buffer. If I do, it might be something that I could convert to bmax and pixmaps.

I will have to dig out my old backups and take a look. I will let you know if I find something useful.


Grey Alien(Posted 2007) [#15]
Excellent that's very kind thanks.


Bremer(Posted 2007) [#16]
I found some rotation code which I will try and convert to bmax and pixmap. Not sure if I have time tomorrow, but I will try and get it done as soon as I can.

Edit: In BlitzPlus this code is rotating a 256x256 image at 61 fps on my computer with a small array used for some precalculated sin/cos values, and at 13 fps using sin/cos directly. I will probably make the converted code for both senarios so that the options are there to choose from.

In my old code the image is first read into an array buffer, which speeds up the code as well. I think that it would be best to do the same in a bmax version because it does speed up the process if doing a lot of rotations.

But we will have to wait and see what I can come up with, once I get some time available, hopefully soon.


Grey Alien(Posted 2007) [#17]
Yes, I used to read into an array first too when doing such things in BPlus. Thing is it needs to retain alpha info...is that possible?


TartanTangerine (was Indiepath)(Posted 2007) [#18]
The simple solution would be to calculate the rotated UV co-ordinates and apply them? no?


Dreamora(Posted 2007) [#19]
That would be the simplest and fastest indeed :)


Grey Alien(Posted 2007) [#20]
AHA, now we are getting somewhere. So, uh how do I do that?

Here's a picture just to ensure we are singing from the same hymnsheet.



The code I pasted at the top uses UV coords but doesn't do what I want at the moment. If I use SetRotation 45 for example before I call the code, then it applies rotated clipping to the rotated image. I want rectangular clipping applied to the rotated image. Make sense?


Dreamora(Posted 2007) [#21]
you have to rotate the UV coords on the quad and set the quads rotation to 0.
This means you must create a ccDrawRotateClippedImage function or the like which ensures correct rotation states etc


Grey Alien(Posted 2007) [#22]
So in this line:

frame.setUV(rx / tw, ry / th, rw1 / tw, rh1 / th)

I simply rotate those 4 coords, cool! I was assuming that the UV coords were always drawing a upright rectangular area.

[edit]hmm, just tested and if I alter the UV coords all I get is different scaled or clipped upright rectangles. Nothing draws rotated. Surely I'd need 4 coord pairs to properly map a rotated texture anyway?


Dreamora(Posted 2007) [#23]
The 4 UV coords define the 4 corner points of your quad on the texture, they can have any shape.

And yes you would need 4 pairs.
This means you would need to use a textured poly function or take the polygon function from BM and make sure it uses the texture (it drops the texture by default so you can in simplest case just copy that and make sure the texture isn't dropped)

The regular quad class of BM is for rectangular axis aligned sprites


Grey Alien(Posted 2007) [#24]
hmm, thanks, I'm still nt 100% clear though as this is above my knowledge of textures etc.

For example, you talk about 4 UV coords but is that coordinate pairs (i.e. a pair = x AND y)? I ask because the above code example sets the UV to an x,y and a width and height. That's only two coordinate pairs so it can only define a non-rotated rectangle, it can't defined a rotated rectangle or a rhomboid.

There's no help file for TQuad so I haven't got a clue where it is or what it does. AH, OK found it in max2d.bmx but it's totally uncommented (like most Blitz internal code) so I have no idea what it's doing :-(


tonyg(Posted 2007) [#25]
The rotation is added (multiplied I think) to the xyzuv array which is used when drawing images.
However, I still don't know exactly what you want to do.
From the picture it seems you want to draw a viewport but, because you don't trust Bmax's viewport command, you want to clip the images yourself.
Is that right?
If so, why not 'frame' your images in a fullscreen image with the viewport defined as a masked rectangle? If you want a changeable viewport then draw 4 rectangles on each side of your game screen over the top of your images?
As I don't know what it is you really want to do I don't really want to follow and work on any suggested solution.


Space Fractal(Posted 2007) [#26]
Wrong thread. sorry.


Grey Alien(Posted 2007) [#27]
From the picture it seems you want to draw a viewport but, because you don't trust Bmax's viewport command, you want to clip the images yourself.
Is that right?
Yes that is correct.

If so, why not 'frame' your images in a fullscreen image with the viewport defined as a masked rectangle? If you want a changeable viewport then draw 4 rectangles on each side of your game screen over the top of your images?
The full screen image is changing and to do a full-screen grab from VRAM takes about 300ms on my PC, so alas this won't work.

So I basically want to:

1) get an image
2) rotate the image
3) draw it rotated except have it clipped to a viewport that I define.

That's it, as per the diagram.

It shouldn't be this hard ;-)


tonyg(Posted 2007) [#28]
It shouldn't be this hard ;-)

Maybe it isn't.
Obviously the 'easy' answer would be the Viewport commands but if you have reason not to trust them you won't be using the 'easiest' method.
Anyway, I am, not suggesting doing a full-screen grab but, instead, have a screen-size image with a transparent centre.
Draw this image after all other drawing.
Alternatively, if the size of the border is to change then drawrect the border.



Grey Alien(Posted 2007) [#29]
Thanks Tony. Basically imagine this:

1) I have a normal game screen with sprites, scores etc on it. These are all drawn separately onto a background graphic.
2) I am showing a small "minigame" window on the game screen created in point 1.
3) items in the minigame window must not be drawn outside of the minigame window.

Therefore I can't draw a screen-sized image with a transparent centre as the screen-sized image would be different everytime the mini-game is played due to the arangement of sprites/scores etc on the main-game screen.

Also plain borders as rects won't work because there needs to be main game graphics visible there.

So I go back to the code in the top post. That works fine for NON-rotated images as it can draw them clipped but I want to rotate them and then draw them clipped. This is the not-easy part it seems...


Dreamora(Posted 2007) [#30]
That is what I already suggested above but that creates massive problems when 2 rotate with viewport should be drawed.

Grey:
Yes it means 4 pairs. UV always referres to 2D point coordinates.
You can not use TQuad, you must operate on the vertex buffers yourself I fear. As you've realized yourself, you can only give TQuad 2 coordinates, as it is meant for axis aligned rects, not rotate rects which need 4 (ok correctly 3 as it has a degree 3 freedom, x, y, angle) coordinates.

In DX this means the above mentioned xyzuv array


As I mentioned above, BM already has an internal polygon function which allows to draw a polygon. I would suggest to have a look at this one and create your function basing on that.
That or use one of the texture poly codes here on the boards.

Past that point the only thing you need to do is doing the math to transform the UV coordinates correctly. Because normally you calculate the new position of the vertices in the calculation. But what you do is rotate the image and calculate the new coordinates of the vertices within this image ie you calculate the inverse rotation.
So far no problem. But in a lot of cases the coordinates will not be within the original texture. At that point you must clamp the texture to prevent auto repeat of the texture. (normally when you would have a UV coord of 1.1 you would see the regular texture and part of a repeated one)
This is doable and not to hard, but needs lower level coding than using Max2D commads and therefor might need adjustment in the future.

Sadly the Scissor Test like the one GL does was not added to DX until DX9 ...
For that reason there is no simple solution on DX for this issue unless you want to include the user created DX9 driver as default into your framework ...


Grey Alien(Posted 2007) [#31]
Dreamora: thanks for the detail. I understand some of what you say, but some is above my level, oh well. I understand about repeating of the texture and the need to clamp it, but as you say the code is probably bitchin.

I've certainly seem the BMax viewport not work properly on some video cards (it just didn't apply it at all), it was probably the DX viewport. If BRL fixed it (if that was even possible), that would indeed solve my problems, but I don't know enough to fix it myself.

Sadly I may have to give up on this now. I have another thread running ( http://www.blitzbasic.com/Community/posts.php?topic=73042 ) about pre-rotating a pix-map and drawing that clipped. That would work for me.


Dreamora(Posted 2007) [#32]
The problem can not be solved with DX7. Its a pure hardware restriction. the systems that have problems do not nearly meet the BM minimum requirement for Max2D. They fallback to pure emulation most likely or pretend to DX that they are capable and just fail to really work.
DX7 has no scissor test (the thing that cuts drawing on GL drivers). Using viewports creates problems far larger (thats what I've tried, using real viewports).
What BRL did was using clipplanes like in 3D to restrict the drawing. and the systems that fail on the viewport end actually do so because of their graphic cards that don't support 4 clip planes but only 2 (most likely meant for near and far but no side).

If you only need to restrict the viewport to right and down you could do that and have it working on those systems as you would only need 2 clip planes. but with x,y,width,height in the viewport it wouldn't work I fear


Bremer(Posted 2007) [#33]
I have not found time to work on a bmax conversion, but here is some blitzplus code which does rotate a 256x256 image. I will still try and find time to convert it, but unfortunately time to code is not something I have plenty of these days.




Grey Alien(Posted 2007) [#34]
Dreamora: Yeah I see what you mean about the two clip planes. For most of my sprites, 2 would be all that was needed e.g. left and bottom OR top and right. As you say the only problem would be if the sprite was BIGGER than the viewport and needed to be clipped top and bottom and maybe left too...

zawran: thanks for that. Having a quick peek it doesn't appear to support an alpha channel (as BPlus didn't anyway) but no doubt that could be worked out in BMax.


Bremer(Posted 2007) [#35]
Readpixel and writepixel in bmax uses RBGA, 32 bit so it supports alpha.


Grey Alien(Posted 2007) [#36]
groovy. It seems ImaginaryHuman (aka AngelDaniel) has come up with the goods in my other thread on this topic. I still need to test it though.


Bremer(Posted 2007) [#37]
I decided to take some time out for coding anyways, and this is my code converted to Bmax and where it checks the source image for out of bounds, so that the image doesn't repeat itself on the destination image like it would in a rotozoomer.

This example is hardcoded for a 256x256 image, like the BlitzPlus code was. I don't have time to make it more generic, but with the addition of some local variables that should be doable.




Grey Alien(Posted 2007) [#38]
seems pretty neat I'll have to compare this to the other code.


Bremer(Posted 2007) [#39]
I could not get the other code to work, I just get a blank square, but it might be, that I do not know how it works.

It seemed like it was just as quick using sin/cos directly compaired to the pre-calc array, but I didn't get around to do any timed tests, it just appeared to be as fast, so you might not have to use the array.


Grey Alien(Posted 2007) [#40]
I did some precalced sin/cos ages ago and it really isn't faster on a modern PC.


Bremer(Posted 2007) [#41]
Here is the code without hardcoded values. The rotate function really only works with square images with an equal number of pixels.