Pixel-perfect screen scaling - big problem

BlitzMax Forums/BlitzMax Programming/Pixel-perfect screen scaling - big problem

Adam Novagen

(Posted 2015) [#1]

Summary question at the bottom, the rest of the post is mostly context and my results so far.

Alright, so I'm about to pull my hair out on this one. Current project is a retro-styled game. The game runs at a 400x224 base resolution; this resolution is non-negotiable. In a window, I wish to be able to scale this up as desired, but primarily to x2, x3 and x4 resolutions; 800x448, 1200x672 and 1600x898, respectively.

Until now, I've been doing this satisfactorily using SetVirtualResolution and SetScale... Or so I thought. However, today I noticed there are multiple errors in the scaling algorithms. Here is a sample screenshot of the game at its native 400x224 resolution, but scaled up to 2x in GIMP:

Placeholder graphics, bear with me

Simple enough, yes? Now here is that exact same scene, but rendered in an actual 800x448 window with a virtual resolution:

Just in case you're having any trouble seeing the horrific pixel distortion, here are some of the most obvious problems highlighted:

This was taken from a computer with a GTX 770. Out of curiousity, I tried this exact same program on another computer, one with Intel integrated graphics supplied by a Core 2 Duo E7500, a machine which I use as my low-end benchmark to check performance and optimization. The results were nowhere near as bad, but there were still plenty of imperfections, with some of the most notable highlighted here:

As near as I can figure, this is a hardware-dependent issue with... Scaling? Floating-point errors? I don't even know, but the bottom line seems pretty clear: to get this game to scale up properly and consistently, I need to be able to just render the game at 400x224, then scale up the entire screen at once. Thing is, I tried both GrabImage and GrabPixmap, and it will probably not surprise any of the veteran coders here to learn that both were far too slow for realtime.

So... What do I do here? I obviously need to draw everything at 1:1 and then scale the screen up, but I have no idea how to achieve this short of the aforementioned attempts. Back in Blitz3D, for example, this literally would have been as easy as using SetBuffer TextureBuffer(), then carrying on all the drawing operations as normal; BlitzMax being a completely different beast, I seem to be in way over my head.

Summary: what is the simplest/best way to take a 400x224 area of the backbuffer and scale it up to match the window resolution, in 60Hz realtime?

FBEpyon

(Posted 2015) [#2]

I use SetVirtualResolution for what you are trying to do in Max, but I also include a viewport and set origin..

So like:


Graphics 800,448
SetVirtualResolution(400,224)
SetOrigin(0,0)
SetViewport(0,0,400,224)

This seems to work for me, and you don't need to do any SetScale as this should be already taken care now..

Adam Novagen

(Posted 2015) [#3]

I already use SetVirtualResolution, as I said in my post. SetOrigin and SetViewport too. This is precisely what is causing the problem, as the scaling from SetVirtualResolution is responsible for the pixel distortion (which appears to be caused by slight texture offsets). This is why I need a way to render 1:1 and then scale the 400x224 area up to match the window.

EDIT: It's probably worth mentioning that the screenshots shown in the OP are tile-based maps made with 16x16px tiles. The pixel offset issues are always at the edge of the tiles; everything inside each tile scales fine.

FBEpyon

(Posted 2015) [#4]

I suggest you look at this if you are trying to do something with the 16:9 resolution..

9/16 = 0.5625

your resolution equals 0.56 which is not true 16:9

https://pacoup.com/2011/06/12/list-of-true-169-resolutions/

Anything divisible by 8 resolution will work with what you are trying to do, but if its not divisible by 8 then it will scale weird unless you work out the viewport and origin your self.

For classic NES style resolutions I usually go with a 256x224, so I setup the virtual res to 256,224 and then I setup the normal to 800,600 with a viewport of 32,8,256,224 and a origin of 32,8

This works most of the time for me...

I have also noticed that if could be the GraphicsDriver you are using.. try changing it to DirectX9 instead of OpenGL if you are having problems with Nvidia graphics card.


'SetGraphicsDriver D3D7Max2DDriver()
SetGraphicsDriver D3D9Max2DDriver()
'SetGraphicsDriver GLMax2DDriver()

Graphics 800,448
SetVirtualResolution 400,224
SetOrigin 0,0
SetViewport 0,0,400,224

Adam Novagen

(Posted 2015) [#5]

With a resolution height of 224 it is impossible to achieve true 16:9. 398x224 is 0.56281407035175879396984924623116 and 399x224 is 0.56140350877192982456140350877193. However, 400 and 224 are both evenly divisible by 8, so that is clearly not all that's going on here. This entire problem can be circumvented with what I have been trying to find since my first post: a solution to allow me to draw everything to a single image/texture/pixmap, which I can then draw to the backbuffer and scale up uniformly.

FBEpyon

(Posted 2015) [#6]

Global DWIDTH:Int = 256
Global DHEIGHT:Int = 224

Global RWIDTH:Int = 512
Global RHRIGHT:Int = 448

SetGraphicsDriver GLMax2DDriver()

Graphics RWIDTH,RHRIGHT
'SetVirtualResolution DWIDTH,DHEIGHT
'SetOrigin 0,0
'SetViewport 0,0,400,224

Global timer:TTimer = CreateTimer(60)

Global player:TPlayer = TPlayer.Create(DWIDTH/2,DHEIGHT/2)

While Not KeyHit(KEY_ESCAPE) And Not JoyDown(6)

	player.Contorls()
	player.draw_timer()

	Cls
	
	'place all pre-draw here
	
	player.draw()

	Local backbuffer:TImage = CreateImage(400,224,,DYNAMICIMAGE)
	GrabImage(backbuffer,0,0)

	Cls
	
	SetClsColor 0,0,0
	
	SetScale 2,2
		
	DrawImage backbuffer,0,0	
		
	SetScale 1,1	
		
	WaitTimer(timer)
	
	Flip
	
Wend
End

Try this works fine on low ends.. and its simple enough to fix your problem I would think. Just follow what I'm doing in the main loop, also you don't need virtual resolution for this..

Floyd

(Posted 2015) [#7]

With a resolution height of 224 it is impossible to achieve true 16:9

That's because 224 is not a multiple of 9.

225 would work as a width because it is divisible by 9. It is 25*9 so a width of 25*16 = 400 would give 16:9.

I can't see why the scaling up doesn't work exactly no matter what the ratio. A quick look at the source code for Max2D was unenlightening, too much other stuff I don't know.

Kryzon

(Posted 2015) [#8]

Hey Adam, give the following a try on the problem computer:

SuperStrict

Import brl.max2d
Import brl.timer

Const ORIGINAL_WIDTH:Int	= 400 'Taken from the imported image.
Const ORIGINAL_HEIGHT:Int	= 224 'It's the true size in pixels of the game screen.

Const SCALE_FACTOR:Float = 2.0

Const GAME_SCREEN_WIDTH:Int		= ORIGINAL_WIDTH * SCALE_FACTOR 'Actual size of the game window.
Const GAME_SCREEN_HEIGHT:Int	= ORIGINAL_HEIGHT * SCALE_FACTOR


SetGraphicsDriver D3D9Max2DDriver()	'Direct3D 9.

Global currentGraphics:TGraphics = Graphics( GAME_SCREEN_WIDTH, GAME_SCREEN_HEIGHT, 0, 30 )
Global currentDriver:TGraphicsDriver = GetGraphicsDriver() 
Global d3dDev:IDirect3DDevice9
Global d3dImageSurface:IDirect3DSurface9
Global d3dBackBufferSurface:IDirect3DSurface9
Global pow2Width:Int, pow2Height:Int

Local timer:TTimer = CreateTimer( 30 )

'After the graphics object has been created and set.

If currentDriver = D3D9Max2DDriver()

	'We're using the Direct3D 9 renderer.	
	'Obtain a D3D device.
	
	d3dDev = TD3D9Graphics( TMax2dGraphics( currentGraphics )._graphics ).GetDirect3DDevice()
	
	'Create an image with a "render-target" internal texture.
		
	Local renderTarget:TImage = createD3DRenderTarget( pow2Width, pow2Height )

	'Start the game, load resources etc.

	Local myImage:TImage = LoadImage( "indieGame.png", 0 ) 'Non-filtered image.
	
	While Not ( KeyHit( KEY_ESCAPE ) Or AppTerminate() )
	
		WaitTimer( timer )
		
		'Drawing to the render target.

		beginD3DRenderTarget()
		TMax2dDriver( currentDriver ).SetResolution( pow2Width, pow2Height ) 'Total size of the render-target texture. This messes with projection.
		
			'Draw all the graphics of your game.
			
			SetScale( 1.0, 1.0 )
			SetClsColor( 255, 120, 50)
			Cls
			
			DrawImage( myImage, 0, 0 )
			
			SetClsColor( 0, 0, 0 )
		
		endD3DRenderTarget() 
		
		'Now drawing to the backbuffer.
		
		TMax2dDriver( currentDriver ).SetResolution( GraphicsWidth(), GraphicsHeight() ) 'Restore the internal resolution.
		
		Cls 'If the backbuffer will always be filled, this CLS call may be unnecessary.
		
		'Set a scale to fit the image in the whole window.
				
		SetScale( SCALE_FACTOR, SCALE_FACTOR )
		
		'Draw the render target to the backbuffer, for the user to see.
		
		DrawImage( renderTarget, 0, 0 )
		
		'Flip comes at the END, after the backbuffer has been drawn to.
		
		Flip
		
	Wend
	
ElseIf currentDriver = GLMax2DDriver()

	'OpenGL renderer.
	'Not yet written.
	
	RuntimeError "OpenGL version not yet implemented."
	
EndIf


Function createD3DRenderTarget:TImage( pow2w:Int Var, pow2h:Int Var )

	Local image:TImage = CreateImage( ORIGINAL_WIDTH, ORIGINAL_HEIGHT, 1, 0 ) '1-frame image.
	image.seqs[ 0 ]=GraphicsSeq
	
	'Set the first frame to be a manipulated D3D image frame.
	
	Local d3dFrame:TD3D9ImageFrame = New TD3D9ImageFrame
	
	pow2w = Pow2Size( ORIGINAL_WIDTH )
	pow2h = Pow2Size( ORIGINAL_HEIGHT )
	
	If d3dDev.CreateTexture( pow2width, pow2height, 1, D3DUSAGE_RENDERTARGET, D3DFMT_A8R8G8B8, D3DPOOL_DEFAULT, d3dFrame._texture, Null ) < 0
		RuntimeError "Unable to create a D3D 9 render target.~n"
		Return Null
	EndIf
		
	'Taken from 'd3d9max2d.bmx' from the 'D3D9Max2D' brl module.
	
	TD3D9Graphics( TMax2dGraphics( currentGraphics )._graphics ).AutoRelease d3dFrame._texture
		
	d3dFrame._magfilter=D3DTFG_POINT 'Optionally, D3DTFG_LINEAR on all three, for a filtered render-target.
	d3dFrame._minfilter=D3DTFG_POINT
	d3dFrame._mipfilter=D3DTFG_POINT	
	
	d3dFrame._uscale=1.0 / pow2width
	d3dFrame._vscale=1.0 / pow2height
	
	Local u0#,u1#=ORIGINAL_WIDTH * d3dFrame._uscale
	Local v0#,v1#=ORIGINAL_HEIGHT * d3dFrame._vscale

	d3dFrame._fverts[4]=u0
	d3dFrame._fverts[5]=v0
	d3dFrame._fverts[10]=u1
	d3dFrame._fverts[11]=v0
	d3dFrame._fverts[16]=u1
	d3dFrame._fverts[17]=v1
	d3dFrame._fverts[22]=u0
	d3dFrame._fverts[23]=v1
		
	d3dFrame._seq = GraphicsSeq ''GraphicsSeq' is a global defined in 'brl.mod\graphics.mod\graphics.bmx.'
	image.frames[ 0 ] = d3dFrame
	
	'Get the render-target image surface.
	
	If d3dFrame._texture.GetSurfaceLevel( 0, d3dImageSurface ) < 0
		RuntimeError "Unable to obtain a D3D 9 render target.~n"
		Return Null
	EndIf
	
	'Get the backbuffer surface so we can return to it later.
	
	d3dDev.GetRenderTarget( 0, d3dBackBufferSurface )
	
	Return image

End Function


Function beginD3DRenderTarget()
	
	d3dDev.SetRenderTarget( 0, d3dImageSurface ) 'Set the image as the render-target.
	
End Function


Function endD3DRenderTarget()

	d3dDev.SetRenderTarget( 0, d3dBackBufferSurface ) 'Restore the backbuffer rendering.

End Function


Function Pow2Size:Int( n:Int )
	Local t:Int=1
	While t<n
		t:*2
	Wend
	Return t
End Function


End

Adam Novagen

(Posted 2015) [#9]

Thanks Kryzon. Fortunately, as it happens I managed to land on an old render-to-texture code (http://www.blitzmax.com/codearcs/codearcs.php?code=2222) which has - with a little coordinate tweaking - provided me with the exact solution I need. The only disadvantage is that it requires OpenGL 2.1 or above to work, otherwise it throws Windows EXCEPTION_ACCESS_VIOLATION. That's usually solvable with a simple driver update, so I'm now just in the process of writing an error prompt to catch systems that are still running pre-2.1 OGL.

Kryzon

(Posted 2015) [#10]

The thing is, on a Windows system without a GPU that supports OpenGL 2.1, you can still have render-target functionality if you use Direct3D 9 (supporting Windows XP and above).

Adam Novagen

(Posted 2015) [#11]

Hm. Tempting. I'm not particularly worried about sub-2.1 systems since anyone that's limping along like that probably won't be gaming much anyway, but I'll investigate this code tomorrow (completely burnt right now). Thank you!

skidracer

(Posted 2015) [#12]

I'm not going to try and guess the problem / cause of above but the way I see it is you have two options.

1. clean pixel scaling with 1:n scaling where n is integer

2. subpixel blending where your platform game glides along on a a subpixel position in screen space and alpha borders around all sprites ensure zero artefact edges

for (1) you need to throw out any virtualisation of screen space for 1 as the final screen location on the users display needs to be correct as in integer coordinate. LCD screens mean you should only ever run games in native resolution of device to avoid any dependency on thirdy party wonkery in the display chain.

for (2) all source needs to either be clamped to border with border color set to alpha 0 or be laid out on a sprite sheet with 2^n alpha 0 gutters where n is the likely mip map levels accessed when the driver goes to render your sprites at fluid subpixel steps

Adam Novagen

(Posted 2015) [#13]

Been quiet for a little while, combination of being busy when I'm not lazy and lazy when I'm not busy. Kryzon, I just wanted to let you know that I have finally ported in your render target code, and it works like a charm. This gives me an broad, ideal range what with DX9 being standard on every Windows system running Vista or up, and common on almost every XP system as well. I also found your code very modular and easy to use, and decently commented too; I really appreciate the effort. Thank you!

GW	(Posted 2015) [#14]

Ya. Thanks Kryzon!
I've been digging around for a DX9 RTT example. You should put that in the code archives.

Kryzon

(Posted 2015) [#15]

You're welcome.
Please feel free to put it there in the archives -- I'd just move the obscure SetResolution calls to inside their respective 'begin' and 'end' D3DRenderTarget functions.

GW	(Posted 2015) [#16]

I put it in the Archives. I wrapped it in a type with a matching api of the OpenGl RTT by Klepto.
Thanks again Kryzon, I was actually digging around for this when you posted it. perfect timing.