Easier GUI Mouseclick/icon checks
Blitz3D Forums/Blitz3D Beginners Area/Easier GUI Mouseclick/icon checks
| ||
| Just posting here to hear everyone's thoughts/suggestions... One part of coding games or anything with a language such as B3D (which lacks any native GUI capability, save for mouseX, MouseY and the various GetKey/GetMouse functions,) is that I hate.. actually detest, having to code in a GUI. Having to either create some kinda data structure to handle menus, or record/pre-define the positions/functions of various icons etc. had a sudden thought whilst being otherwise occupied (where all the best ideas come!) that maybe a kinda hifdden interfface could be used, and this hidden interface is what the mouse clicks are checked on. How it may work, is that the hidden interface is a blank full-screen image or sprite etc, with different coloured shapes (presumably rectangles, though anything works!) arranged at the icon positions and scaled to size if necessary. Instead of checking the MosueX and MouseY between various coordinates and seeing if the coords are within the bioundaries for whichever icon, simply a (hopefully faster????) check is made on the proportional X/Y of the image, and the RGB (or aRGB) value returned with something like ReadPixelFast from the HIDDEN interface buffer. This way, it can be ascertained what icon is clicked by testing for, say, $FF00FF (magenta?) then icon #1 is clicked, but if $000000 is returned, (black?) then no icon is clicked, and so no result. This may take a little organisation in setting up, but I'm certain it would be easier than checking through atual icon X,Y and width heights to find what's been clicked on. In theory, I''m thinking it would be faster too... ? Is this a possibility, has anyone tried something like this before? Or.. am I completely insane since there's a MUCH simpler way of doing it all that I'm too dumb to notice ;) NOTE: External progs, such as DevilsGUI system, or say, windows API calls etc. are beyond the sscope of this, since Im, more concerned about really customised UI systems. |
| ||
| I created a very simple 3D GUI for use with my Arkanoid level editor. It only has buttons and some simply requesters (Yes/No and a color picker), but I got it working like I wanted too. It uses pickable quads for the buttons. For the requesters, I just created a quad to which a texture is applied. To simulate the buttons, I created invisible (but pickable) quads in front of the requester-quad. By checking which triangle was clicked, I could see if the user clicked the "Yes" or "No" button on the requester. You could check it out here: http://users.telenet.be/vge/Arkanoid3D/Arkanoid3D.zip The full source-code of the editor and the game are included, as is the source-code of the 3D-GUI. But: I don't know how to make it pixel-perfect, so using it could result in slightly blurry buttons. |
| ||
| Nice.. I think I actually gave that game a try some time ago. I was a big fan of the original, so it was good to see :D But yeah, 3D UI is made a bit easier with CameraPicks and linePicks etc., and a more simple interfacte with just a couple of buttons is probably better coded specifically for those buttons. I was really considering smmething along the lines of having a larger number of possible icons to click, and where the same icons may appear on different 'menu screens'. --- incidentally, for pixel-perfectness, you just need to make the pick radii smaller and the quad textures to have flag 8 (mipmapped) disabled. Thwere's something about pixel-perfect sprites in the code-arcs, dunno, that might apply to the quads too ? |
| ||
| What I meann with "not pixel-perfect" is the fact that the buttons aren't the exact size on the screen when they are rendered. The button's image can be 256x32 pixels, but when rendered, the quad could cover a size of 300x38 pixels on the screen when the button is rendered. So I would need to find the exact distance to the camera (and the exact size of the button quad) to get the rendered-size equal to the image-size. I didn't bother with that, as it was only a simply editor. That also explains why the text on the buttons could be blurry, as 1 pixel of a character (on the image/texture) could cover 1.15 pixels or so onscreen when the quad is rendered with that text on it. I don't know if this makes any sense. |
| ||
| Bugger picks :) What to do, is align the 3d GUI elements so your 3d elements dimensions, match exactly the 2d pixels on the screen. Then all you do, is to rectsoverlap checks. Obviously, what you do is create every 3d GUI element on the one surface, and initially do a camerapick, to see if the GUI is checked. For elements that need to be hidden, simply move there vertices off screen, as making them invisible via vertexalpha will still cause them to be picked. |
| ||
| Seems that the 3D route is much preferred over old 2#D stuff anyway then :) |
| ||
| I still use 2d for my own editors, as i can't be assed setting up 3d :) |
| ||
| So would my suggestion be of use? |
| ||
| It sounds to me like it has potential. I was going to redesign my GUI anyway... will try this out and see if it's faster. One enhancement to your original suggestion though (assuming I've understood correctly) - rather than assigning arbitrary colours to buttons like $FF00FF, just make each colour that gadget's index in your GUI's "all-gadgets" bank or array (or its Handle() ). That way no ifs or loops are required: as long as the colour isn't your "null" colour (say $FFFFFF), just access the bank or array index with the value returned by ReadPixelFast. Hmmm... on the other hand, would it still be faster taking into account the time taken to render the interface buffer? If each gadget had a second quad, with the vertices already set to the index colour, it might do away with the need for a loop there... Certainly worth a test, simply by virtue of being different and therefore more fun. |
| ||
| Surely the ultimate speed comes from just doing rectoverlaps? |
| ||
| That'd be fastest for checking whether the mouse is over one specific gadget, but you still need to iterate through a list of gadgets to check whether they overlap the mouse. I'm sure it could be optimised, but the main difference is that there is no loop at all in Malice's method; you go straight from the test to the gadget (or to no gadget). O(1) vs. O(n) (I think). Of course, that advantage is irrelevant if you're also doing something else that loops through all the gadgets anyway. Also, as a bonus, you don't need to test which gadget is on top if more than one overlap the cursor, because the magic of rendering handles that. Not that I think this is a significant difference. And if you were really clever, you could take advantage of masked textures to allow gadgets to be any shape you want! I don't think you'd see a significant speed improvement in either case (both are probably pretty fast ways of doing it unless you have ten bajillion gadgets slowing down your system) but personally I'm drawn to the simplicity of the idea. |
| ||
| I agree, the speed difference would be minimal, and checking handles with RectOverlap would be potentially more reliable. RectOverlaps though, still require so many variables, such as the size and position of the icons and possibly a mouse cursor image. With the 'colour method', there's only the actual mouse pointer location and the pixel from an image buffer. Though again, this is a very minimal gain in memory, and perhaps lost out since using the image buffer itself. The mention of other shapes I think is most relevant, although presumably most 'button' type objects would be rectangular or perhaps with slightly rounded edges, but circular buttons or trangular arrows for some circumstances may be nice. I wonder if from a user point of view, though, if the pixel-perfectness is potentially a disadvantage, as soemtimes it may be beneficial to have a rectangular bounding box rather than require clicking so precisely. The simplicity was what really gave me the serious thought about the idea, and why I posted in the first place. I am sure there are at least some circumstanes where such an implementation may be required. The iteration of checking would never need to be specific to handles, only to colours. Whether this makes a difference or not is really due to how it's implemented I imagine, some relevance may need to be given to the colours according to the 'menu screen' in which they appear, but this is little different from handles. Yasha, it seems you've gone a little more in depth than me, of course, the original premise posted may well do with some optimisation, and things like the use of banks is interesting, something I'd not gotten as far as thinking. I'm working on something myself, more of a tech - demo test, really and I'd be interested in your results too. If there's any serious benefits or disadvantages they may well come to light through actual 'practical' use of the idea, rather than just theorising the possibilities :) |
| ||
| Okay... some further thoughts... Since the ARGB value returned from ReadPixel is a somewhat arbitrary, bbut remains a uniquely identifiable value, which is not as random as say, an image handle (numerical value) then there's certainly some opportunities there. If the actual value returned is somehow linked to the functionality of the gadget (one would need to be pretty specific about the colours they used in the hidden UI image buffer) but the theory goes.. Bitwise operations on a return value could be a real advantage. Even so much as potentially CHANGING the colours within the image, this change could be simple, such as changing the RED component from 0 to 255 etc. reflecting a toggle, or maybe the blue component might be a scale of values. Whilst only the GREEN component is required for identifying the specific gadget. This would limit a total number of gadgets to 255 but that ought to be plenty. A direct array or such of the gadget colours (or just green component) can be predefined, which should be a little easier and quicker than checking individual handles, since again, we can deal with bitwise operation on a single byte. This ought to be faster... |
| ||
| Ah, i understand what your up to now :) Very interesting idea! |
| ||
| Yeah I thought you seemed a little dismissive at first, Ross... Wondered if you did see what I was getting at. Who knows, it may turn out to be a complete waste of time and way too much ework for no real benefit, but it's certainly intriguing to find out :) All in all, I wonder why B3D ever incorporated the gadget UI stuff that I hear BlitzPlus has? |
| ||
| How are you handling gadgets that are hidden? |
| ||
| Alpha (vertex alpha) would work; just set it to zero and the gadget doesn't get rendered to the "interface buffer", so it can't be picked up by the mouse. Or in a pure-2D system... simply don't draw it. |
| ||
| I'm sure alphaed vertices still get picked by a camerapick. |
| ||
| I think we're describing different things. Here's what I have in mind (which upon closer inspection, I don't think is actually what Malice was suggesting... sorry): 1) Get hold of a pixel-perfect quad system. Not too hard to design a simple one, or just rip the core out of Draw3D or SpriteCandy or whatever your preferred drawing system is. 2) The way these normally work is with the drawing surface (or surfaces, as a surface can't have multiple textures side-by-side, so you need one for each image, but this is a detail - you can fit the main GUI into one surface) as a child of the camera. Now create a second camera/surface arrangement to act as the "interface buffer". (Quad systems should be fullbright anyway, but it's essential for this to work, so make sure it's fullbright.) 3) Set the interface camera's ClsColor to $FFFFFF. Apply a new, masked (not alpha) copy of the GUI texture to the interface surface, and go over it replacing all the non-masked areas with completely white pixels. 4) Each time you add a gadget to the main system, add a copy of it within the interface setup. VertexColor those vertices with the index number of that gadget in whatever list you're using (i.e. gadget 0 is coloured 0,0,0; gadget 1 is coloured 0,0,1; gadget 256 is coloured 0,1,0 etc.). 5) Every time you need to update the GUI, make sure the interface system is also updated with each gadget's new vertex positions. If the gadget is hidden or otherwise disabled, also set its alpha to 0. Don't change its colour. 6) Every time you need to get input from the GUI, render the interface scene without the rest of the world, and store it somehow. Then ReadPixelFast at the mouse coordinates; if the pixel is less than $FFFFFF (disregarding alpha), clickedGadget.TGadget = GlobalGadgetList(pixel)... or Object.TGadget(pixel)... or Object.TGadget(PeekInt(GadgetBank, pixel)). Whichever. That's what I immediately thought of when reading Malice's idea. No idea how close it is to the mark. Although it might not be the fastest way around the problem, it's so simple and straightforward (things like window ordering or disabling gadgets pretty much take care of themselves) that it would be a good foundation for a general-purpose system. I think I will be giving it a try. |
| ||
| It seems we do have our wires crossed. My idea was purely 2D, I didn't want to address 3D stuff at all. The importance of my idea was the ability to identify the location of the mouse cursor by a particular colour, which not only mayy represent a specific 'gadget', but things like the 'state' and 'value' of the gadget could also be known, without having to iterate through all the gadgets individually. What it seems Yasha's doing, is taking the idea of the (never rendered - only the buffer is checked) 'overlay', and applying that to a 3D window, so I suppose ultimately the effect could be the same, so I'm still interested :) it would certainly improve accuracy of mouseclicks, since these would be registered against 2D coords rather than interacting with the 3D world. |
| ||
| The importance of my idea was the ability to identify the location of the mouse cursor by a particular colour, which not only mayy represent a specific 'gadget', but things like the 'state' and 'value' of the gadget could also be known, without having to iterate through all the gadgets individually. I briefly skimmed the suggestion so forgive me if I've missed something. While this color based gadget type may sound clever in theory I don't think it is very practical and there is little scope for expansion of properties. For example, how would you handle properties such as visibile / active, scale etc... I think it would be very messy to code and you should just stick to tried and tested methods. Checking through 100's of gadgets to see if the mouse has overlapped is trivial in terms of speed with rects overlap. It's unlikely that you would have a menu that cluttered in the first place. I think you are trying to optimise when it isn't really necessary. |
| ||
| While this color based gadget type may sound clever in theory I don't think it is very practical and there is little scope for expansion of properties. For example, how would you handle properties such as visibile / active, scale etc... I think it would be very messy to code and you should just stick to tried and tested methods. Checking through 100's of gadgets to see if the mouse has overlapped is trivial in terms of speed with rects overlap. It's unlikely that you would have a menu that cluttered in the first place. I think you are trying to optimise when it isn't really necessary. First, please dont confuse the original ieda with something that would mimic the functionality of something like Visual Basic gadgets. I never actually brought 'gadgets' into the discussion, nor 3D, nor overlapping gadgets. Not that I object to any of the discussion on those. What I was really aiming for with the initial thought, was a simpler front-end for a typical game, or perhaps tthe panel of control 'icons' during gameplay. Some form of menu system with buttons, maybe sliders and checkboxes at the most, but nothing too complex. Consider the concept for this icon-driven menu system or UI or whatever you want to call it, in 2D (though it would work equally well for 3D) The code for doing such, to me, is a pain, even with a pretty neat Type based system, mainly, having to identify about 6 pieces of information for every possible icon. i.e. IF MOUSE BUTTON CLICKED X of topleft Y of topleft Width Height MouseX MouseY Admittedly, simplified by some careful arrangement / scaling and relationship of the positions of the icons etc. bt still so many pieces of information may be needed to chweck through just to identify if a particular icon was clicked on. Even then, the return value (traditiohally) will only tell you something like: A Type instance or handle relating to that icon What's then required is iteration through types, or some other method to identify the funcitonality related to that particular icon. However, with the 'colour-idea' (I should probably think up some kinda name for it), only 2 pieces of information are needed. MouseX MouseY The return value can then identify a more specific 'description' of the icon and its functionality, including perhaps, its current state and maybe even values stored. The opportunities to work wioth the bit values of the result are also much more accessible, than, say, extra fields in a Type or something. Scaling can't be a problem, since in creating the 'vissible UI' then the hidden colour buffer is going to be drawn up from that. That should never be an issue. As for hundreds of gadgets etc. and overlapping, I do not see any reason why a UI would be made with overlapping objects. There will always be something that is considered "At the front" and it is this that would be used to determine the colour on the hidden buffer. It may sound messy at first, but once you can envisage the actual simplicity of literally just ReadPixelFast(MouseX),MouseY(),HiddenImageBuffer()) giving you an exact description of the target icon that has been clicked, then I think there COULD be some potential in there that just needs to find the right slot to fill :) this all started purely as an alternative to checking 6 pieces of information for n different 'icons' each of which would need be checked every time the mouse clicks. |
| ||
| I like your idea Malice. In my severely limited experience coding in Blitz I've had troubled times writing GUIs with only a few buttons. I've used a MouseInRect function approach, but your's is a lot neater - in fact, you'd just need a select case structure for the ReadPixelFast call and you can find the button without going through a list - great idea. |
| ||
A very simplified examle:; Disable buttons by removing the button reference bit valuefrom this. Global ENABLED_BUTTON_BITS=255 While (Not (KeyDown(1))) Draw VisibleUIImage,0,0 If MouseHit(1) Local MX=MouseX(),MY=MouseY() Flushmouse Local Button=GetButtonPress(MX,MY) DoButtonFunction(Button) End if Flip Wend Function GetButtonPress(X,Y) LockBuffer(ImageBuffer(InvisibleUIImage)) Local ColourValue=ReadPixelFast(X,Y,ImageBuffer(InvisibleUIImage)) UnlockBuffer (ImageBuffer(InvisibleUIImage)) ;We can see if the button is enabled or disabled directly: If ((ENABLED_BUTTON_BITS And ((ColourValue And 255) Shr 8))=((ColourValue And 255) Shr 8) ) Then Return ((ColourValue And 255) Shr 8)) ; Else, return 0 for no button clicked... Return False End Function So, there's no rectoverlap, no list iteration and no need to know any more than the mouse location and a single return value identifieds the specific button by reference, not needing to be obtained from a handle or type object, and even ensures that a 'disabled' button has no effect. |
   |