Need advice on pixel plotting
Blitz3D Forums/Blitz3D Beginners Area/Need advice on pixel plotting
| ||
| I'm new to the site but I've had blitz 3d since I bought it in 2005 or 2006. I've run into a problem with the engine I'm designing for advanced 2d game play. I'm using a complicated system of functions to setup a camera that draws to the back buffer from an array and my issue is not with the camera's functions themselves, the system runs super fast with just them running, but as soon as I put in the write pixel fast command it slows to a near stop at 1024 by 768. It runs fine at smaller sizes such as if I attempt to render 400 by 300. I need to know if any changes have been made to blitz 3d to allow a faster plotting of pixels onto the buffer. I'm using write pixel fast properly, with lock buffer and unlock buffer, but the issue is that the command used 1024 multiplied by 768 is way to slow! How can Blitz draw an image that is the same size and yet I cant do the same thing pulling my source from an array? I need help and advice on how to proceed. If I can get this to run as fast as one drawimage command then my engine will be able to do some impressive stuff. |
| ||
| The engine I'm developing has the potential to be a 3d engine, mind you it would cost an unbelievable amount of RAM. However, the 2d form of the engine can already allow rotation of multiple images as well as the rotation of the camera. It can also allow dynamic zooming in and out of the image. Its a fairly impressive piece of work but the thing limiting it is the size of the box that the camera plots to on screen. I have to limit the camera size to 400 by 300 in order to get it running at real time. If I can solve this issue by some other method of plotting the pixels onto the screen then I can speed up the system, and have a full screen camera. It will also allow for ray based lighting and transparency once I settle the color variable issue I have with having to use the (read pixel) esk color code. I could easily use a RRR,GGG,BBB integer that works with the color rrr,ggg,bbb, plot pixel method. Anyone have any advice on alternatives to writepixelfast? |
| ||
| Hum, saw the topic in the BlitzMax area first... (copied answer): has any improvements been made on the plotting or writing of pixels onto the screen, backbuffer mind you, that speeds it up? No. Modern hardware developments have all been in the 3D and shader realm. WritePixelFast uses pure software: you write bytes to memory from the CPU, and it uploads the altered buffer back to the GPU when you unlock it again the "slow way" (as for Plot, it locks and unlocks the buffer for every call, so that's even worse). It's completely CPU-bound and has to go through the slow channel as well to get back to the graphics card. Drawing a 1024x768 buffer with the CPU I would entirely expect to cause serious performance problems. Drawing an image is faster because it can blit the whole thing in one fast primitive operation. The way to do this sort of thing fast is: - work out a way to achieve what you want using accelerated operations (i.e. polygons, trickery with 3D objects) - use shaders, oft-demanded but not available in B3D (requires a completely different programming model) A 2D game engine using 3D graphics (e.g. Hectic's Draw3D library) will perform much better than native B3D 2D graphics. I'd advise that you not use the 2D engine at all if you can avoid it. 2D is simply very slow. The 3D engine has transparency, rotation etc. built in. I know it's a real downer to say this at this stage in development, but trying to do this stuff in the unaccelerated pixel 2D engine is a complete waste of time. You should be using the 3D engine if you care at all about performance, because it's thousands of times faster. Check out Draw3D to find all of this stuff done for you already: http://www.blitzbasic.com/Community/posts.php?topic=92270 (There's also an older, smaller version that is obsolete but fun as well: http://www.blitzbasic.com/Community/posts.php?topic=78304 ) Last edited 2012 |
| ||
| Thank you for your answer, and I respect your thoughts on how the method I'm using is inferior to the 3d approach, but this project is already showing promise and since its my very first major attempt at a game engine I will continue work on it. However, since I cant speed it up any, can anyone give me answers on two other ideas that I'd like to incorporate into the engine? The first is that although I've gotten my engine to run and run well considering that it is inferior to modern models, I can solve my speed issue If I can learn how to have the camera run on one core and the AI on the other, both accessing the same array that serves as the level. Can blitz use a multiple core setup to have two separate programs running that share resources or can I run one function set on one core and tell another core to run the other? The camera system would probably allow for that setup. The other issue is that I don't understand the variable that read pixel fast outputs and that write pixel fast uses. I was using a stacked variable 255,255,255 R,G,B but I don't know how to separate the ARGB variable into its distinct parts. |
| ||
| (Drawing an image is faster because it can blit the whole thing in one fast primitive operation.) copied from above, So, the operation exists that can quickly draw the image, is there any possible way I can use that operation only instead of from a file, I would draw the x,y, and color data from an array? I could even set up an array that is meant only for the camera's view port and that way every pixel is in line for drawing? I could use the round about way of saving the data into bitmap format or etc once I learn the format and design a function to save the pixel data in that format. Then I could re load it and draw onto the screen. But even though I've used that method with a 3d game demo I worked on, for a mirrored surface and using two cameras, I'd rather avoid that method unless necessary. But, could I somehow setup an array to work as an draw image call or load image call? If I stored the pixel color data in the appropriate way then it could run as fast as drawing one image to screen. Last edited 2012 |
| ||
| As long as you know what you're doing, that's a perfectly good reason to keep working, and I hope you get it working. Can blitz use a multiple core setup The official answer is "no", unless you want to go way overkill and have a separate server application on the same machine that passes messages to the renderer (this might be appropriate for a multiplayer game, since it'd have to communicate over the network with other clients anyway, but is a bit silly for SP). The unofficial answer is "yes", using the FastPointer library by Mikhail Vostrikov (bottom of the page, free). However, because B3D is not set up for multithreading, you use this at your own risk and with no support from the language. You will need to be fully confident in the use of pointers, function pointers, and rather hackish synchronisation methods, to get anything out of this (the rest of us are here to answer questions, but that can never compete with an actual language feature). The big problem is the lack of synchronisation. Rendering needs the state of the game logic to be "complete" so that it can draw a coherent scene. Unfortunately locks aren't something that can easily be provided as a library feature, so syncing your render thread with your AI thread may be quite a challenge. Syncing is an as-yet unsolved problem in B3D (the old textbook solutions don't work on modern CPUs thanks to instruction reordering). The other issue is that I don't understand the variable that read pixel fast outputs and that write pixel fast uses. I was using a stacked variable 255,255,255 R,G,B but I don't know how to separate the ARGB variable into its distinct parts. The variable is a four-byte integer. Each byte represents one of A, R, G, B as a value from 0-255: A is the highest byte and B is the lowest. To extract a byte, first mask the other three and then shift it to the low position: a = (argb And $FF000000) Shr 24 r = (argb And $FF0000) Shr 16 g = (argb And $FF00) Shr 8 b = argb And $FF To build up a colour from individual bytes, first shift the bytes to the right positions, then compose: argb = (a Shl 24) Or (r Shl 16) Or (g Shl 8) Or b is there any possible way I can use that operation only instead of from a file With a lot of hacking you can use memcpy operations to move the contents of a bank or array "over" where an image is supposed to be, but 1) this is Deep Magic, and 2) in practice it doesn't actually save you very much (might be worth doing, if you need to eke out every millisecond, but don't expect wonders). The reason is because images reside in specialised video memory and are handled by DirectX. They can only be interacted with as data by being copied to general system memory. The same restriction that slows down WritePixelFast applies, only you skip a tiny bit of overhead. Copying the bank to the "locked" image is superfast, but "unlocking" the image often is not. Still, if you don't mind working with pointers, give this a go and see what happens. Best I can suggest is to not Write or Plot every pixel every frame, only the changes to an intermediate buffer image (or bank, I guess); then use DrawImage. This will save you the cost of rewriting most of the unchanged pixels. Also, look at this: http://www.blitzbasic.com/Community/posts.php?topic=87923 It's pretty fast. Maybe it has some useful ideas for you to borrow. |
| ||
| Your defiantly much smarter than I am when it comes to programming. I think I understood some of what you said. What did you think of using the (write file) method of saving an image and then reloading it. It didn't slow down my 3d demo when I used a screenshot of one camera pasted onto another object as a mirror. Could that work here if I can figure out how to save into bmp format? |
| ||
| I was using one integer to store the red green and blue zero to 255 values. It was a 255,255,255 variable that needed to be unstacked via a function that first found red, then used the red variable to erase the red from the green variable after dividing the original variable by 1000 to put the blue into the decimal spots and thus erasing it. There was the chance that it would round up the variable but it was worth the risk for one pixel off color. Is that similar to the ARGB you explained? |
| ||
| Aside from the sentimental reasons for continuing this project, since I'm not using the graphics engine for my project, this program could work for many more computers regardless of direct X or graphics card. Also, you could use the graphics card for other stuff like cool 3d interactive menus n stuff. But I want to get this running well so I can start putting in effects like ray based lighting, the pixel to pixel collision system that allows drawing your collision effects into the art of the level and not having to worry about X Y Z of the image you'd otherwise be using to make the level. The levels are painted into existence and will be able to be multi layered. I can scan in a 3d model into my array and then use a 3d program to add color and collision values. The engine is so expandable in my eyes and to give up just because a 2005 built computer cant run it at 1024 by 768 without slowdown is just not in my mind a good enough reason. By the way, I'm not knowledgeable about hardware, how much faster would you say the latest tech is compared to a very well built computer from 2005? I know its hard to ask not knowing what parts were used, but if they are at least twice as fast then i'm ok. |
| ||
| It didn't slow down my 3d demo when I used a screenshot of one camera pasted onto another object as a mirror. Could that work here if I can figure out how to save into bmp format? It would work... I'd be amazed if it wasn't terribly slow (disk access is usually really slow), but I guess test and see what happens. I'm not sure I correctly understand your idea. If you can find the FreeImage library (floating around the Toolbox or Code Archives here somewhere) that might save you the trouble of working out image formats... I think you can just pass a bank and parameters to it or something. Haven't used it myself. I think what you're proposing sounds like a roundabout way of doing the same thing as I mentioned above, though, copying memory onto an image area; only using the disk. Old example code of poking directly to image memory: Unfortunately I can no longer find the magic tables that helped get hold of the right offsets for this, which is a pain (although you could just copy those in the example). a function that first found red, then used the red variable to erase the red from the green variable after dividing the original variable by 1000 to put the blue into the decimal spots and thus erasing it. There was the chance that it would round up the variable but it was worth the risk for one pixel off color. Is that similar to the ARGB you explained? Similar but it sounds less efficient. To be honest you can just copy the code from the post above, it will work as it is for most purposes. if they are at least twice as fast then i'm ok. Uh... probably, I guess. You probably don't need to worry about those details at this time. Just focus on getting what you want working. Last edited 2012 |
| ||
| To reiterate, the 3d demo I built was just to test ideas I had on how to construct a 3d engine for a fps. the first camera was placed at the object that was reflective, then pointed toward the player. Then after it rendered to the back buffer I saved a screenshot and then reloaded that screenshot to the texture pasted onto the 3d model. it provided a mirror surface with the main character's reflection. I used a similar method to create a distorted visor that the character looked through into the real world. But I just want to save a file from an array so that I can reload it and draw it onto the screen. That's my roundabout solution. |
| ||
| I attempted to solve my speed issue with a few ideas that all failed to prove faster. I tried a 3d solution with 1024 by 768 small squares the size of a pixel, changing their color to match the color the camera would normally plot, but it proved to slow to even consider. I had the camera range set to only show the depth of the pixels, so its not feasible. The other solution was drawing a single animated image with a frame for every color. That solution proved slower than the system I started with. using drawanimimage 1024 by 768 times was worse than using writepixel fast the same amount. So I've decided to limit the camera's output to just 400 by 300 and have a screen resolution of just 640 by 480. I'll just have to have a border to serve as the hud. I would dare to set the graphics to 400 by 300 but would that be able to run on most graphics cards? Or would it be too weird? What do you think? |
| ||
| rotation of multiple images...... zooming in and out of the image...... transparency Are you familiar with the Extended Blitz2d dll? |
| ||
| I may be able to use blitz 3d, but there are allot of things I still do not understand about it and programming in general. The real strength of my engine is that you can influence every pixel, and by selecting one pixel you can effect the owner of that pixel with collision or lighting, etc. I have many ideas for games that require the depth of control I've put into this engine. If you know of a way to help me speed this up by offering an alternative method of plotting onto the screen then I'd be very grateful. As it stands, the only thing slowing this engine down is the fact that I cant figure out a way to quickly and efficiently draw the camera onto the screen. I'm thinking of making this freeware, as well as the paint program and other tools necessary to use it, but if its going of any use to anyone then I need to either figure out how to send the pixel data held in the camera array over where the graphics card can use it and quickly draw it out, or have a way to have the array or bank belong there in the first place and to quickly change and read those integers. Any ideas? |
| ||
| or have a way to have the array or bank belong there in the first place and to quickly change and read those integers. This is essentially a reinvention of the concept of the "fragment shader". On this, I can only repeat what I've said above: there is no way to do this from the CPU. The connection to the graphics card is simply not fast enough, and no amount of "more modern hardware" will help, because this isn't something hardware designers care about improving. You need to use an engine that can let you write this stuff as a shader program, and Blitz3D does not provide access to shaders in any form. While software renderers do exist - take a look at the Quake 1 and Quake 2 source code, they have a phenomenal software renderer capable of getting hundreds of FPS even at high resolutions (Carmack is a god, basically) - they don't offer anything like the kind of power you're describing: everything is done in bulk-operations and "works down" to the pixel level rather than "working up" from it. Custom pixel-level effects (where a pixel is affected by other fragments, e.g. blurring, where a pixel is averaged with the fragment around it; rather than simply dictated to by a higher-level construct like polygons) require a much higher degree of processing, and for that you need shaders. (Either that or I still don't understand your goals, which is entirely possible.) |
| ||
| Well, I'll try to state my goal one more time even tho I think there is no way for it to work. I created an array with three dimensions, level(100,100,3) by thinking of the first dimension as the x variable and the second as the y, the last dimension in the array is the red,green,blue values of each pixel located at the x,y you give when plotting that pixel or changing the values. This could be a 3d engine if I added the z dimension to it. By having access to the color values and being able to change them I can do all sorts of things. I came up with a set of formulas that allow me to know the slope x and slope y and even slope z of a line, if you think of those slopes as the amount you add to a variable each movement along the line. That allowed me to create a rotating camera and vise versa a plotter that would insert the pixels into the array where they were needed to represent the characters or etc. The level array would load from file which means that you could paint the level then insert your characters into it. By adding another integer to the final dimension, I could add room or area values that the camera would skip over if that pixel wasn't in the current room and would allow for transparency if I used the extra z layers for depth. I could run ray based lighting with the same formulas that allowed me to capture and plot pixels in any angle line. I was able to zoom in and out by any variable, even floating point. Overall, it allows all of the control I wanted for any project I would use. Also, its dead simple except the plotting which has slowed it to a crawl unless I keep it under 400 by 300. So there. Unless I can somehow plot the pixels faster, this engine will be useless except on a really fast computer as is. Any thoughts? |
| ||
| 1) Have you tried the FastPointer method shown above? It should shave a couple of milliseconds off using WritePixelFast to plot everything. 2) Have you timed the loop itself? How long it takes B3D to do a nested pair of For x = 0 to ... For y = 0 to... is your basic minimum you won't be able to go below (this should not be crushing the computer, though). 3) Using a similar (but even more nasty and hackish) trick to the bank hack, it may be possible to remove the overhead of a function call for each pixel, by tricking B3D into mapping what it thinks is an array over the top of the locked buffer. You could either do something like this: http://www.blitzbasic.com/Community/posts.php?topic=65801#735708 ...to reseat a C-style array "over" a locked buffer (note that this would crash in debug mode, but is likely the fastest option); or you could use FastPointer's "ArrayPointer" function combined with RtlMoveMemory to move a buffer's contents into a Dim array (slower but probably safer). ...if that doesn't help (I can elaborate if you need help implementing that), then my only other suggestion is to use a language with a more powerful optimising compiler, like BlitzMax or in this case, really you need C. B3D loses a lot of CPU cycles to inefficient machine code (it's actually not much faster performance-wise than top-end interpreted languages), which is why you can't easily match the Quake renderer's speeds. |
| ||
| Well, as I've mentioned before, I'm not as good as other programmers when it comes to understanding all of what is being said. I can craft programs pretty well but when you start into the "Deep magic" of programming then I'm a little lost. I am self taught in programming and as I know very little of hardware or the deeper points of software I am limited to the tools blitz supplies that I can comprehend. I understand what your suggesting, but I doubt I can implement it. So I've decided to put this project onto the back burner until I either learn c++ or until I learn more about the deeper magic of programming. I am working presently on a slight modification of this idea but instead of going on a pixel to pixel basis, I'm using one integer to represent an image or its frame. I have a three layer array for backdrop, mid ground, and foreground. I'm working on the level editor and using some of my other projects I've got it running well enough. I appreciate the suggestions and help you've given but until I'm able to implement these ideas I'll have to work on other projects. Thank you. |
| ||
| Oh, to answer your questions, (1) off hand I don't remember Fastpointer Method, could you spell it out for me? (2) I haven't bothered to time the loop itself yet because the rendition of the camera I've set up isn't including all the code necessary to manage the array. There are so many options for how to implement my system that I haven't settled on one yet. For instance, I could have the level on one layer and draw all items and characters on another and then check collisions with the level layer without ever worrying of overwriting the level by accident when I move the characters or effects. I can include in the array the extra integer necessary for light data or for area, room ,transparency, etc. Until I can get the engine to draw as fast as can be managed it really wasn't worth the additional effort to set it up if it was too slow. Remember, my hopes were to have the camera be as efficient as drawing one full screen image. That would have been Blazing fast! But now that it is proving too slow on older computers to even work as a game engine let alone a paint program, I'm having to shelf it until I can improve it. If you want to continue working on this with me I'll be glad to put in my inspiration. We could make this a completely public, free, project. If we could get it to work the way I had in-visioned then it could be something spectacular. |
| ||
| Well I am kinda curious to see what's going on and where I might be able to make any suggestions for speed improvements. That becomes a lot easier if you intend to make it open sourced. I would suggest sticking it up on GitHub or Google code or something. As for "FastPointer methods", there are a few ways to interpret that, but all of them basically mean using trickery to "reseat" something with very fast access times over a different piece of memory that wouldn't normally have those fast accesses (i.e. image buffers, requiring a whole function call to set or get each pixel). |
| ||
| My access point is currently a public library with limited access. I can get my father, who I'm living at home with cause I'm poor, to download stuff for me and maybe act as upload as well but I've no Idea whether or not I want to open source this yet. I'll need to think about how I want to send the files to you if I decide to work on this publicly or with a partner. I could probably winzip the whole folder and send it all via email, the program isn't very long. Let me think about it some more before I make a decision. As for what you've described, it sounds right. I can design the camera to pull the pixels from the level, then paste them into a camera named array which we would then transfer to the appropriate place to then use with a draw image call. That is if I understand the suggestion. |
   |