Memory error when setting up an array
BlitzMax Forums/BlitzMax Beginners Area/Memory error when setting up an array
| ||
| Hi Guys I have been getting an error message after writing a new function. I have narrowed it down to one line of code, which is setting up a new array. I have some very large arrays in the code: Global world [1100,1100,210] Global light [1100,1100,210] Global state[1050,1050,210] It doesn't matter what order they are in, it is always the last one that triggers the error. Is there a maximum amount of memory allowable for arrays? Must they be a certain size, or is it possible to allocate more memory to them? Thanks Dave The error message was: mygame(791,0xa074e540) malloc: *** mmap(size=926101504) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug mygame(791,0xa074e540) malloc: *** mmap(size=926101504) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug |
| ||
| Is there a maximum amount of memory allowable for arrays? There's a maximum amount of memory available to 32-bit applications, and it looks like you've gone over it. That's a huge amount of memory... do you really intend to have two hundred million objects in your scene? (This is one reason why there's a thread asking for 64-bit support.) Last edited 2012 |
| ||
| Those arrays take up about one gigabyte each (1100*1100*210*4). The three of them will require ~3GB total. Under 32-bit, windows will allow a single application to access up to 2GB of RAM, total. The only way around that is to either apply some windows tweaks that can extend it to 3GB (but your app still wouldn't work on 'stock' windows), or monkey around with some special virtual memory approach. Realistically, there is no way you'll be able to use arrays that large from a 32-bit application. |
| ||
| Thanks for the feedback. I really appreciate it. It's time I started to think less ambitiously, or start being smarter about how I handle the data. |
| ||
| Just stream sections of your map to/from the disk in much smaller chunks. That seems unnecessary, makes me wonder what you are actually doing?!?! |
| ||
| Why do you need them this large? |
| ||
| Yea, that's ridiculously large specially considering that they are objects. If you post a description of what you are doing maybe we can figure out an alternative for you. Last edited 2012 |
| ||
| Wow man, you're trying to put a square into a circle with those super-arrays O_O . So far there's not going to be a 64-bit B/Max. Like Jesse said, if you're telling us what you're trying to pull, we may be able to figure out another way. |
| ||
| Hi Guys. Thanks for the help. You are right. I am trying to squeeze too much out of an array. I have reduced the size of them by half, and ditched one of them and I am fine. But there is probably a much better way to do what I am doing, if you have some ideas. Basically, I have a 3D world displayed in an isometric graphics. 1000 x 1000 x 120 high. I would like to move around the world without stopping for load times. I have accomplished this already, but no doubt it isn't managing memory effectively. The first array stores the value that dictates what kind of block it is. The second array is storing the data for the level of light on each block. I can probably add another notch in the first array to hold this data, but I don't know if it would use less memory anyway. - Oh just had an idea. I don't need to know the light for every block in the whole world, only the world we can see. A big saving there already. Cheers Dave |
| ||
| do you really need it 120 high? it's king of a lot considering its an isometric tile map. also instead of saving all of your data in objects why don't you save it in an integer array. use the first 8 bits(0-255), the next 8 bits(0-255) for the light and the next 8 bits... for the state, and it will also leave you 8 bits... for anything else you want to do with it. I mentioned the values between 0-255 because thats what you mentioned in a previous thread. I would like to move around the world without stopping for load times. if you use threading you wont have to wait but you will have to be creative at loading the various parts. you can load a certain segment only when approaching close to it. that way it will look fluid, not worry about load time, and you can make it the map any size you want. |
| ||
| Threading? Just read up on it, and it opens up a world of possibility. Thanks so much. |
| ||
| 120 elevations is quite a big high. My maps are probably gonna be a maximum of 1024x1024 grids, with maybe 4 or 5 elevations tops (units move up a cliff or something). You could do this with Field Game_Grid : Int[1024,1024,5] Each of those values has either True or False, depending if a tile is occupied or not. The field above produces and array of 5,242,880 integers. This saves a lot of memory and you can easily create methods to manipulate the state of a tile on the fly. |
| ||
| Hiya, @SystemError51 As you're only storing true or false in your array, for an instant memory saving just declare the array as :Byte, if you wanted maximum saving then store each entry at the bit level. :- Global IntegerArray:Int[1024,1024,5] ' 5 levels up Global ByteArray:Byte[256,256,5] ' 5 levels up Global BitArray:Int[32,32,1] ' 32 levels up Print SizeOf(IntegerArray) Print SizeOf(ByteArray) Print SizeOf(BitArray) To be honest what with all computers having thousand of meg of memory, it's probably not worth the saving to go to the bit level, but it would make a difference in speed when saving and loading files. Food for thought. Last edited 2012 Last edited 2012 |
| ||
| I'd imagine you have a lot of empty cells in this array meaning an entirely different data structure is probably going to be better in the long run for memory efficiency... |
| ||
| Run Length Encoding 1 Run Length Encoding 2 |
| ||
| Why don't you do some sort of post array initialization? Something like World.AddTile(x) World.AddTile(z) etc. World.Initialize() 'Count all of the tiles added and create array based off of this data |
   |