Memory Management: Limit Length of String?

BlitzMax Forums/BlitzMax Programming/Memory Management: Limit Length of String?

Grisu(Posted 2013) [#1]
Hi there,

is it possible to limit the maximum length of a string variable (e.g. 12 chars) in order to save some ram?
Or does BMX allocate the string size dynamically (depending on the usage)?

Thanks!
Grisu


xlsior(Posted 2013) [#2]
Since you can create strings that are hundreds of megabytes or more depending on free memory, they must be created dynamically.

IIRC string size is 1 byte for each character, plus 4 bytes overhead.

(so a 100 character string should take 100+4=104 bytes)


Yasha(Posted 2013) [#3]
You can view the implementation of strings in mod/brl.mod/blitz.mod/blitz_string.h (and .c).

Strings are allocated dynamically and sized to the exact number of characters (so they don't have any special buffering treatment within the string class). Characters appear to be fixed-width two-byte wchars, so the size of a string is two times the length, plus twelve bytes of object header, rounded up to the nearest 16 (if I understand the GC correctly, which I probably don't).


If you really care about saving every last byte, and you know how many strings you have and that they're all the same size in advance, you could conceivably allocate them as arrays of raw character data in an untyped memory block instead? This isn't going to be worth your time in the majority of cases though.


Grisu(Posted 2013) [#4]
Then I have to find other ways to reduce my apps memory consumption.

"Int vs Byte" declaration, would such a change make some difference?
Or does BMX convert a variable to Int anyway?


Brucey(Posted 2013) [#5]
"Int vs Byte" declaration, would such a change make some difference?

Good grief!?! How much memory does your PC have?
Int/Byte isn't likely to make a huge difference unless you have thousands upon thousands of values you are storing in large arrays.

Fields in types tend to pad out to 32-bit alignment.
Field order can make a little difference, but it only matters if you have lots and lots and lots of types.
For example, having Byte, Int, Byte, Int, Byte, Byte Fields will take up more space than if all the Bytes are grouped together.
But again, unless you are using a significant number of types, you won't see much difference.


Midimaster(Posted 2013) [#6]
why do you think, the strings are your problem? Normally they do not matter in memory consuption.

Make a short research thru your file and tell us:

1.
How much Bytes do all aour resources have together?

2.
How many image files?
How big are the biggest ten?
How many pixels width height has the biggest one?

3.
How many audio files?
How big are the biggest ten?
How many seconds playing time has the biggest one?

4.
How many text/data files?
How big are the biggest ten?


Grisu(Posted 2013) [#7]
Thanks for the explanation Brucey.

My PC has enough memory, I'm just trying to optimise my code where possible. As more and more stations make it into my database I want to keep it light-weight.

I regrouped and shrinked the number of my variables to:
Field name$,filename$,genre:Byte,rating:Byte,fav:Byte,recent:Byte

This seems to work just fine.


ImaginaryHuman(Posted 2013) [#8]
Can you use compression at all?

Bytes will save you 75% of memory over its, possibly, depending how they're stored by the system... but they aren't faster.


ImaginaryHuman(Posted 2013) [#9]
You might also consider bitfields, using 1 bit to represent a status and store 32 of them together.


Grisu(Posted 2013) [#10]
I think, I'm now quite near the limit. For the memory savings I would get the amount of work is not "worth" it.

With 1.700+ small data files (zips) I need some overhead structure in order to reduce HDD access though.


Midimaster(Posted 2013) [#11]
are these 1700+ zips READ ONLY? Or do the contents change often? I asked, because if you pack them together in bigger files it would reduce the HDD use and rise the speed of reading all...


Grisu(Posted 2013) [#12]
They are changed frequently and the user can edit them as well. On top of that there are logo files that come with most of the zips. That's why I keep a fraction of the data in memory in order to create list views etc.

The only thing that might decrease the loading speed a bit is to mutithread the whole sequence via Bah.Libcurl.

If you like you may check out the app to see what it does: