[MaxIDE] [TextStream] Unicode Bug on OSX

Archives Forums/BlitzMax Bug Reports/[MaxIDE] [TextStream] Unicode Bug on OSX

Blitzcoder(Posted 2009) [#1]
Hey BRL,

MaxIDE has a problem to save special chars in UTF-8. Here's an example:

The File contains the character 'ö'. It should be saved in 2 Bytes 0xC3B6, but it gets saved as 1 Byte 0xF6. 0xF6 is the unicode-codepos of 'ö'.

This bug only appears on MacOSX. Linux and Windows work fine.

Here are 2 Hexdumps of the string "Hallöle".

This is how it should be saved:
48 61 6C 6C C3 B6 6C 65

This is, how it gets saved:
48 61 6C 6C F6 6C 65

You see, 0xF6 is getting saved instead of 0xC3B6.

This is a very nasty bug, because other Text-Editors on OSX cannot read these files properly, and we germans often use the chars ö,ü, and ä ;-)

Best regards,

Blitzcoder


marksibly(Posted 2009) [#2]
Hi,

The IDE (actually, SaveText) currently saves documents in a Win32 8 bit LATIN1 format unless unicode chars >255 are present in the document, in which case it uses UTF16.

This used to be the only way to get those far out French/German chars into a document, but has been superceded by UTF8 these days I guess. Well, on MacOS/Linux at least.

I'm a little wary of just assuming *everything* is UTF8 these days (at the very least, some existing .bmx files such as yours will blow up!) so what's probably needed is some kind of 'Use UTF8' option in the IDE.

Will think more on this...