C++ ToLower and ToUpper with UTF-8 strings

Monkey Forums/Monkey Bug Reports/C++ ToLower and ToUpper with UTF-8 strings

devolonter

(Posted 2013) [#1]

Hi,

C++ ToLower and ToUpper do not work properly with UTF-8 strings.

Following code:

Function Main()
	Print "UüuÜ".ToUpper()
	Print "UüuÜ".ToLower()
End Function

Prints the following in the C++ Tool and Desktop targets:

UüuÜ
UüuÜ

Tested with trans V1.59

marksibly

(Posted 2013) [#2]

Not sure what you mean - what output were you expecting? I see U?U? and u?u? (with ? inside a 'diamond' shape) here...

Note that ToUpper/ToLower currently only work with a-z, as that's the only 'definition' of upper/lower case I'm aware of - are there others?

devolonter

(Posted 2013) [#3]

Mark,

It seems that forum not correctly displays UTF-8 symbols. Actually in my example I have *u-umlaut* in *?* places. OK, I've uploaded an example file - http://devolonter.ru/monkey/utf8.monkey In my project I need to transform cyrillic. Is there any chance that these methods will support transform other symbols besides a-z?

marksibly

(Posted 2013) [#4]

Give this a try - only tested on Mac though:

https://github.com/blitz-research/monkey/blob/develop/modules/monkey/native/lang.cpp

Also, can you see umlauts in this? Works on Mac...

Function Main()
	Local str:="ZoÃ« SaldaÃ±a played in La maldiciÃ³n del padre Cardona.Ã«Ã¨Ã±"
	
	Print str
	Print str.ToLower()
	Print str.ToUpper()
End

...and the output...

ZoÃ« SaldaÃ±a played in La maldiciÃ³n del padre Cardona.Ã«Ã¨Ã±
zoÃ« saldaÃ±a played in la maldiciÃ³n del padre cardona.Ã«Ã¨Ã±
ZOÃ‹ SALDAÃ‘A PLAYED IN LA MALDICIÃ“N DEL PADRE CARDONA.Ã‹ÃˆÃ‘

devolonter

(Posted 2013) [#5]

Thanks for that! It works here (Windows 8 RU), but needs a small fixes. I've uploaded the fixed version - http://devolonter.ru/monkey/lang.cpp I hope that doesn't break anything on Mac though.

marksibly

(Posted 2013) [#6]

Oops, forgot the 2nd towupper/towlower.

However, changing the locale name to "" breaks it on Mac...will look into it.

marksibly

(Posted 2013) [#7]

Ok, try the github dev version again - works here on windows (mingw and msvc), linux and macos.

devolonter

(Posted 2013) [#8]

Just tried it and it works fine here too. Thank you very much for a quick fix!