Savetext format(s)

BlitzMax Forums/BlitzMax Beginners Area/Savetext format(s)

Pierrou(Posted 2011) [#1]
Hi,

I'm currently trying to finish a reading game for French-speaking speech & language therapists. It allows the user to read text files, modify them and then save them again (the original text files are put in an array, character by character, first). Of course the texts are written in French, so they are full of diactritics and other weird characters (ô, ù, é...)

On Windows, it all works fine, I have no problem writing text files using the Savetext command. On MacOS, the Savetext command automatically uses Latin1 format, unless any of the character has a character code greater than 255 => then it switches to UTF16 (so does it on Windows). Latin1 isn't easily displayed on Mac, although Word or OpenOffice can do it...

Is there a *simple * way to automatically save the text in UTF16 format?

I was thinking of
1/ saving the text the usual way using Savetext
2/ opening the newly created textstream and loading its content somewhere
3/ adding, if they're not already there, the big endian or little endian UTF16 bytes in front of anything else
4/writing the stream again

(EDIT : or maybe I should not use savetext and directly write a textstream in UTF16 format, more simply?)

But there must be a better solution. If you know it please tell me.

Thanks a lot in advance for your help,

Last edited 2011


AdamRedwoods(Posted 2011) [#2]
You may have to go to the CString way...

see also:
http://blitzmax.com/bmdocs/command.php?name=SaveText&ref=2d_cat


Pierrou(Posted 2011) [#3]
I should be able to find something that works on both systems. Thank you for your answer!


Kryzon(Posted 2011) [#4]
Check the source code under BlitzMax\mods\BRL.mod\TextStream.mod\TextStream.BMX.

The mapping function SaveText() at the end of the code does the checking mentioned in the SaveText documentation.
You could REM'ark this checking and hard-code this:
format = TTextStream.UTF16LE

'or 

format = TTextStream.UTF16BE

to force it to always use UTF-16. Then build that module with BMK and be done with it.
Later on uncomment and restore the default.

Last edited 2011


Zeke(Posted 2011) [#5]
or just copy/paste SaveText() function to your project, and modify function:

Function SaveText( str$,url:Object )

	Local format

?BigEndian
	format=TTextStream.UTF16BE
?LittleEndian
	format=TTextStream.UTF16LE
?

	Local stream:TStream=WriteStream( url )
...
...
...
...

this way you dont need to change any module.


Kryzon(Posted 2011) [#6]
D'oh! nice going :)


Pierrou(Posted 2011) [#7]
Good idea, I have no time to try it right now but I'm going to do it... Thanks a lot!


Pierrou(Posted 2011) [#8]
Kryzon, Zeke, thank you again : changes only took a few seconds and ... now it works!!


Pierrou(Posted 2011) [#9]
Hello again.

I'm still currently trying to finish a reading game for French-speaking speech & language therapists. It allows the user to read text files, modify them and then save them again (the original text files are put in an array, character by character, first). Of course the texts are written in French, so they are full of diactritics and other weird characters (ô, ù, é...). It runs on Windows and MacOS.

I decided to put the whole text into a string and use the original Savetext command on Windows . On MacOS, I'm using the modified version suggested by Zeke just above.

I get Latin1 text files on Windows, and UTF16BE/LE on MacOS, which is exactly what I want.

On my Win7 laptop, the text files can be recognized and read without any problem by the notepad or OpenOffice 3.3

But on my XP eeePc OpenOffice 3.3 asks for the text format before opening it... It's almost the same on the Mac : TextEdit can read the file but OpenOffice 3.3 and Word need me to tell them the text format...

Do you know why? And most of all, is there a way to add a few bytes or something to the text files so that they open without needing the user to do anything?

Thanks in advance,