String splitting to Arrays

BlitzMax Forums/BlitzMax Programming/String splitting to Arrays

klepto2

(Posted 2005) [#1]

In my module I have got 2 methods for splitting Strings and send them to an Array. I have made 2 functions out of them to share them with you.

The first one called Return_strip devides the string as often it finds the delivered ASCII code in the String.

The second one called Return_strip2 devides the string as long as the delivered _strip:string is. i.e: the stripstring is
" :" the main string will be divided two times.
("Load Map:Test.dat" will result in 1. "load" and 2. "Map:Test.dat" with the first function but with the second one you will get : 1."Load" 2."Map" 3."Test.Dat")


Strict

Local I:Int



Print "Return_strip2 : ('Load Map:Test.dat')"

Local Strip:String[]
Strip = Return_Strip2("Load Map:Test.dat"," :")
For I = 0 To Strip.length-1
Print I + " : " + Strip[I]
Next

Print "Return_strip  : ('Load Map:Test.dat')"

Local Strip2:String[]
Strip2 = Return_Strip("Load Map:Test.dat",32)
For I = 0 To Strip2.length-1
Print I + " : " + Strip2[I]
Next




Function Return_Strip2:String[](_String:String,_strip:String)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Mid(_strip,I+1,1))
			If sp_p = - 1 Or I > _strip.length -1 Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Perturbatio

(Posted 2005) [#2]

I added something similar to the code archives except it returns a list rather than an array.

klepto2

(Posted 2005) [#3]

Well, I haven't looked in the Codearchives for a long time.
And in an other topic someone searched for this.

Yours is nice, I wished I had seen it before.

Perturbatio

(Posted 2005) [#4]

As long as yours works it doesn't really matter.

Hmm... it might be interesting to speed test them to find out which is faster.

klepto2

(Posted 2005) [#5]

Would be nice but it seems both needs 0 millisecs.

Tested with this code:


Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Time2 = MilliSecs()
time3 = time2 - time

Print "needed time: " + time3

Rem
For I = 0 To Strip2.length-1
Print I + " : " + Strip2[I]
Next
End Rem

Print "Pertubatios :"

time = MilliSecs()
Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Time2 = MilliSecs() 
time3 = time2 - time

Print "needed time: " + time3 
Rem
If myList Then
	For a$ = EachIn myList
		Print a$
	Next
EndIf
End Rem

Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()
time3 = time2 - time
Print "needed time: " + time3




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function

Results :

1: 0
2: 0
3: 14 (last one was a test if my test is correct)

Perturbatio

(Posted 2005) [#6]

well, a better test would be to do it a thousand times:

Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2



Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function

This shows that mine is slower (which is what I suspected).
It is sometimes however handy to have the list functionality.

*EDIT*
Mine can be sped up slightly by not creating the list each loop (i.e. create it outside the loop and the assign to it inside).

Perturbatio

(Posted 2005) [#7]

Actually, this is weird, mine performs better if you run it first:



Local I:Int
Global time:Int
Global time2:Int
Global time3:Int


Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2






Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function

*EDIT*
solved it:



Local I:Int
Local time:Int
Local time2:Int


Print "Pertubatios :"

time = MilliSecs()
'Local myList:TList
For Local b:Int = 1 To 1000
	Local myList:TList = SplitString("This is a longer test string that I am using to test this split string test thing", " ")
	myList= Null
	FlushMem
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2


Print "Return_strip  : "
Local Strip2:String[]
time = MilliSecs()
For Local a:Int = 1 To 1000
	Strip2 = Return_Strip("This is a longer test string that I am using to test this split string test thing",32)
	strip2 = Null
	FlushMem
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2






Print "Testloop : "
time = MilliSecs()
For i = 1 To 10000000
Next
Time2 = MilliSecs()-time
Print "needed time: " + time2




Function Return_Strip:String[](_String:String,_strip:Int)

	Local Text_Array : String[1]
	Local R_Text:String = _string
	Local i:Int = 0

	Repeat
		If R_Text.Length = 0 Then Exit
			Local sp_p:Int = R_Text.Find(Chr(_strip))
			If sp_p = - 1  Then
				Text_Array[I] = R_Text
				Exit
			End If
			Text_Array[I] = Left(R_Text,sp_p)
			R_Text = Right(R_Text,(R_text.Length - sp_p)-1)
			I:+1
			Text_Array = Text_array[..I+1]
	Forever
	
	Return Text_array	
	
End Function

Function SplitString:TList(inString:String, Delim:String)
	Local tempList : TList = New TList
	Local currentChar : String = ""
	Local count : Int = 0
	Local TokenStart : Int = 0
	
	If Len(Delim)<>1 Then Return Null
	
	inString = Trim(inString)
	
	For count = 0 Until Len(inString)
		If inString[count..count+1] = delim Then
			tempList.AddLast(inString[TokenStart..Count])
			TokenStart = count + 1
		End If
	Next
	tempList.AddLast(inString[TokenStart..Count])	
	Return tempList
End Function

Flushmem is your friend.

I get:
Pertubatios :
needed time: 25
Return_strip :
needed time: 15
Testloop :
needed time: 43
in Debug Mode

and:
Pertubatios :
needed time: 18
Return_strip :
needed time: 14
Testloop :
needed time: 11
in Release.

Yours is still faster, but it makes me feel better that mine isn't quite as horrendously slow as I thought.

Jay Kyburz

(Posted 2005) [#8]

mind if i ask why you start your vars with underscore, is this a basic convention?

klepto2

(Posted 2005) [#9]

@Hakea :If you mean '_' this with underscore then I actually use it because I have seen somewhere else and it looks much better for me and it helps me not to loose the overview in greater projects.

@Pertubatio : Yeah that flushmem() works wonders ;)