Modifiers to Random numbers to help AI learn..

Blitz3D Forums/Blitz3D Beginners Area/Modifiers to Random numbers to help AI learn..

psychicbottle

(Posted 2013) [#1]

Ok so I am coding a program for a proof of concept kinda, beside the point.

In this program I need my "player" to randomly choose a function to "run" for example he could randomly use MOVE and he would randomly move BUT say I wanted move to be picked more often, then how would I do that?

Yasha

(Posted 2013) [#2]

Since you're selecting integers from a short range, there is a very simple way to do this that doesn't involve complex weighting algorithms:

1. Normalise your weights so that they are all whole numbers (or better, only work with integer weights, and keep them as small as possible)

2. Sum the weights for each option (so if A has weight 1, B has weight 2 and C has weight 4, the total we need is 7)

3. Use this number N to allocate a bank or array of this many slots

4. Place each option into as many array slots as match its weighting (A goes in the first slot, B is put in both the second and third slots, C goes in the remaining four slots)

5. Generate a random integer X between 0 and N-1

6. Return the option at array[X] as your weighted-rand result!

As you can see, this will rapidly become inefficient if you have a very large number of options, a very large difference in weighting requiring massive duplication, or need to generate floating-point results along a smooth curve. But it's a start and sounds like it should be sufficient for your purposes.

psychicbottle

(Posted 2013) [#3]

ooooh ok I see!, clever! I think I understand fully but I will need to try and make a example for myself just to make sure, thank you for the fast response and help!

psychicbottle

(Posted 2013) [#4]

hmmm I am having far more trouble with this than I thought... any sort of example you could give me in code? not the entire thing, I'm not here to mooch just something to help me understand a bit more..

K	(Posted 2013) [#5]

Text diagram time!

ARRAY_SIZE=7
CONTENT  |A|B|B|C|C|C|C|
SLOT_#    0-1-2-3-4-5-6

Or... (A becomes 1 etc.)

Dim S%(7)
S(0)=1
S(1)=2
S(2)=2
S(3)=3
S(4)=3
S(5)=3
S(6)=3
Chosen_Action=S( Rand(0,6) )

Different AI's could load different templates from files to banks using ReadBytes(). Become comfortable with Arrays before using Banks extensively.

To integrate, use a 1-D 'Blitz Array' rather than a Dim() one...

Type Enemy
Field mesh%
Field Health%,typ%
Field Speed#
Field AI[10];Assuming you only weight 10 slots.
End Type

psychicbottle

(Posted 2013) [#6]

but wait how could I dynamically add weights in the program while it's running?

K	(Posted 2013) [#7]

Aha.. good query.

Dim S%(7)
S(0)=1
S(1)=2
S(2)=2
S(3)=3
S(4)=3
S(5)=3
;Assuming 5 slots.
Target=2;We want to add a 2 at the expense of 3 in this example.
Repeat
If (t=0)And(S(k)=Target)
If (t=1)And(S(k)<>Target)t=2
k=k+1
Until t=2
S(k)=Target

This code changes slot 3 toa value of 2. To do at the expense of the value below, you need a different routine to count backwards. They could beturned into Functions accepting B_Arrays as input...

Function AddUp(Array[5],Target)
Repeat
If (t=0)And(S(k)=Target)
If (t=1)And(S(k)<>Target)t=2
k=k+1
If k>4and t<>2 t=-1 : Exit
Until t=2
If t<>-1 S(k)=Target
End Function

Function AddDn(Array[5],Target)
k=5
Repeat
If (t=0)And(S(k)=Target)
If (t=1)And(S(k)<>Target)t=2
k=k-1
If k<0and t<>2 t=-1 : Exit
Until t=2
If t<>-1 S(k)=Target
End Function

psychicbottle

(Posted 2013) [#8]

wait sorry I am not good with Arrays so I might be missing something but what is the variable "t" represent...? and K I suppose :P

K	(Posted 2013) [#9]

K is the slot number... call it what you want. t signals several possibilities.

-1>>Target is not in array
+0>>Not found target.
+1>>Found target area
+2>>Found end of target area; k now pointed at area+/- 1

Sorry, inadequate explaining.

Kryzon

(Posted 2013) [#10]

Remember you can also use Banks to perform what Yasha said. Banks have the added benefit of simpler handles. You access banks with integer handles, so you can store these handles in types, send them to functions etc. and you don't need to have a defined size for them.

That is, you don't need to set a size limit; you can query the bank size at the moment you compute the random number.

;Press any key to draw 10 random weighted values.

Graphics 400,300,0,2
SeedRnd MilliSecs()

Local myValues% = CreateBank()

;Add weighted values.
AddWeightedValue(myValues, 1, 10) 
AddWeightedValue(myValues, 2, 5)
AddWeightedValue(myValues, 3, 2)

Local tempValue%
While Not KeyHit(1)
	Cls
	For i = 1 To 10
		;Retrieve a random, weighted value. Value '1' should come more often, followed by '2' and then '3'.
		tempValue = GetWeightedValue(myValues)

		Color 255-(tempValue-3)*126,255-(tempValue-2)*126,255-(tempValue-1)*126
		Locate 170+(tempValue-1)*20,70+(i-1)*15
		Print tempValue
	Next
	WaitKey()
Wend


Function AddWeightedValue(bank%, value%, weight%=1)
	Local oldSize% = BankSize(bank)

	;Bank resizing is handled inside functions, so you never need to know the actual size of any bank.
	ResizeBank bank, (oldSize+(weight*4))
	
	If weight > 0 Then
		For index = 0 To weight-1
			PokeInt bank,oldSize+(index*4),value
		Next
	EndIf
End Function


Function resetBank(bank%)
	ResizeBank bank,0
End Function


Function GetWeightedValue(bank%)
	Local randomSeek = Rand(0, (BankSize(bank)-4)/4)
	Return ( PeekInt(bank, randomSeek*4) )
End Function


End

Stevie G

(Posted 2013) [#11]

You could avoid arrays completely by storing your possible states in a string, like so :

[bbcode]
Global W$

W = WEIGHTcreate( 5,8,2 )

Print W
For l = 1 To 10
Print WEIGHTgetRandom( W )
Next

W = WEIGHTcreate( 2,2,2 )
Print W
For l = 1 To 10
Print WEIGHTgetRandom( W )
Next

MouseWait
End

Function WEIGHTgetRandom$( Txt$ )

Local L = Len( Txt )

Return Mid( Txt, Rand(1,L),1 )

End Function

Function WEIGHTcreate$( run, move, shoot )

Local Weight$

Weight = Weight + String( "R", run )
Weight = Weight + String( "M", move )
Weight = Weight + String( "S", shoot )

Return Weight

End Function
[/bbcode]

The string can be easily recreated whenever required.

dynaman

(Posted 2013) [#12]

Have more of the numbers be for "move", if the three options are Move, Run, and stand still then.

Pick a number between 1 and 100
1-50 is move.
51 - 75 is run
76 - 100 is walk.

move is twice as likely to be choosen. Easy to understand and change on the fly if needed.

psychicbottle

(Posted 2013) [#13]

Ok, I have basically just set up the needed things (I think) to allocate Data about the robot sooo here you can do anything cool with it then please share with all of us (I am verrry stuck on my next step haha)

Graphics 800,600,32,2
Type robot
Field X#,Y#,Image,Sens.Sensor,child$,Bank$
End Type 

Type Sensor
Field X#,Y#,Image,parent
End Type

Type Objective
Field X#,Y#,Image
End Type

For i = 1 To 1
R.Robot = New Robot 
R\X# = GraphicsWidth()/2
R\Y# = GraphicsHeight()/2
R\Child$ = "Main"
R\Bank$ = WEIGHTcreate$(20,3,4)
Next

O.Objective = New Objective
O\X# = Rnd(20,GraphicsWidth)
O\Y# = Rnd(20,GraphicsHeight)


SetBuffer BackBuffer()

circle()

While Not KeyHit(1)

	Cls()
			UpdateRobots()
			UpdateObjective()

	Flip

Wend

Function UpdateObjective()
Color 255,0,0
For O.Objective = Each Objective
	Rect O\X#,O\Y#,10,10,0
Next
End Function


Function UpdateRobots()
Color 100,255,100
For R.Robot = Each Robot
	Rect R\X-40,R\Y-40,80,80,1
	UpdateSensors(R\X#,R\Y#)
	PREMoveRobot()
Next
End Function

Function PREMoveRobot()
	X# = Rnd(-5,5)
	Y# = Rnd(-5,5)
	MoveRobot(X#,Y#)
End Function
Function MoveRobot(X#,Y#)
For R.Robot = Each Robot
R\X# = R\X#+X#
R\Y# = R\Y#+Y#
Next
End Function


Function UpdateSensors(X#,Y#)
Color 255,255,255
For Sens.Sensor = Each Sensor
	Rect Sens\X#+X#,Sens\Y#+Y#,30,30,1
Next
End Function

Function CreateSensor.Sensor(X#,Y#,Sx,Sy) 
For t = 1 To 1
newS.Sensor = New Sensor
	newS\X# = Sx 
	newS\Y# = Sy 
	DebugLog "HAH: " + newS\X# + " , " + newS\Y#
Next
End Function 

Function Circle()
D = 360/8
D2 = 1
	For degrees=1 To 360; Step though all the degrees in a circle (360 in all)
	

; The next line calculates the Y coordinate of the point of the circle using the Sin
; command, and multiplies it by 100 (to get a larger radius, try and change it).
Sy=Sin(degrees)*100

Sy=-Sy ; Invert Y coordinate to represent it properly on screen.

; The next line calculates the X coordinate of the point of the circle using the Cos,
; command, and multiplies it by 100 (to get a larger radius, try and change it).
Sx=Cos(degrees)*100

If D = degrees
DebugLog "WHHHAAATTT"
For i = 1 To 1
For R.Robot = Each Robot 

If R\Child = "Main"

R\X# = GraphicsWidth()/2
R\Y# = GraphicsHeight()/2
R\Sens = CreateSensor(R\X,R\Y,Sx-15,Sy-15)
EndIf
Next
Next
D2 = D2 + 1
D = D2 * 45

EndIf

Next

End Function

Function WEIGHTgetRandom$( Txt$ )

	Local L = Len( Txt )
	
	Return Mid( Txt, Rand(1,L),1 )
	
End Function

Function WEIGHTcreate$( run, move, shoot )

	Local Weight$
	
	Weight = Weight + String( "R", run )
	Weight = Weight + String( "M", move )
	Weight = Weight + String( "S", shoot )

	Return Weight
	
End Function

EDIT: right now all it does is make the "random" choice to move in a random direction at a random speed, I am trying to make it decide what "Action" made it move closer to the little red box on the screen and then make that choice less "random"