Docmods Summaries Tweak

BlitzMax Forums/BlitzMax Module Tweaks/Docmods Summaries Tweak

Brucey

(Posted 2006) [#1]

The problem I have with docmods documentation is that if you are looking for something and can't remember what it's called, it can take ages scrolling through the page looking for it.

This tweak enhances docmods to provide "Summaries", as well as a couple of bug fixes ;-)

Fixes :
* Functions split over multiple lines (by using ..) are now shown correctly.
* Methods, Fields or Functions documented inside a Type which isn't documented, are now no longer listed against the Module, but rather the Type to which they belong.

New :
* Summary sets provided for module and types. Makes finding what you want much easier. Globals, consts and fields are shown in a comma separated list. Types, methods and functions are shown with their "short" text description. (Blitz Runtime docs are a nice example showing the enhanced documentation). Types also have their own sets of summaries for fields, consts, globals, methods and functions.
* Keywords shortcut added to top of doc when appropriate.

Notes :
* Still only documents bbdoc'd code. ie. Fields of a type that aren't bbdoc'd are not displayed. (Since some fields are deemed private and bmax currently doesnt allow for the distinction between private and public inside types, it seems silly to show *everything*, less you attempt to utilise code you shouldn't!! ;-)

docmods.bmx


Strict

Framework brl.basic

Import "docparser.bmx"

Local docs:TDocs=ParseMods()

docs.Sort

If AppArgs.length=2 And AppArgs[1].ToLower()="sync"
	SyncDocs docs
Else
	DocMods docs
	SyncDocs docs
EndIf

Function ParseMods:TDocs()

	Local docs:TDocs=New TDocs
	
	Local mods:TList=EnumModules()
	
	For Local modid$=EachIn mods
			
		Local ident$=ModuleIdent( modid )
		Local modDir$=ModulePath( modid )
		Local bmxFile$=ModuleSource( modid )
		Local docDir$=modDir+"/doc"
		
		If FileType( modDir+"/"+ident+".bmx" )=FILETYPE_FILE
			If FileType( docDir )<>FILETYPE_DIR
				CreateDir docDir
				If FileType( docDir )<>FILETYPE_DIR
					Print "Failed to created directory:"+docDir
					Continue
				EndIf
			EndIf

			Local toker:TBMXToker=TBMXToker.Create( bmxFile )

			Local parser:TDocParser=TDocParser.WithToker( toker )
			
			parser.Parse docs
	
			parser.Close

		EndIf
		
	Next

	Return docs
	
End Function

Function DocMods( docs:TDocs )

	For Local t:TDocs=EachIn docs.kids
	
		If t.kind<>T_MODULE Continue
		
		ChangeDir ModulePath( t.ident.tolower() )+"/doc"	'linux fix for "BRL." case problem

		Local stdio:TStream=StandardIOStream
		StandardIOStream=WriteFile( "commands.html" )

		t.CacheTexts ' pre cache of text for output (allows us to use it for summaries etc) - BaH 03/09/2006
		t.EmitHtml

		StandardIOStream.Close
		StandardIOStream=stdio
	
	Next
End Function

Function SyncDocs( docs:TDocs )

	Local comms:TList=New TList
	Local index:TList=New TList
	
	Local stdio:TStream=StandardIOStream
	StandardIOStream=WriteFile( BlitzMaxPath()+"/doc/bmxmods/navbar.html" )
	
	Print "<html><head>"
	Print "<link rel=stylesheet type=text/css href='../bmxstyle.css'>"
	Print "<script>function toggle(a){if(a.display!='block')a.display='block';else a.display='none';}</script>"
	Print "</head><body class=navbar>"
	
	Print "<b>Module reference</b><br>"
	
	For Local t:TDocs=EachIn docs.kids

		If t.kind<>T_MODULE Continue
		
		Local modid$=t.ident.ToLower()
		Local modln$=modid.Replace(".","_")
		Local moddesc$=t.bbdoc[6..].Trim()
		Local i=moddesc.Find("~n")
		If i<>-1 moddesc=moddesc[..i]
		
		Local url$="../../mod/"+modid.Replace(".",".mod/")+".mod/doc/commands.html"
		
		Print "<a onClick=toggle("+modln+".style) class=navbig href="+url+" target=main>"+moddesc+"</a><br>"
		Print "<div id="+modln+" class=entries>"
		For Local p:TDocs=EachIn t.kids
			' kind is Type and is not doc'd and there are no doc'd kids
			If p.kind = T_TYPE And (Not p.bbdoc And p.kids.count() = 0) Then
				Continue
			End If
			Local turl$=url+"#"+p.ident
			Print "&nbsp;<a class=navsmall href="+turl+" target=main>"+p.ident+"</a><br>"
			index.AddLast p.ident+":"+turl
			Local i=p.proto.Find( " " )
			If i<>-1 comms.AddLast p.proto[i+1..].Trim()+"|"+turl[5..]
		Next
		Print "</div>"

	Next
	
	Print "<br>"
	Print "<b>Alphabetic index</b><br>"

	Local arr:Object[]=index.ToArray()
	arr.Sort
	For Local link$=EachIn arr
		Local i=link.Find( ":" )
		If i=-1 Throw "chunks"
		Local ident$=link[..i],url$=link[i+1..]
		Print "<a class=navsmall href="+url+" target=main>"+ident+"</a><br>"
	Next
	
	Print "</body></html>"
	
	StandardIOStream.Close
	StandardIOStream=WriteStream( BlitzMaxPath()+"/doc/bmxmods/commands.txt" )
	For Local t$=EachIn comms
		Print t
	Next
	StandardIOStream.Close
	StandardIOStream=stdio

End Function

bmxtoker.bmx

Strict

Import BRL.Map
Import BRL.Stream

'----- Simple BMX Parser -----

Const T_EOF=		-1
Const T_EOL=		10

Const T_IDENT=		$10001
Const T_INTLIT=		$10002
Const T_FLOATLIT=	$10003
Const T_STRINGLIT=	$10004
Const T_STRINGERR=	$10005

Const T_REM=		$20001
Const T_ENDREM=		$20002
Const T_FUNCTION=	$20003
Const T_ENDFUNCTION=$20004
Const T_TYPE=		$20005
Const T_ENDTYPE=	$20006
Const T_CONST=		$20007
Const T_METHOD=		$20008
Const T_ENDMETHOD=	$20009
Const T_GLOBAL=		$2000a
Const T_INCLUDE=		$2000b ' Added by BaH - 01/06/2006
Const T_IMPORT=		$2000d
Const T_FIELD=		$2000e
Const T_EXTENDS=	$2000f
Const T_ABSTRACT=	$20010
Const T_FINAL=		$20011
Const T_MODULE=		$2000c
Const T_MODULEINFO=	$20012
Const T_EXTERN=	$20013 ' Added by BaH - 03/09/2006
Const T_ENDEXTERN=	$20014 ' Added by BaH - 03/09/2006

Const T_DOTDOT=		$30001
Const T_ARRAYDECL=	$30002

Const T_KEYWORD=	$40000

Type TIntValue
	Field value:Int
End Type

Function IntValue:TIntValue( value )
	Local t:TIntValue=New TIntValue
	t.value=value
	Return t
End Function

Function IsSpace( char )
	Return char<=Asc(" ") And char<>Asc("~n")
End Function

Function IsAlpha( char )
	Return ( char>=Asc("a") And char<=Asc("z") ) Or ( char>=Asc("A") And char<=Asc("Z") )
End Function

Function IsNumeric( char )
	Return char>=Asc("0") And char<=Asc("9")
End Function

Function IsAlphaNumeric( char )
	Return IsAlpha( char ) Or IsNumeric( char )
End Function

Type TBmxToker

	Field _filename:String ' Added by BaH - 25/05/2006

	Field _spc
	Field _pos,_line$
	Field _toke,_text$
	Field _stream:TStream
	
	Global _keywords:TMap
	
	Method Delete()
		Close
	End Method

	Method Bump()
	
		If _toke=T_EOF Return _toke
	
		If _pos>=_line.length
			_pos=0
			If _stream.Eof()
				_toke=T_EOF
				_text=""
				Return
			EndIf
			_line=_stream.ReadLine().Trim()+"~n"
		EndIf
		
		Local from=_pos
		While _pos<_line.length And IsSpace( _line[_pos] )
			_pos:+1
		Wend
		
		If _spc And _pos<>from
			_text=_line[from.._pos]
			_toke=Asc(" ")
			Return _toke
		EndIf
		
		from=_pos
		Local char=_line[_pos]
		_pos:+1
		
		If char=Asc("'")
			_pos=_line.length
			_toke=T_EOL
		Else If char=Asc("~n")
			_toke=T_EOL
		Else If isAlpha(char) Or char=Asc("_")
			While IsAlphaNumeric(_line[_pos]) Or _line[_pos]=Asc("_")
				_pos:+1
			Wend
			_toke=T_IDENT
			Local id$=_line[from.._pos].ToLower()
			If id="end" And _line[_pos]=Asc(" ") And IsAlpha(_line[_pos+1])
				Local t_pos=_pos+2
				While IsAlphaNumeric(_line[t_pos]) Or _line[t_pos]=Asc("_")
					t_pos:+1
				Wend
				Local id$="end"+_line[_pos+1..t_pos].ToLower()
				Local v:TIntValue=TIntValue( _keywords.ValueForKey( id ) )
				If v
					_pos=t_pos
					_toke=v.value
				EndIf
			EndIf
			If _toke=T_IDENT
				Local v:TIntValue=TIntValue( _keywords.ValueForKey( id ) )
				If v _toke=v.value
				If _toke=T_REM
					_text=""
					Repeat
						If _stream.Eof()
							_pos=0
							_line=""
							Return _toke
						EndIf
						_line=_stream.ReadLine().Trim()+"~n"
						If _line[..6].ToLower()="endrem" Or _line[..7].ToLower()="end rem" Exit
						_text:+_line
					Forever
					_pos=_line.length
					Return _toke
				Else
					' Complete lines if they continue onto other lines - BaH 03/09/2006
					If _toke = T_FUNCTION Or _toke = T_METHOD Or _toke = T_CONST Or _toke = T_GLOBAL Or _toke = T_FIELD Then
						If _line.find("..") >= 0 Then
							_line = getFullLine()
						End If
					End If
				EndIf
			EndIf
		Else If IsNumeric(char)
			While IsNumeric(_line[_pos])
				_pos:+1
			Wend
			_toke=T_INTLIT
			If _line[_pos]=Asc(".")
				_pos:+1
				While IsNumeric(_line[_pos])
					_pos:+1
				Wend
				_toke=T_FLOATLIT
			EndIf
		Else If char=Asc("~q")
			While _line[_pos]<>Asc("~q") And _line[_pos]<>Asc("~n")
				_pos:+1
			Wend
			If _line[_pos]=Asc("~q")
				_pos:+1
				_toke=T_STRINGLIT
			Else
				_toke=T_STRINGERR
			EndIf
		Else
			_toke=char
		EndIf
		
		_text=_line[from.._pos]
		
		Return _toke
		
	End Method
	
	' Completes a line that takes up more than one actual line - BaH 03/09/2006
	Method getFullLine:String()
		Local first:Int = True
		Local fullline:String
		Local line:String = _line
		Local pos:Int = 0
		Local from:Int
		
		#loop
		Repeat
		
			If Not first Then
				If _stream.Eof()
					pos=0
					line=""
					Return line
				EndIf
				line = _stream.ReadLine().Trim()+"~n"
			End If
			
			first = False

			If line.tolower().Trim() = "rem" Then
				Repeat
					If _stream.Eof()
						pos=0
						line=""
						Return fullline
					EndIf
					line = _stream.ReadLine().Trim()+"~n"
					If line[..6].ToLower()="endrem" Or line[..7].ToLower()="end rem" Then
						Continue loop
					End If
				Forever
			End If

			pos = 0
			from = pos
			While pos < line.length And IsSpace( line[pos] )
				pos:+1
			Wend
			
			from = pos

			Local char:Int = line[pos]
			pos:+1
			While char <> Asc("~n")

				If char = Asc(".") And line[pos] = Asc(".") Then
					fullline:+ line[from..pos-1]
					Exit
				End If
				
				If char = Asc("'") Then
					Exit
				End If
				
				char=line[pos]
				pos:+1 
			Wend
			
			If char = Asc("~n") Then
				fullline :+ line[from..pos-1]
				Exit
			End If

		Forever

		Return fullline
	End Method
	
	Method Curr()
		Return _toke
	End Method
	
	Method Text$()
		Return _text
	End Method
	
	Method Line$()
		Return _line
	End Method
	
	Method Parse$( toke )
		If Curr()<>toke Throw "Unexpected token"
		Local t$=Text()
		Bump
		Return t
	End Method
	
	Method CParse$( toke )
		If Curr()<>toke Return
		Local t$=Text()
		Bump
		Return t
	End Method
	
	Method ParseUntil$( toke )
		Local t$
		While Curr()<>toke
			If Curr()=T_EOF Throw "Unexpected EOF"
			t:+Text()
			Bump
		Wend
		Bump
		Return t
	End Method
	
	Method Spaces( enable )
		If enable
			_spc:+1
		Else
			_spc:-1
		EndIf
	End Method
	
	Method Close()
		If _stream _stream.Close
		_stream=Null
	End Method
	
	Function CreateKeywords()
		If _keywords Return
		Function kw( id$,toke )
			_keywords.insert id,IntValue(toke)
		End Function
		_keywords=New TMap
		kw "rem",T_REM
		kw "endrem",T_ENDREM
		kw "function",T_FUNCTION
		kw "endfunction",T_ENDFUNCTION
		kw "method",T_METHOD
		kw "endmethod",T_ENDMETHOD
		kw "const",T_CONST
		kw "global",T_GLOBAL
		kw "include",T_INCLUDE ' Added by BaH - 01/06/2006
		kw "import",T_IMPORT
		kw "field",T_FIELD
		kw "type",T_TYPE
		kw "endtype",T_ENDTYPE
		kw "extends",T_EXTENDS
		kw "abstract",T_ABSTRACT
		kw "final",T_FINAL
		kw "module",T_MODULE
		kw "moduleinfo",T_MODULEINFO
		kw "extern",T_EXTERN ' Added by BaH - 03/09/2006
		kw "endextern",T_ENDEXTERN ' Added by BaH - 03/09/2006
	End Function	
	
	Function Create:TBmxToker( url:Object )
		CreateKeywords
		Local stream:TStream=ReadStream( url )
		If Not stream Throw "Unable to read stream: "+url.ToString()
		Local t:TBmxToker=New TBmxToker
		t._filename = url.ToString() ' Added by BaH - 25/05/2006
		t._stream=stream
		t.Bump
		Return t
	End Function

End Type

docparser.bmx

Strict

Import BRL.StandardIO
Import BRL.MaxUtil

Import "bmxtoker.bmx"

Function FindIdent( pre$,text$,i Var,e Var )

	i=text.Find( pre,i )
	
	If i=-1 Return False
	
	If i>0 And text[i-1]>32 Return False
	
	e=i+1
	While e<text.length
		Local c=text[e]
		If c=Asc(".") And e+1<text.length And IsAlpha( text[e+1] )
			e:+2
			Continue
		EndIf
		If c<>Asc("_") And Not IsAlphaNumeric(c) Exit
		e:+1
	Wend
	Return True
End Function

Function FormatText( t$ Var )

	If t.length And IsAlphaNumeric( t[t.length-1] ) t:+"."

	Local i,e

	i=0
	While FindIdent( "#",t,i,e )
	
		While e<t.length-1 And t[e]=Asc(".") And IsAlpha( t[e+1] )
			e:+2
			While e<t.length And IsAlpha(t[e])
				e:+1
			Wend
		Wend
		
		Local id$=t[i+1..e],q=id.Find("."),ln$

		If q=-1
			ln="<a href=#"+id+">"+id+"</a>"
		Else
			ln="<a href=../../../"
			Repeat
				ln:+id[..q]+".mod/"
				id=id[q+1..]
				q=id.Find(".")
			Until q=-1
			ln:+"doc/commands.html#"+id+">"+id+"</a>"
		EndIf
			
		t=t[..i]+ln+t[e..]
		i:+ln.length
	Wend

	i=0
	While FindIdent( "@",t,i,e )
		Local p$="<b>"+t[i+1..e]+"</b>"
		t=t[..i]+p+t[e..]
		i:+p.length
	Wend

End Function

Type TDocs

	Field kind		'T_CONST, T_GLOBAL etc
	Field ident$	'identifier
	Field proto$	'Global/Function etc text
	Field bbdoc$	'rem/endrem bbdoc: text
	Field infos:TList=New TList	'ModuleInfo's
	Field imports:TList = New TList ' Added by BaH - 25/05/2006
	Field parent:TDocs ' Added by BaH - 25/05/2006
	Field textCache:TTextCache
	
	Field kids:TList=New TList
	
	Method Sort()
		kids.Sort
		For Local t:TDocs=EachIn kids
			t.Sort
		Next
	End Method
	
	' Added by BaH - 25/05/2006
	' Checks for extra references to the same imports...
	Method containsImport:Int(_file:String)
		If imports.contains(_file) Then
			Return True
		End If
		If parent <> Null Then
			Return parent.containsImport(_file)
		End If
		
		Return False
	End Method
	
	Method Compare( with:Object )
		Local t:TDocs=TDocs(with)

		If kind=t.kind
			If kind=T_MODULE 
				Return bbdoc.ToLower().Compare( t.bbdoc.ToLower() )
			EndIf
			Return ident.ToLower().Compare( t.ident.ToLower() )
		EndIf

		If kind=T_MODULE Return -1
		If t.kind=T_MODULE Return 1
		
		If kind=T_CONST Return -1
		If t.kind=T_CONST Return 1

		If kind=T_GLOBAL Return -1
		If t.kind=T_GLOBAL Return 1
		
		If kind=T_FIELD Return -1
		If t.kind=T_FIELD Return 1

		If kind=T_METHOD Return -1
		If t.kind=T_METHOD Return 1

		If kind=T_FUNCTION Return -1
		If t.kind=T_FUNCTION Return 1
		
		If kind=T_TYPE Return -1
		If t.kind=T_TYPE Return 1
		
		If kind=T_EOL Return -1
		If t.kind=T_EOL Return 1
		
		Throw "OOps!"
		
	End Method
	
	Method CacheTexts()
		If textCache = Null Then
			textCache = TTextCache.Create(bbdoc, kind)
		
			If Not kids.IsEmpty()
				Local tkind=0
				For Local doc:TDocs=EachIn kids
					doc.CacheTexts()
				Next
			EndIf

		End If

	End Method
	
	Method EmitHtml$(summary:Int = False)
		Local example$

		Local stream:TStream=ReadStream( ident+".bmx" )
		If stream
			While Not stream.Eof()
				example:+stream.ReadLine()+"~n"
			Wend
			stream.Close
			example=example.Trim()
		EndIf
		
		If Not summary Then
			Select kind
			Case 0
			Case T_MODULE
				Print "<html><head><title>"+textCache.shortdesc+" reference</title>"
				Print "<link rel=stylesheet Type=text/css href='../../../../doc/bmxstyle.css'>"
				Print "</head><body>"
				Local n_consts
				Local n_globals
				Local n_functions
				Local n_types
				Local n_keywords:Int
				For Local t:TDocs=EachIn kids
					Select t.kind
					Case T_CONST n_consts:+1
					Case T_GLOBAL n_globals:+1
					Case T_FUNCTION n_functions:+1
					Case T_TYPE
						' Type is doc'd or there are doc'd kids
						If t.bbdoc Or t.kids.count() > 0 Then
							n_types:+1
						End If
					Case T_EOL n_keywords:+1
					End Select
				Next
				
				Print "<table width=100% cellspacing=0><tr align=center><td class=small>&nbsp;</td>"
				Print "<td class=small width=1%><b>"+ident+":</b></td>"
				If n_consts Print "<td class=small width=1%><a href=#consts class=small>Constants</a></td>"
				If n_globals Print "<td class=small width=1%><a href=#globals class=small>Globals</a></td>"
				If n_functions Print "<td class=small width=1%><a href=#functions class=small>Functions</a></td>"
				If n_types Print "<td class=small width=1%><a href=#types class=small>Types</a></td>"
				If n_keywords Print "<td class=small width=1%><a href=#keywords class=small>Keywords</a></td>"
				
				If Not infos.IsEmpty()
					Print "<td class=small width=1%><a href=#modinfo class=small>Modinfo</a></td>"
				EndIf
				
				Local t$=ModuleSource( ident.ToLower() )
				Local i=t.Find( "/mod/" )
				If i<>-1
					t="../../../.."+t[i..]
					Print "<td class=small width=1%><a href='"+t+"' class=small>Source</a></td>"
				EndIf
				Print "<td class=small>&nbsp;</td></tr></table>"
				
				Local stream:TStream=ReadStream( "intro.bbdoc" )
				
				If stream
					Print "<h1>"+textCache.shortdesc+"</h1>"
					If textCache.longdesc Print textCache.longdesc.Trim()
				Else
					stream=ReadStream( "intro.html" )
					If Not stream
						Print "<h1>"+textCache.shortdesc+"</h1>"
						If textCache.longdesc Print textCache.longdesc.Trim()
					EndIf
				EndIf
				
				If stream
					Local intro$
					While Not stream.Eof()
						intro:+stream.ReadLine()+"~n"
					Wend
					Local i=intro.Find("<body>")
					If i<>-1
						intro=intro[i+6..]
						i=intro.find("</body>")
						If i<>-1 intro=intro[..i]
					EndIf
					stream.close
					intro=intro.Trim()
					FormatText intro
					Print intro
				EndIf
	
				' Show summaries
				If Not kids.IsEmpty()
					Local s:String = Null
					Local tkind=0
					Local count:Int = 0
					For Local doc:TDocs=EachIn kids
						' kind is Type and is not doc'd and there are no doc'd kids
						If doc.kind = T_TYPE And (Not doc.bbdoc And doc.kids.count() = 0) Then
							Continue
						End If
					
						s = "<table class=doc width=100%>"
						
						If kind<>T_TYPE And doc.kind<>tkind
							count = 0
							If tkind <> 0 Then
								If tkind = T_CONST Or tkind = T_GLOBAL Or tkind = T_EOL Then
									Print "</td></tr>"
								End If
								Print "</table>"
							End If
							Select doc.kind
								Case T_CONST Print "<h2><a name=consts></a>Constants Summary</h2>" + s + "<tr><td colspan=2>"
								Case T_GLOBAL Print "<h2><a name=globals></a>Globals Summary</h2>" + s + "<tr><td colspan=2>"
								Case T_FUNCTION Print "<h2><a name=functions></a>Functions Summary</h2>" + s
								Case T_TYPE Print "<h2><a name=types></a>Types Summary</h2>" + s
								Case T_EOL Print "<h2><a name=keywords></a>Keywords Summary</h2>" + s + "<tr><td colspan=2>"
								Default
									Continue
							End Select
							tkind=doc.kind
						EndIf
						count:+ 1
						If count > 1 Then
							If tkind = T_CONST Or tkind = T_GLOBAL Or tkind = T_EOL Then
								Print ", "
							End If
						End If 
						doc.EmitHtml(True)
					Next
					If s Then
						Print "</table>"
					End If
				EndIf
						
			Default
				Print "<table class=doc width=100% cellspacing=3 id="+ident+">"
				Print "<tr><td class=doctop colspan=2>"+proto+"</td></tr>"
				If textCache.returns
					Print "<tr><td class=docleft width=1%>Returns</td><td class=docright>"+textCache.returns+"</td></tr>"
				EndIf
				If textCache.shortdesc
					Print "<tr><td class=docleft width=1%>Description</td><td class=docright>"+textCache.shortdesc+"</td></tr>"
				EndIf
				If textCache.longdesc
					Print "<tr><td class=docleft width=1%>Information</td><td class=docright>"+textCache.longdesc+"</td></tr>"
				EndIf
				If example
					Print "<tr><td class=docleft width=1%><a href="+ident+".bmx class=small>Example</a></td><td class=docright><pre>"+example+"</pre></td></tr>"
				EndIf
				Print "</table>"
			End Select

			If kind = T_TYPE Then
				' Show summaries
				If Not kids.IsEmpty()
					Local s:String = Null
					Local tkind=0
					Local count:Int = 0
					For Local doc:TDocs=EachIn kids
						s = "<table class=doc width=90% align=center>" + ..
							"<tr ><th class=doctop colspan=2 align=left>"
						If doc.kind<>tkind
							count = 0
							If tkind <> 0 Then
								If tkind = T_CONST Or tkind = T_GLOBAL Or tkind = T_EOL Or tkind = T_FIELD Then
									Print "</td></tr>"
								End If
								Print "</table>"
							End If
							Select doc.kind
								Case T_CONST Print s + "<a name=" + ident + "_consts></a>Constants Summary</th></tr><tr><td colspan=2>"
								Case T_FIELD Print s + "<a name=" + ident + "_fields></a>Fields Summary</th></tr><tr><td colspan=2>"
								Case T_GLOBAL Print s + "<a name=" + ident + "_globals></a>Globals Summary</th></tr><tr><td colspan=2>"
								Case T_FUNCTION Print s + "<a name=" + ident + "_functions></a>Functions Summary</th></tr>"
								Case T_METHOD Print s + "<a name=" + ident + "_methods></a>Methods Summary</th></tr>"
								Default
									Continue
							End Select
							tkind=doc.kind
						EndIf
						count:+ 1
						If count > 1 Then
							If tkind = T_CONST Or tkind = T_GLOBAL Or tkind = T_FIELD Then
								Print ", "
							End If
						End If 
						doc.EmitHtml(True)
					Next
					If s Then
						Print "</table>"
					End If
				EndIf
			End If

			If Not kids.IsEmpty()
				Local tkind=0
				For Local doc:TDocs=EachIn kids
					' kind is Type and is not doc'd and there are no doc'd kids
					If doc.kind = T_TYPE And (Not doc.bbdoc And doc.kids.count() = 0) Then
						Continue
					End If

					If kind<>T_TYPE And doc.kind<>tkind
						Print "<h2"
						Select doc.kind
						Case T_CONST Print " id=constsdet>Constants"
						Case T_GLOBAL Print " id=globalsdet>Globals"
						Case T_FUNCTION Print " id=functionsdet>Functions"
						Case T_TYPE Print " id=typesdet>Types"
						Case T_FIELD Print ">Fields"
						Case T_METHOD Print ">Methods"
						Case T_EOL Print ">Keywords"
						End Select
						Print "</h2>"
						tkind=doc.kind
					EndIf
					doc.EmitHtml
					If kind<>T_TYPE Print "<br>"
				Next
			EndIf			
		Else
			Select kind
				Case T_CONST, T_GLOBAL, T_FIELD, T_EOL
					Print "<a href=#" + ident + ">" + ident + "</a>"
				Case T_FUNCTION, T_METHOD, T_TYPE
					Print "<tr><td class=docleft width=1%><a href=#" + ident + ">" + ident + "</a></td><td class=docright>"
					If textCache.shortdesc
						Print textCache.shortdesc
					Else
						Print "&nbsp;"
					EndIf
					Print "</td></tr>"
			End Select
		End If
		

		
		Select kind
		Case T_MODULE
			If Not infos.IsEmpty()
				Print "<h2 id=modinfo>Module Information</h2>"
				Print "<table width=100%>"
				For Local t$=EachIn infos
					t=t[1..t.length-1]
					If Not t Continue
					Local i=t.Find(":")
					If i=-1 Continue
					Local key$=t[..i].Trim()
					Local val$=t[i+1..].Trim()
					Print "<tr><th width=1%>"+key+"</th><td>"+val+"</td></tr>"
				Next
			EndIf
			Print "</body></html>"
		End Select
		
	End Method

End Type

Type TDocParser

	Field toker:TBMXToker
	
	Method Close()
		If Not toker Return
		toker.Close
		toker=Null
	End Method

	' Added by BaH - 25/05/2006
	Method processImport(parent:TDocs, txt:String)
	
		If txt.find(".bmx") > 0 Then
	
			Local importFile:String = ExtractDir(toker._filename) + "/" + txt[1..txt.length-1]
			Local _file:String = StripDir(importFile).toLower()
			
			If Not parent.containsImport(_file) Then
				If FileType(importFile) = 1 Then ' exists !
					parent.imports.AddLast(_file)
					Local _toker:TBMXToker=TBMXToker.Create(importFile)
					Local _parser:TDocParser=TDocParser.WithToker(_toker)
					_parser.Parse parent
					_parser.Close
				End If
			End If
		End If
	
	End Method

	Method Parse( parent:TDocs )
	
		If Not toker Throw "closed"
	
		While toker.Curr()<>T_EOF

			If toker.Curr()=T_ENDTYPE 
				If parent.kind=T_TYPE Return
				toker.Bump
				Continue
			EndIf

			' Added by BaH - 25/05/2006, modified 01/06/2006
			If toker.Curr()=T_IMPORT Or toker.Curr()=T_INCLUDE Then
				If parent.kind=T_MODULE
					If toker.Bump()=T_STRINGLIT
						processImport(parent, Toker.Text())
					End If
				End If
			End If
			
			If toker.Curr()=T_MODULEINFO
				If parent.kind=T_MODULE 
					If toker.Bump()=T_STRINGLIT
						parent.infos.AddLast toker.Text()
					EndIf
				EndIf
				toker.Bump
				Continue
			EndIf
		
			Local skip:Int = False
			Local kind:Int
			Local bbdoc$
			
			If toker.Curr()<>T_REM 
				kind = toker.Bump()
				
				' Fix to stop fields / methods etc appearing at top of docs if type not doc'd.
				If kind = T_TYPE Then
					skip = True
				Else
					Continue
				End If
			EndIf
			
			If Not skip Then
				If toker.Text()[..6]<>"bbdoc:"
					toker.Bump
					Continue
				EndIf
				bbdoc = toker.Text()

			
				kind=toker.Bump()
				If kind<>T_CONST And kind<>T_GLOBAL And kind<>T_FUNCTION And kind<>T_METHOD..
				And kind<>T_MODULE And kind<>T_TYPE And kind<>T_FIELD And kind<>T_EOL
					toker.Bump
					Continue
				EndIf
			End If
			
			Local ident$,proto$
			If kind=T_EOL
				Local i=bbdoc.Find( "keyword:" )
				If i=-1 Continue
				ident=bbdoc[i+8..].Replace( "~q","" )
				proto="Keyword "+ident.Trim()
			Else
				proto=toker.Line()
				If toker.Bump()<>T_IDENT Continue
				ident=toker.Text()
				If toker.Bump()=Asc(".") And toker.Bump()=T_IDENT
					ident:+"."+toker.Text()
				EndIf
				toker.Bump
				Select kind
				Case T_CONST,T_GLOBAL
					Local i=proto.Find( "=" )
					If i<>-1 proto=proto[..i]
				Case T_FUNCTION,T_METHOD
					Local i=proto.Find( "=" )
					While i<>-1 And proto.Find( ")",i+1 )<>-1
						i=proto.Find( "=",i+1 )
					Wend
					If i<>-1 proto=proto[..i]
					
					
					i=proto.toLower().Find( "nodebug" )
					While i<>-1 And proto.Find( ")",i+1 )<>-1
						i=proto.toLower().Find( "nodebug",i+1 )
					Wend
					If i<>-1 proto=proto[..i]
					
				End Select
			EndIf
			
			'got a valid Doc!
			Local docs:TDocs=New TDocs
			docs.kind=kind
			docs.ident=ident.Trim()
			docs.proto=proto.Trim()
			docs.bbdoc=bbdoc.Trim()
			docs.parent = parent ' Added by BaH - 25/05/2006

			parent.kids.AddLast docs
			
			If kind=T_MODULE Or kind=T_TYPE
				Parse docs
			EndIf
			
		Wend
	
	End Method
	
	Function WithToker:TDocParser( toker:TBmxToker )
	
		Local t:TDocParser=New TDocParser
		t.toker=toker
		Return t

	End Function
	
End Type

Type TTextCache
	Field returns:String
	Field keyword:String
	Field shortdesc:String
	Field longdesc:String
	Field parameters:TList = New TList

	Function Create:TTextCache(bbdoc:String, kind:Int)
		Local this:TTextCache = New TTextCache
		
		Local d:String = bbdoc
		
		Local i=d.Find("~n")
		If i=-1 i=d.length
		this.shortdesc=d[6..i].Trim()
		d=d[i+1..].Trim()
		If kind<>T_MODULE 
			FormatText this.shortdesc
		EndIf
		
	     If d.Find( "returns:" )=0
			Local i=d.Find("~n")
			If i=-1 i=d.length
			this.returns=d[8..i].Trim()
			d=d[i+1..].Trim()
			FormatText this.returns
		Else If d.Find( "keyword:" )=0
			Local i=d.Find("~n")
			If i=-1 i=d.length
			this.keyword=d[8..i].Trim()
			d=d[i+1..].Trim()
			FormatText this.keyword
		EndIf
		While d.Find("parameter:") = 0
			Local i=d.Find("~n")
			If i=-1 i=d.length
			Local param:String=d[10..i].Trim()
			d=d[i+1..].Trim()
			FormatText param
			this.parameters.addLast(param)
		Wend
		If d.Find( "about:" )=0
			this.longdesc=d[6..].Trim()
			FormatText this.longdesc
		EndIf	
		
		Return this
	End Function
End Type

Win32 binary is provided here (93kb) for your convenience.
(tested on linux & win32)

:o)

Yan	(Posted 2006) [#2]

Top job Mr Brucey!

A couple of things, 'bmxparser.bmx' is called 'docparser.bmx' here.

Is that link for the binary correct as I couldn't find it anywhere?

/me goes to peruse me new docs... :o)

Brucey

(Posted 2006) [#3]

Apologies...

Text and link fixed above...