BlitzMax Doxygen

BlitzMax Forums/BlitzMax Programming/BlitzMax Doxygen

SystemError51

(Posted 2012) [#1]

It's been a while since I've been posting into this category, I also couldn't find anything on the subject.

Some of you may be familiar with a little something called Doxygen. It is a very clever source code parsing tool that creates nicely designed documentation for you, in HTML and LaTeX. I'm sure some of you are using it as it is easy as pie to maintain documentation in large-scale projects.

Doxygen is a documentation system for C++, C, Java, Objective-C, Python, IDL (Corba and Microsoft flavors), Fortran, VHDL, PHP, C#, and to some extent D.

Now the question is, if anyone has ever created something similar to that for BlitzMax, or if someone came across a tool like that? It would be very advantageous for a lot of people, I would say.

I'm wondering about this since I'm back on B/Max, so any insight is as always appreciated and taken on board.

matibee

(Posted 2012) [#2]

There's a document generator built in to Blitzmax. Can't recall the ins and outs at the minute though.

Russell

(Posted 2012) [#3]

Yep. If you click on 'source' at the top of most of the Bmax help pages, you'll see the source for that particular module. Look for 'bbdoc:' inside rem/end comment blocks.

Russell

Derron

(Posted 2012) [#4]

I wrote a simple php script which parses my bmx sources, handles globals, child variables/methods/functions, comments ...

Using your own parser is a nice way to allow custom documentations (I needed a documentation about lua-readable/writeable objects - so I also parse {metatags}). Also nice is the possibility to vary the documentation style (@param, ...). "Ignore patterns" are also nifty - if you don't like to document some external libs (bruceys mods, some other helper libs).

Parser and template filling took only some hours to code - and it fits 100% of your needs at the end - may be that is a possibility for you.

bye
Ron

SystemError51

(Posted 2012) [#5]

Would it be possible for you to share that little PHP gem with me?

Derron

(Posted 2012) [#6]

Some things could be improved but it does what it should for me.

It works for things like:

'Summary: all kind of characters walking through the building (players, terrorists and so on)
Type TFigures {_exposeToLua="selected"}
	Field Name:String		= "unknown"
	Field pos:TPosition		= TPosition.Create(0.0,0.0) {_exposeToLua}		'pos.y is difference to y of building
	Field dx:Float			= 0.0 'pixels per second
	Field initialdx:Float	= 0.0
	field target:TPosition	= TPosition.Create(-1,-1) {_exposeToLua}
...

index.php

<?php  header("Content-Type: text/html; charset=utf-8"); ?>
<?
	$ignoreTypes = array("TxmlError", "TxmlXPathContext", "TTCPStream", "TSockAddr");
	$ignoreFiles = array(	"/source/external/libxml/libxml.bmx",
							"/source/external/persistence.mod/persistence.bmx",
							"/source/external/bnetex/bnetex.bmx",
							"/source/basefunctions_zip.bmx"
						);


	$action	= isset($_GET["action"]) ? $_GET["action"] : "index";
	$path	= isset($_GET["mainfile"]) ? $_GET["mainfile"] : "";

	switch($action) {
		case "build":
					BuildDoc($path);
					break;
		default:
					ShowIndex();
					break;
	}


	function BuildDoc($path) {
		global $ignoreTypes;
		global $ignoreFiles;
		require_once("includes/parser.class.php");

		$export = "ALL";
		if(isset($_GET["onlyLua"]) && $_GET["onlyLua"] ==1)
			$export = "LUA";


		$result = "<pre>";
		$result.= parser::parse($path, dirname(parser::MyRealPath($path)), $ignoreFiles);
		$result.= "</pre>";
		$template = file_get_contents("template.html");

		parser::$data["currentLines"] = null;

		$elementMethodsBox = "
			<div id='element' class='even%even%'>
				<div id='header' class='even%even%'>
					<div id='wrapper' class='%LUA%'>
						<span id='typeString'>%type%</span>
						<span id='name'>%name%</span>
						<span id='returns'>%returns%</span>
						<span id='params'>( %params% )</span>
						<div class='clear'></div>
					</div>
				</div>
				%comment%
			</div>
		";

		$elementVarsBox = "
			<div id='element' class='even%even%'>
				<div id='header' class='even%even%'>
					<div id='wrapper' class='%LUA%'>
						<span id='typeString'>%type%</span>
						<span id='name'>%name%</span>
						<span id='varType'>%varType%</span>
						<div class='clear'></div>
					</div>
				</div>
				%comment%
			</div>
		";



// extends -> funktion etc in grau anzeigen

		$result = "";
		foreach(parser::$data["foundTypes"] as $name=>$type) {
			if(in_array($name, $ignoreTypes)) { continue; }

			$luaForType = "";
			if ($type->hasMetaKey("_exposetolua")) {
				//print $type->name." meta: ".$type->getMetaKey("_exposetolua")."<br />";
				switch($type->getMetaKey("_exposetolua")) {
					case "" :			$luaForType = "LuaAll";break;
					case "selected" :	$luaForType = "LuaSelected";break;
					default:			$luaForType = "";break;
				}
			}

			if($export == "ALL" OR ($export=="LUA" AND $luaForType<>"")) {
				$result.="<div id='type'>";
				$result.="	<div id='header'>";
				$result.="		<span id='name' class='$luaForType'>$name</span>";
				$result.="		<span id='comment'>{$type->comment}</span>";
				$result.="	</div>";
				$result.="	<div id='elements'>";
			}
			usort($type->children, 'compare_byName');

			$even = 0;
			foreach($type->children as $name=>$func) {
				$even = 1-$even;
				$comment = $func->comment <> "" ? "<div id='comment'>".$func->comment."<br /></div>" : "";
				$meta = count($func->meta)>0 ? "<div id='meta'>".$func->getMetaString()."<br /></div>" : "";

				$lua = "";
				if($luaForType <> "") {
					if ($luaForType == "LuaAll") {
						$lua = "LuaAll";
					}else{
						if ($func->hasMetaKey("_exposetolua")) {
							switch($func->getMetaKey("_exposetolua")) {
								case "" :			$lua = "LuaAll";break;
								case "selected" :	$lua = "LuaSelected";break;
								default:			$lua = "";break;
							}
						}
					}
				}

				//fill non-vars
				if(in_array($func->getType(), array("method", "function", "type"))) {
					$replaceA = array("%even%", "%type%","%name%","%returns%","%params%", "%comment%", "%meta%", "%LUA%");
					$replaceB = array($even, $func->getType(),$func->name,$func->returns <> "" ? ":".$func->returns : "",$func->params, $comment, $meta, $lua);
					if($export == "ALL" OR ($export=="LUA" AND $lua<>""))
						$result.= str_replace($replaceA, $replaceB, $elementMethodsBox);
				}else{
				//fill vars
					$replaceA = array("%even%", "%type%","%name%","%varType%","%comment%", "%meta%", "%LUA%");
					$replaceB = array($even, $func->getType(),$func->name,$func->varType <> "" ? ":".$func->varType : "",$comment, $meta, $lua);
					if($export == "ALL" OR ($export=="LUA" AND $lua<>""))
						$result.= str_replace($replaceA, $replaceB, $elementVarsBox);
				}
			}
			$result.="	</div>";
			$result.="</div>";
		}

		$template = str_ireplace("%content%", $result, $template);

		print $template;
	}

	function compare_byName($a, $b){
		return strnatcasecmp($a->name, $b->name);
	}

	function ShowIndex() {
		print getHtmlHeader("Index");

		print "		<form method='get'>\n";
		print "			<label for='mainfile'>Projektdatei</label>\n";
		print "			<input type='' name='mainfile' id='mainfile' value='/ABSOLUTEPATH/mainfile.bmx' style='width:400px;'/><br />\n";
		print "			<label for='onlyLua'>Exportieren:</label>\n<select name='onlyLua'><option value='1'>Nur Lua</option><option value='0'>Alles</option></select><br />\n";
		print "			<label for='mainfile'>Aktion</label>\n";
		print "			<input type='hidden' name='action' value='build' />\n";
		print "			<input type='submit' value='abschicken' />\n";
		print "		</form>\n";
		print getHtmlFooter();
	}



	function getHtmlHeader($title) {
		$result = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">'."\n";
		$result.= "<html xmlns='http://www.w3.org/1999/xhtml'>\n";
		$result.= "<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />\n";
		$result.= "	<head>";
		$result.= "		<title>$title</title>";
		$result.= "	</head>";
		$result.= "	<body>";
		return $result;
	}

	function getHtmlFooter() {
		$result = "	</body>";
		$result.= "</html>";
		return $result;
	}

?>

includes/parser.class.php

<?
require_once("dataobject.class.php");


class parser {
	static $patterns = array(
						'includes'			=> "/\b(Import|Include).*?\"(?P<name>.*?).bmx\"/i",
						'typesSingle'		=> "/type\b\s(?P<name>[a-zA-Z0-9_ \{\}]+)/i",
						'fieldAndConst'		=> "/(field|global)\s+(?P<name>\w+)\s*:\s*(?P<type>\w*)/i",
						);
						/* unused
						'methods'			=> "/(.*?Rem\b[?P<doc>.*?][EndRem|End Rem])?[\n|\r\n|\n\r]?[ \t]?\bmethod\s(?P<name>.*?)[\n|\r\n|\n\r](?P<content>.*?)end\s?method/si",
						'comments'			=> "/.*?Rem(?P<content>.*?)End\s?Rem/si",
						'types'				=> "/[ \t]?\btype\s(?P<name>[a-zA-Z0-9_ \{\}]+)[\n|\r\n|\n\r](?P<content>.*?)end\s?type/si",
						'functions'			=> "/[ \t]?\bfunction\s(?P<name>.*?)[\n|\r\n|\n\r](?P<content>.*?)end\s?function/si",
						*/
	static $data = array();

	function MyRealPath($path) {
		$pattern = '/\w+\/\.\.\//';
		while(preg_match($pattern,$path)){
			$path = preg_replace($pattern, '', $path);
		}
		return $path;
	}

	public function parse($file,$baseUrl, $ignoreImports=null) {
//		print "parsing ... ".$file."<br />";
		$result		= "";
		if(!file_exists($file))
			return $result."file not found.";

		$content = file_get_contents($file);
		$content = str_ireplace("..\n", "", $content);
		$content = str_ireplace("..\r\n", "", $content);
		//easy linebreak
		$content = str_ireplace("\r\n", "\n", $content);
		$content = str_ireplace("\n\r", "\n", $content);

		$lines = explode("\n", $content);
		foreach($lines as &$line)
			$line = trim($line);

		self::$data["baseUrl"] = $baseUrl;
		self::$data["currentLines"] = $lines;
		self::$data["ignoreImports"] = is_array($ignoreImports) ? $ignoreImports : array();

		$fileObject = new fileObject($file);

		//find types/classes...
		$result.= self::findTypes($fileObject, $lines);


		//find imports/includes
		$result.= self::findImports($fileObject, $lines);
		return $result;
	}

	function findImports($fileObject, $lines) {
		$result = "";
		foreach($lines as $lineNum=>$line) {
			if( self::isComment($line) )
				continue;

			preg_match_all(self::$patterns["includes"], $line, $foundItems);
			if( !isset($foundItems["name"]) OR count($foundItems["name"])<=0 )
				continue;

			if( !array_key_exists("parsedFiles", self::$data) )
				self::$data["parsedFiles"] = array();

			$fileName = $foundItems["name"][0];
			$fileName.=".bmx";
			$filePath = str_ireplace(self::$data["baseUrl"], "", parser::MyRealPath(dirname($fileObject->uri)."/".$fileName));

			////////////////// remove AND ... for normal
			if(!in_array($filePath, self::$data["parsedFiles"]) ){ // AND count(self::$data["parsedFiles"]) <3) {
				$result.= "import file: ".$filePath."\n";
				if(!in_array($filePath, self::$data["ignoreImports"])) {
					//print $filePath." - - ".print_r(self::$data["ignoreImports"],1)."<br />";
					$obj = new fileObject($filePath);
					$obj->setFoundOnLine($lineNum);
					$obj->setParent($fileObject);
					self::$data["parsedFiles"][] = $obj;
					$result.= self::parse(self::$data["baseUrl"].$filePath, self::$data["baseUrl"], self::$data["ignoreImports"]);
				}
			}
		}
		return $result;
	}

	function findTypes($fileObject, $lines) {
		$inType		= false;
		$result		= "";
		foreach($lines as $lineNum=>$line) {
			if( self::isComment($line) )
				continue;

			if($inType) {
				if(stripos($line, "endtype")===false AND stripos($line, "end type")===false )
					$typeLines[$lineNum] = $line;
				else {
					$inType = false;
					$result.= self::findChildElements($fileObject, $obj, $typeLines);
					$typeLines = array();
				}
			}else{
				$typeLines = array();
				preg_match_all(self::$patterns["typesSingle"], $line, $foundItems);
				if( !isset($foundItems["name"]) OR count($foundItems["name"])<=0 )
					continue;

				$inType = true;

				$name = $foundItems["name"][0];
				$parts		= explode("{", $name);
				$meta		= isset($parts[1]) ? substr($parts[1],0, strpos($parts[1], "}")) : "";
				$parts		= explode(" ", trim($parts[0]));
				$name		= $parts[0];
				$extends	= isset($parts[1]) && strtolower($parts[1]) == "extends" && isset($parts[2]) ? $parts[2] : "";
				$abstract	= isset($parts[1]) && strtolower($parts[1]) == "abstract" ? true : false;

				if( $name == "")
					continue;
				$obj = new typeObject($name);
				$obj->setComment(self::getCommentForLine($lineNum));
				$obj->setMeta(self::getMetaForLine($lineNum));
				$obj->setFoundOnLine($lineNum);
				$obj->setParent($fileObject);
				$obj->setMeta($meta);
				$obj->setExtends($extends);
				$obj->setAbstract($abstract);

				self::$data["foundTypes"][$name] = $obj;

				$result.= "  type: ".$name."<br />";
			}
		}
		return $result;
	}

	function getLine($lineNum) {
		return isset(self::$data["currentLines"][$lineNum] ) ? self::$data["currentLines"][$lineNum] : "";
	}

	function getMetaForLine($lineNum) {
		$meta = self::getInLineMeta( self::getLine($lineNum) );
		$line = trim(self::getLine($lineNum-1));

		if( stripos($line, "{")===0 ) {
			$begin	= strpos($line, "{")+1;
			$end	= strpos($line, "{");
			$meta .= ",".substr($line, $begin, $end-$begin);
		}
		//print $meta."<br />";
		return $meta;
	}

	function getCommentForLine($lineNum) {
		$comment = self::getInLineComment( self::getLine($lineNum) );

		$line = self::getLine($lineNum-1);

		if( stripos($line, "endrem")===0 OR stripos($line, "end rem")===0) {
			$commentEnd		= $lineNum-1;
			$commentStart	= $lineNum-1;
			$line			= self::getLine($commentEnd-1);
			while( stripos($line, "rem") !== 0 ) {
				$commentStart	= $commentStart -1;
				$line			= self::getLine($commentStart-1);
			}

			for($i=$commentStart;$i<$commentEnd;$i++) {
				$comment.= self::getLine($i)." ";
			}
			$comment = rtrim($comment);
			//print "<i>".$comment."</i> (".$commentStart."-".$commentEnd.")";
		}
		if( stripos($line, "'")===0) {
			$commentEnd		= $lineNum-1;
			$commentStart	= $lineNum-1;
			$line			= self::getLine($commentEnd-1);
			while( stripos($line, "'") === 0 ) {
				$commentStart	= $commentStart -1;
				$line			= self::getLine($commentStart-1);
			}

			for($i=$commentStart;$i<=$commentEnd;$i++) {
				$comment.= substr(self::getLine($i), 1)." ";
			}
			$comment = rtrim($comment);
			//print "<i>".$comment."</i> (".$commentStart."-".$commentEnd.")";
		}



		return $comment;
	}

	function getInLineComment($line) {
		$parts		= explode("'", $line);
		return isset($parts[1]) ? $parts[1] : "";
	}

	function getInLineMeta($line) {
		$parts		= explode("{", $line);
		return isset($parts[1]) ? substr($parts[1], 0, strpos($parts[1], "}")) : "";
	}

	function findChildElements($fileObject, $typeObject, $lines) {
		$result = "";
		foreach($lines as $lineNum=>$line) {
			//find comments
			if( self::isComment($line) )
				continue;

			//find methods/functions
			if(stripos($line, "method")===0 OR stripos($line, "function")===0 ) {
				$parts		= explode("(", $line);
				$name		= explode(":", $parts[0]);
				$returns	= isset($name[1]) ? $name[1] : "";
				$name		= substr($name[0], strpos($name[0], " ")+1);

				$parts		= explode(")", $parts[1]);
				$params		= $parts[0];
				$meta		= isset($parts[1]) && strpos($parts[1], "{") ? substr($parts[1],strpos($parts[1], "{"), strlen($parts[1])-strpos($parts[1], "}")) : "";


				$obj = new functionObject($name, stripos($line, "method")===0 ? 1 : 0);

				$obj->setReturns($returns);
				$obj->setParams($params);

				$result.= "    ".$name."\n";
			}

			//find variables/constants
			$foundField	= stripos($line, "field")===0 ? 1 : 0;
			$foundConst	= stripos($line, "const")===0 ? 1 : 0;

			if( $foundField OR $foundConst ) {
				preg_match_all(self::$patterns["fieldAndConst"], $line, $foundItems);
				if( !isset($foundItems["name"]) OR count($foundItems["name"])<=0 )
					continue;

				$obj = new variableObject($foundItems["name"][0], $foundField ? 0 : 1, $foundItems["type"][0]);
				$result.= "    ".$foundItems["name"][0]."\n";
			}



			//common things (common for fields, const, types, methods, ...)
			if(isset($obj)) {
				$obj->setFoundOnLine($lineNum);
				if($typeObject) {
					$obj->setParent($typeObject);
					$typeObject->children[strtolower($obj->name)] = $obj;
				}else{
					$obj->setParent($fileObject);
					$fileObject->children[strtolower($obj->name)] = $obj;
				}

				$obj->setComment(self::getCommentForLine($lineNum));
				$obj->setMeta(self::getMetaForLine($lineNum));
			}
		}
		return $result;
	}


	function isComment($line) {
		if( stripos($line, "'") === 0 ) {
			return true;
		}
		if( stripos($line, "rem") === 0 ) {
			self::$data["inComment"] = true;
			return true;
		}
		if( stripos($line, "endrem") === 0 OR stripos($line, "end rem") === 0 ) {
			self::$data["inComment"] = false;
			return true;
		}
		if( !isset(self::$data["inComment"]) )
			self::$data["inComment"] = false;

		return self::$data["inComment"];
	}
}

?>

includes/dataobject.class.php

<?

class dataObject {
	public $name			= "";
	public $data;
	private $_type			= "";
	private $_parentObject	= null;
	public $foundOnLine		= 0;
	public $meta			= array();
	public $comment			= "";
	public $children		= array();

	function __construct($name, $type="data") {
		$this->_baseInit($name, $type);
	}

	function setParent($parentObject) {
		$this->_parentObject = $parentObject;
	}

	function setFoundOnLine($line) {
		$this->foundOnLine = $line;
	}

	function setMeta($meta) {
		//$this->meta = array();

		$arr = explode(",", $meta);
		foreach($arr as $meta) {
			$meta = explode("=",$meta);
			$key  = $meta[0];
			$value= isset($meta[1]) ? str_replace("\"", "", $meta[1]) : "";
			//print $this->name." - ".$key."<br />";
			if($key <> "")
				$this->meta[strtolower($key)] = $value;
		}
	}

	function getMetaString() {
		$res = "MetaString: ";
		foreach($this->meta as $key=>$value) {
			$res.= $key;
			if($value <> "")
				$res." = ".$value;
			$res.= ", ";
		}
		return substr($res, 0, -2);
	}

	function hasMetaKey($key) {
		return array_key_exists(strtolower($key), $this->meta);
	}

	function getMetaKey($key) {
		return $this->meta[strtolower($key)];
	}


	function setComment($comment) {
		$this->comment = $comment;
	}

	function getType() {
		return $this->_type;
	}

	function _baseInit($name, $type) {
		$this->name = $name;
		$this->_type = $type;
	}
}


class fileObject extends dataObject {
	public $uri	= "";

	function __construct($file) {
		$this->_baseInit($file, "file");
		$this->setUri($file);
	}

	function setUri($uri) {
		$this->uri = $uri;
	}

}

class typeObject extends dataObject {
	public $extends = "";
	public $abstract= false;

	function __construct($name) {
		$this->_baseInit($name, "type");
	}

	function setExtends($extends) {
		$this->extends = $extends;
	}

	function setAbstract($abstract) {
		$this->abstract = $abstract;
	}

}

class functionObject extends dataObject {
	public $returns = "";
	public $params = array();

	function __construct($name, $type) {
		if($type == 0)
			$this->_baseInit($name, "function");
		if($type == 1)
			$this->_baseInit($name, "method");
	}

	function setReturns($val) {
		$this->returns = $val;
	}

	function setParams($val) {
		$this->params = $val;
	}

}

class variableObject extends dataObject {
	public $varType= "";

	function __construct($name, $type, $varType ) {
		if($type == 0)
			$this->_baseInit($name, "var");
		if($type == 1)
			$this->_baseInit($name, "const");

		$this->varType = $varType;
	}

}

?>

template.html

<html>
<head>
	<style>
		body					{ font-family: Sans-Serif; font-size:13px;}
		#type					{ border: 1px solid #ddf; margin-bottom:1.5em; }
		#type>#header>#name		{ display:block; background: #ccd; font-weight: bold; font-size:1.1em;}
		#comment				{ background: #f0f0f0; display:block; font-style: italic; color:#555;}
		#element				{ margin-bottom: 2px;}
		#element #header.even0	{ background: #e0e0f0;}
		#element #header.even1	{ background: #e5e5f5;}
		#type>#header>#name.LuaAll,
		.LuaAll					{ background: #ffeffa;}
		#type>#header>#name.LuaSelected,
		.LuaSelected			{ background: #ffe5f5;}
		#element span 			{ float:left; }
		#element #typeString	{ display:block; width:3.8em; color: #999;}
		#element #name			{ font-weight:bold; }
		.clear					{ clear:both; }
		#element>#comment		{ padding-left: 3.8em; padding-bottom:1em;}
	</style>
</head>
<body>
	%CONTENT%
</body>
</html>

bye
Ron

SystemError51

(Posted 2012) [#7]

Wow - awesome Ron!

I shall try this out and report back :)

Derron

(Posted 2012) [#8]

Remember not to host it accessible from outside without adding further checks to the filename (I host it only in a internal-only-directory of my apache) else someone will be able to do bad things :D.

Best case here: prepend your Main-Working-directory (eg. "C:\ProgrammingMyWork") to the filename given in index.php and do some further checks...

replace

$path	= isset($_GET["mainfile"]) ? $_GET["mainfile"] : "";

with

$path	= isset($_GET["mainfile"]) ? $_GET["mainfile"] : "";
$path	= str_replace('\','/', $path); // win, unix... slash backslash
$path	= str_replace("../", "", $path); //some try to get out of directory bounds
$path	= "C:\Programming\MyWork".$path;

//finish if not a blitzmax file (disable the check if custom extensions are possible)
if( strtolower(pathinfo($path, PATHINFO_EXTENSION)) <> "bmx" )
	return;

There are other improvements possible (template files for methods, types, ... to make it easier for them to get collapseable ( [+] and [-] signs at the header ) and so on.

bye
Ron