Harpl language
Monkey Archive Forums/Monkey Projects/Harpl language
| ||
| In the form of an academic/learning project I'm writing a language (compiler, assembler and VM) from scratch using Monkey. It is just a personal learning project, but I wish to get something usable at some point. The idea is to have a script language that is written in Monkey so it can provide scripting for a wide range of platforms without having to rewrite anything. The project is on very early status, but as this is a big project and I think others may find it interesting, I've opensourced the development. I'm sure it won't have any nice speed whenn used from JavaScript on an HTML5 application, but I'm curious to see how far it can go (if it goes anywhere at the end!) The project page is here: http://code.google.com/p/harpl-project/ Mainly, I have a proper lexer, a small limited expression compiler and a small idea of how the whole compilation process will be done and some draft ideas of the language virtual assembler/bytecode. Everything in the project is open source under the new BSD license, but not yet usable for anything. |
| ||
| Wow, that's awesome. Haven't looked at it yet but building an own language isn't that easy as it sounds. :) |
| ||
| Yes, it's long and complicated but that's where the fun begins. Also I need a big decent project to really stress test Jungle Ide, so all in all, I hope we'll get some useful things from this work. |
| ||
| interesting project, what kind of language will it be similar to? BASIC? |
| ||
Yes, my idea is it will be a very strict procedural basic, The idea is to produce a language that is easy to code with and easy to maintain and integrate into your own applications and APIs. Later I'll add arrays and I hope to be able to add some OO capabilities too, but time will tell. Now it understand something as simple as:Var MyVariable as Integer = 45, MyVariable2 as Integer = MyVariable * 4, Another as String = 'Hello World!'; Nothing that you can do with it! But it'll be (hopefully) growing slowly. Ideas are welcome of course, dev is open. Just take into account this is not meant to be faster than any other JIT languages or the like, this is a sort of personal research project, we'll see where it ends. |
| ||
| I've been making some progress and I'll soon have a first compilable Hello World! program. It's incredible the amount of work a simple VM compiler can have in its background. The good news is that I expect the current structure to allow a fast "grow" of the language itself (we'll see). I'm planing on adding functions soon in a way that any script can define and call a Monkey function, and also the Monkey host program will be able to call any Harpl function. (a sort of dynamic runtime link system will be required here). Also, the whole engine can expose its global variables to Monkey and allow the host program to modify them. I don't expect this language to be the fastest in the world, but I think it can perform quite decently on some targets as it is using direct memory addresses (internally arrays) to store its data instead of any reflection or hashtable approach. This should make data access quite fast, and data allocation is designed to be pre-measured on the compilation process, so it should be faster than some alternatives... we'll see... |
| ||
| Managed to compile and run a Hello World! program. Incredible long process to just output a string but... it's starting to shape. Hope to have something usable soon. |
| ||
| The joy of complex-large developments. I've just realised I would need a template precompiler for Monkey in order to finish this in a sane way. Well.. Time for a small additional dev... I will make it also as part of the Harpl language project, but as a separate thing. |
| ||
| After using a template parser, everything has started to be faster. Now the Harpl language is able to compile and execute programs that declare local variables of type Integer, String, Float or Boolean and it is able to make some basic arithmetic operations with the data. Hurray! If anyone is needed a template parser for Monkey, this is mine: [monkeycode]Import os Global Error:Bool = False Function Main() if AppArgs.Length<>2 Then PrintInfoHeader Print "Wrong parameters number." Return -1 EndIf Local filePath:String = AppArgs[1] if filePath.StartsWith(".") or filePath.Contains(":") = false Then filePath = os.RealPath(filePath) if FileType(filePath)<>1 Then PrintInfoHeader Print "Wrong file. File was not found or it is not a file." Return -1 endif Local text:String = LoadString(AppArgs[1]) if text = "" Then PrintInfoHeader Print "File was empty" Return -1 EndIf 'Print "Parsing document: " + os.StripAll(filePath) 'Print "Located at: " + GetDir(filePath ) Print "Harpl-Monkey template parser." Print "Parsing " + filePath text = ParseDoc(text,GetDir(filePath)) if Error = 0 then SaveString(text,filePath) Print "File properly parsed." Return 0 Else Print "Error procesing source code. File was not modified" Return 4 End if End Function PrintInfoHeader() Print "Monkey template parser for the Harpl source code." Print "This is a simple parser app, pass the file to be pre-compiled as the first paramter, and you're good to go!" End Function GetPath:String(file1:String, relative:String) Return os.RealPath(file1 + relative) End Function GetDir:String(filename:String) filename = filename.Replace("/","\") Local lastIndex:Int = filename.FindLast("\") Return filename[.. lastIndex+1] End Function ParseDoc:String(text:String, replaces:Replacer[], fileLocation:String, prefix:String, level:Int) Local lines:String[] = text.Split("~n") Local lines2:list.List<String> = new list.List<String> Local ignore:Int = 0 For Local i:Int = 0 until lines.Length Local teststring:String = lines[i].Trim() if teststring.ToLower.StartsWith("'loadtemplate ") if ignore = 0 then 'Print "Template load requested!" Local line:String = prefix+lines[i] For Local rep:Replacer = EachIn replaces line = line.Replace("{%" + rep.find.Trim + "%}",rep.replace.Trim) next lines2.AddLast(line) Local prefix2:String = prefix + lines[i][.. lines[i].Find("'")] local data:String[] = line.Trim.Split(",") Local repList:List<Replacer> = new List<Replacer> For Local i:Int = 1 until data.Length Local repData:String[] = data[i].Split("=") Local rep:Replacer = new Replacer if repData.Length = 2 rep.find = repData[0] rep.replace = repData[1] repList.AddLast(rep) endif Next local repArray:Replacer[] = repList.ToArray() Local filelocation:String = os.RealPath(fileLocation + data[0][14..].Trim()).Replace("/","\") 'Print "Requesting template located at: " + filelocation if level = 0 then lines2.AddLast(prefix2 + "'#Region Code generated by the Harpl-Monkey template. Loaded from: " +data[0][14..].Trim() ) if FileType(filelocation) <>1 Then Print "The file [" + filelocation + "] can't be found." Error = true else lines2.AddLast(ParseDoc(LoadString(filelocation),repArray,GetDir(filelocation),prefix2,level+1)) EndIf if level = 0 then lines2.AddLast(prefix2 + "'#End Region") endif ignore+=1 ElseIf lines[i].Trim().ToLower = "'endtemplate" ignore-=1 end if ignore = 0 Then Local line:String = prefix+lines[i] For Local rep:Replacer = EachIn replaces line = line.Replace("{%" + rep.find.Trim + "%}",rep.replace.Trim) next lines2.AddLast(line) endif Next Local result:String, first:Bool = true For Local s:String = eachin lines2 if first Then result = s first = false else result = result + "~n" + s EndIf Next if ignore<>0 Then Error = true Return result End Class Replacer Field find:String Field replace:String End Function ParseDoc:String(text:String, fileLocation:String) Return ParseDoc(text,[], fileLocation,"",0) End[/monkeycode] It parser a document looking for 'loadtemplate sentence, and places or updates the source code contained in the template into the actual source code document. It has this syntax: 'LoadTemplate myfile.monkey, argument1 = value, argument2 = value Templates can define custom arguments, as long as they're enclosed into {% and %}. So you can import this template: File simpletemplate.monkey
For I = {%startindex%} to {%endindex%} step 5
Print I
Next
File samplefile.monkey:
Function Main()
'loadtemplate simpletemplate.monkey, startindex = 5, endindex = 100
'endtemplate
End Function
After appling the template parser:
Function Main()
'loadtemplate simpletemplate.monkey, startindex = 5, endindex = 100
For I = 5 to 100 step 5
Print I
Next
'endtemplate
End Function
If you modify the template file, and call the parser again, the parser will reflect changes accordingly. I've been using this in the Harpl virtual machine as a sort of "inliner" and it allows a good level of modularization of source code. Also, notice that this parser supports recursivenes, so a template can itself load secondary templates. |
| ||
| Added support for a PowerOf operator Yay! Working now on the data comparisong operators and then... program flow! |
| ||
| You seem to come along nicely. I've also built a scripting language for my monkey project and it went without troubles (didn't expected that). Maybe when you finished writing harpl you could reflect on it (what were big obstacles, overview, design, etc.)? |
| ||
| Yes, It's starting to look like a language, but it's too simple right now. In fact, you can't create any real program yet but the more advanced the basis is, the easier it gets to make it grow. today I've added support for bitwise operators (I'm still at this low-level area of the design). No objects, no structs, no arrays yet, no functions. By now, it's only able to compile and execute just chunks of procedural unstructured code, but it works very fast. I hope to have something usable very soon. |
| ||
| I've just finished the data-conversion operator, sich is "as" followed by the datatype. Now all primitive types can be converted from one to another. When converting a boolean to a string, I've opted for "1" for true and "0" for false, instead of using the words "true" or "false". I think numbers are much more universal. I'm going to add parentesis support soon (it should be very easy) and start working on control flow and nested data scopes. Conversion examples (in Harpl): Define Name as String = "Joseph", Age as Integer = 34 Define Major as Boolean = Age > 18 Output Name ++ ", are you older than 18? = " ++ Major as String That's a very silly example, but show the syntax involved. (++ is string concatenation) |
| ||
| While I'm working on the runtime side of this, I'm now considering a full change on the Harpl direction, in order to make it generate LLVM assembler, so it can be compiled to pure machine code, and generate EXEs, or CLI/CLR executables, or Mac executables, or any other architecture supported by the LLVM, wich would include both 32 and 64 bits processors etc etc, but this increases the complexity of the language a bit... But as a learning project it might be interesting... Any suggestion/advice from any fellow developer would be welcome. |
   |