NewsForumsCodeLogsGallerySpecsSearch

Pyramid of Doom ?

Community Forums/Monkey2 Talk/Pyramid of Doom ?

GC-Martijn(Posted 2015) [#1]
That's the word for doing this:

step1(function (value1) {
    step2(value1, function(value2) {
        step3(value2, function(value3) {
            step4(value3, function(value4) {
                // Do something with value4 
            });
        });
    });
});


But can do this using nodejs

Q.fcall(promisedStep1)
.then(promisedStep2)
.then(promisedStep3)
.then(promisedStep4)
.then(function (value4) {
    // Do something with value4 
})
.catch(function (error) {
    // Handle any error from all above steps 
})
.done();


https://www.npmjs.com/package/q

Will this be possible in MX2 ?


muddy_shoes(Posted 2015) [#2]
Will which bit be possible in MX2? You can already create chained calls if you want. The primary language feature in your example that isn't in Monkey isn't the way the "pyramid" has been collapsed, it's the passing of functions and inline anonymous declaration of them.


GC-Martijn(Posted 2015) [#3]
> create chained calls
Ow din't know that ;)


ImmutableOctet(SKNG)(Posted 2015) [#4]
Oh, so you were talking about the style. I thought you were talking about coroutines and await/async. This is a lot less fun. Grumble, grumble.


GC-Martijn(Posted 2015) [#5]
i can ask now if you want ;) does mx2 have async ?


Nobuyuki(Posted 2015) [#6]
Await, async, yield would be awesome but I get the feeling that those are a long way off and instead we'll get some basic threading library for the release version. Calling it now!


marksibly(Posted 2015) [#7]
Yeah...nah...to be honest, I don't know enough about await, async etc (or the alternatives) to be able to make a call at this point.

I do know that they felt throughly stupid when I had to use them in Winrt. eg: plenty of examples replaced this (not exactly this, but something like it...)...

LoadImage
LoadShader
CompileShader
CreateVertexBuffer
CreateRenderState

...with something like...

LoadImage.Then(
LoadShader.Then(
CompileShader.Then(
CreateVertexBuffer.Then(
CreateRenderState.Then(
someFlag=true;
)))));

...can't remember exactly how it worked, but it seemed pretty bizarre at the time. The rules for handling exceptions were also very confusing.

I guess it was to make it impossible to block the GUI thread, but it seemed pretty extreme compared with something like:

CreateThread( Lambda()
LoadImage
LoadShader
CompileShader
CreateVertexBuffer
CreateRenderState
someFlag=True
End )

...assuming you really even needed to in the first place. Some of the stuff that was async in WinRT didn't really seem like it had to be.

Perhaps JS has to do it this way because there are no threads? Perhaps it's just a better way to do it? Perhaps there are better ways to do it? Like I say, I'm not really experienced enough here to make a sensible call, so for now async/await etc are on the 'weird, interesting ideas' list.

And as has been mentioned, I think you can do the pyramid thing already, eg:

Step1( Lambda( result1:Result )
  Step2( result1:Result,Lambda( result2:Result )
    Step3( result2:Result,Lambda( result3:Result )
      Step4( result3:Result,Lambda( result4:Result )
        someFlag=True
      End )
    End )
  End )
End )


Not all that pretty - and untested! - but again, I just don't know how useful this is in practice. For starters, if the point of the exercise is to prevent the GUI thread blocking, then Step1, Step2, etc will still need to be able to launch a thread to do their work asynchronously (or I guess they could do work 'in chunks' on an idle signal). And if we've got threads, the whole thing can be collapsed into a simple statement sequence running on a separate thread.


nullterm(Posted 2015) [#8]
There's easier ways.





Fun fact, I once worked with an "engine" for a hugely popular game and it had "if" statements 8 or 9 layers deep.


marksibly(Posted 2015) [#9]
> There's easier ways.

But the point of the 'then' or lambda approaches is to allow the code to run asynchronously - similar to running your code on a thread. Which does indeed seem more intuitive to me, but that's probably largely to do with my background.


nullterm(Posted 2015) [#10]
Uhhhh I see, never mind.


GC-Martijn(Posted 2015) [#11]
There are 3 things now in this topic ;)


1. then()
Modern browsers already support ECMAScript 6 specification of Promises.
https://github.com/promises-aplus/promises-spec
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise

for javascript its a cool thing because your don't have to write many code.


example code (maybe not the best one)

function getRow(id) {
    return Q.nfcall(db.query,    'SELECT * FROM table WHERE lookUp = ?', id);
}

function insertItem(request, response) {
    var data = JSON.parse(request.body.data);

    security.verifyToken(data.lookUp).then(function(/* no lookupError */) {
        return getRow(data.lookUp); // potentially catch SQL errors here
    }).then(function(row) {
        return helper.insertFeedItem(row[0].id, data.message);
        // What was the `res` it had returned before?
    }).catch(function(someError) { // lookUpError, companyError, insertError
        return { "response": "error", "msg": someError };
    }).done(function(result) {
        response.json(result);
        response.end();
    });
}


2. async
That weird thing, without that I had many angry clients for some C# projects.
Its weird because I don't really know if its a separate thread or a special weird thing :)

Thing is that you need them to not block your GUI and get those frozen screens.
Good information about this: https://msdn.microsoft.com/en-us/library/hh191443.aspx

Do we need them in MX2 ???
Yes and no, I don't know if MX2 will get threads, or other non blocking things.
For example to download stuff in the background, while you push a GUI button [download now], and you don't have to wait for it.


3. chaining
-> solved


nullterm(Posted 2015) [#12]
Lua has the concept of coroutines, which are like lightweight threads, but they actually run cooperatively with-in the same thread. I've used them in the past for mission scripting. Just found a Lua based system very similar to what I used before.

The advantage compared to threads is no threading issues like locks, threading race conditions, etc. Or UI/graphics actions that can only be executed in the main thread aren't an issue.

The disadvantage compared to threads is that a big long function call or loop will block your main thread. And non-trivial to implement in a C++ based system.

https://github.com/davisdude/Timer




dmaz(Posted 2015) [#13]
right, coroutines are not threads... They've been asked for and I remember a short post from Mark showing a working example.

Async and await look similar to a coroutines though... I think they are basically there just to to simplify the syntax for callbacks. Mark's CreateThread example doesn't accomplish this. so let's see, async sets up the process. Await spawns a thread and yields back to the main thread(or previous calling thread). Then once the spawned thread completes the remainder of the async function is executed.... but it's executed in the main thread and in the main threads context/scope.... so that's actually pretty nice and clean.

So, one way to fix this without async and await (or polling your thread) would be to add a callback function argument to CreateThread...?
CreateThread(
	Lambda()
		LoadImage()
		LoadShader()
		CompileShader()
		CreateVertexBuffer()
		CreateRenderState()
	End,
	Lambda()
		someObject.SetValue(true)
		DoSomethingAfter()
	End,
)
I think await would look better.
MyFunction() Async
	blah.blah

	Await( Lambda()
		LoadImage()
		LoadShader()
		CompileShader()
		CreateVertexBuffer()
		CreateRenderState()
	End)
	
	someObject.SetValue(true)
	DoSomethingAfter()

End

what I'd like to see is good builtin messaging between threads and stuff like
Parallel.For :)


dmaz(Posted 2015) [#14]
or
blah.blah
CreateThread( Lambda()
	LoadImage()
	LoadShader()
	CompileShader()
	CreateVertexBuffer()
	CreateRenderState()
	End
).Callback( Lambda()
	someObject.SetValue(true)
	DoSomethingAfter()
	End
)


change .Callback to .CallbackAdd and you have your 'then's ( async/await still looks cleaner though )


GC-Martijn(Posted 2015) [#15]
I vote for dmaz await,async code.
Mark you realy need to check those things, because people are going to compare your mx2 with other languages.
The new python has it now, nodejs, c# its a new trendy thing :)

It gives more power then not freezing the gui, but to do some othter background stuff to, like special effects in the background.
But then again, its almost or the same as threads...


dmaz(Posted 2015) [#16]
Awesome read about asynchronous experiences from Microsoft's (now ended) internal Midori project...
http://joeduffyblog.com/2015/11/19/asynchronous-everything/
Some of links are really good too.


dmaz(Posted 2015) [#17]
After that link and thinking about this a little more... IMO, Async / Await is the way to go... fast and powerful if built around a scheduler built on structs of course. then we could do stuff like This would be so AWESOME!


marksibly(Posted 2015) [#18]
> Awesome read about asynchronous experiences from Microsoft's (now ended) internal Midori project...

Good find!

Looking back on my WinRT experience, I think the major problem I had was that I was working primarily in c++, which doesn't yet have async/await, only that crappy '.then{ .then{ .then{ .then{ } } } }' stuff. With await, it all starts to look a lot saner:

...await LoadImage()
...await LoadShader()
...await CompileShader()
...await CreateVertexBuffer()
...await CreateRenderState()

Part of the problem for me though is that I'm the guy who has to write async versions of LoadImage, LoadShader etc which, given they're built on top of synchronous C code (eg: fread, glCompileShader etc), means lower level threads are unavoidable if the goal is to avoid blocking the UI. And if there are threads, it's tempting to just leave it up to users to wrap sequences of slow ops in a thread!

Async ops are coroutines though, not full-on threads, but still, the easiest way to implement them in c++ is probably on top of threads.

MS have in fact implement async/await in c++ using 'fibers' which is a form of lightweight thread:

https://paoloseverini.wordpress.com/2014/04/22/async-await-in-c/

I think this is probably the general approach monkey2 should take if it's to attempt async/await, ie: start with something like plain threads and 'Future<T>' and expand it later to support async/await.


dmaz(Posted 2015) [#19]
sure, but I think the goal shouldn't be just to avoid blocking... I'd like to have quick easy syntax to launch and manage threads at the language level not just through a framework. so we can do stuff in parallel and take advantage of multiple cores

that's the first I heard of fibers. They sound like a solid way to implement async/await for monkey2... I don't think we would really require a preemptive solution so yeah, can't wait! :)

[edit] reworded and corrected some typos


Pharmhaus(Posted 2016) [#20]
Groovy and C# both have the ?. Operator.
which is basically a short hand for a null check.
It is not async but prevents nesting Ifs when not necessary.
MyInstance?.MyMethod()

Which would be the aquivalent of
If MyInstance Then
    MyInstance.MyMethod()
Endif

It is also chainable like so
MyInstance?.MyProperty?.MyMethod()

It returns the default value for the result type when a return value is expected and the instance is null.


DruggedBunny(Posted 2016) [#21]
That's pretty cool, actually.