ofx.mod - OpenFrameworks
BlitzMax Forums/Brucey's Modules/ofx.mod - OpenFrameworks
| ||
| Another weekend... another module. This one has been a lot of fun so far - mostly because I've spent too much time "playing" with it, than finished the coding... oh well :-p OpenFrameworks is a framework designed for "creative coding". If you are familiar with Processing, the Java framework, then you'll have an idea of what this is about. It is a cross-platform OpenGL-based framework. It does images, drawing, video, sound, comms, and various other things... and is expandable via addons. (like OpenCV) With the expandibility in mind, I've created it in its own namespace "ofx", rather than "bah", since it makes more sense once new modules begin to appear. I've only worked on the Mac and Windows ports for now. Everything appears to build okay. The API is very incomplete at the moment - I'm still pondering how to do the events/keys/mouse handling. (If anyone wants to help out... I'm happy for the assistance) I've knocked together a couple of examples. They are mostly complete, excepting the key/mouse stuff. I've also built in rudimentary Max2D integration, which means you can mix and match both OpenFrameworks and Max2D. In fact, there is no reason you couldn't use MiniB3D either. The project page is at : http://code.google.com/p/ofxmax/ Currently only available via the SVN repository at : http://ofxmax.googlecode.com/svn/trunk/ One thing to note... It's rather BIG (over 110meg).. so it may take a while to download the first time. (but there's a price to pay for having lots of cool toys). They are looking to release it as LGPL, so at some point I'll be changing the current "compile in the source" method, to link to a DLL instead - which will solve any potential license issues. But I've not done that yet, as it's quicker to throw it all together this way and have a basic working version. Before a "proper" release, this would change, of course. Oh... and the external requirements : BaH.FreeImage BaH.FMOD (I don't make the rules about what audio engine to use with the framework) Installation : The base module ofx.mod should be placed as such : BlitzMax/mod/ofx.mod Examples can currently be found in core.mod/examples and ofxopencv.mod/examples. As ever, comments and whatnot always appreciated. :-) |
| ||
| BaH.FMOD (I don't make the rules about what audio engine to use with the framework) Just curious -- Does it also require fmod if you aren't using any of the sound commands? |
| ||
| That appears to be the case - I don't have control over that though. Hopefully they'll split things up a bit as the project moves forward. They've already pulled out GLUT as a requirement, which was good for me :-) |
| ||
| Now supporting Linux :-) Requires a bunch of libraries, and some luck... but it's running fine here. (also shows some inherent problems with BMK.... but you probably don't care about that). On Ubuntu, you probably want to run this : apt-get install libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev libavcodec-dev libavformat-dev libavutil-dev libswscale-dev freeglut3-dev libasound2-dev libxmu-dev libraw1394-dev (lifted from openFrameworks) That should get you required headers and libraries. Otherwise, call back here and we'll see if we can't work out what's wrong. |
| ||
| Builds and runs nicely on XP. Fun, fun, fun! Thanks Brucey :-) [edit] btw your "hand" example (webcam contour finder) runs at 1195 FPS on my 2006 iMac. I was pretty amazed at the speed. |
| ||
| Got it working! Looks very promising Brucey. Might provide another option on tracking for my putting project. If I get to the point to where I can use this this to measure the speed and angle a rolling golf ball it would be a better solution over the Phigets or the WiiMote solution I came up with. |
| ||
| You should be able to see the tracking working with the opencv example. I've still got a lot of stuff implement but even from the little I've done so far, I can see it has a lot of potential for some interesting projects. Also, if you find any interesting addons for OpenFrameworks, let me know :-) |
| ||
| Yep, I rolled a golf ball through the view area and it tracked great. I just need to know the blob center in at least 2 frames and I am all set! I'll keep my eyes open for addons. |
| ||
| Oh and all of this makes possible Multitouch Tables in Bmax, along the lines of NuiGroup/TouchLib, no? http://www.nuigroup.com/ |
| ||
| I'll work on the rest of the opencv API tonight then... there's direct access to the blobs through it - rather than the combined set that the example shows. |
| ||
| Outstanding! |
| ||
| Added blobs and a blob iterator (see the example on how to get access to them through the contour finder). It uses BlitzMax's enumerator thing... so you can process the list of blobs via an EachIn loop. I've still to implement access to the points list in the blob itself - it took me a lot longer to get the iterator working that it should have... C++ Pah! makes possible Multitouch Tables in Bmax Possibly. I suppose it depends how the driver presents the multi-touch data to the application. (maybe someone has already written some multi-touch addon) |
| ||
| Nice, getting ready to lock myself in my office for a while and play with this. |
| ||
| Nice ! I always wanter to play a bit with Process. Wonder if things like Augmented Reality are easy to do with this kind of framework ? Maybe the toy I was waiting for :) Can't wait to see the end of this one ! |
| ||
| Cool! Thank you for this. I think I'm gonna lock myself in my room as well. :D |
| ||
| Well Brucey thanks to your recent update I am able to measure the speed of a rolling ball. The only drawback is that the ball can't be moving more than a meter per second or so, else I don't get any blobs detected. I theorize that once you implement a more direct access to the blobs (if such a thing exists) it will be better since you won't be relying on the contour finder to get at them. Thanks again for your work on this, it really is very exciting. |
| ||
| Strange error.. I'm using Brucey's optimized bmk_ng under windows, and just got the code from the repo. (Typical MaxIDE Ctrl+D) Building Modules Compiling:ofAppRunner.cpp Compiling:ofArduino.cpp Build Error: failed to compile (1) C:/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/app/ofAppRunner.cpp ??????????????????????????????????????????????????????????????????????????????????????????????????????????????????????d̉ec: no input files Build Error: failed to compile (1) C:/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/app/ofAppRunner.cpp Process complete EDIT: I switched back to the original BMK so I could build and play with it. |
| ||
| That's interesting, because DavidDC had that error... but when he built it a second time it worked fine. On the same file, no less... so I'm not sure what is up with that. |
| ||
| Yes I had exactly the same error with bmk NG. Used Ctrl +D as well. Vanished on second attempt. Didn't do a thing other than press Ctrl +D twice. |
| ||
| Hmm, noticed some updates, what have you been up to Brucey :) |
| ||
| Brucey, here's an addon that would be great to have: ofxFiducialFinder ofxFiducialFinder finds and tracks fiducials from video source. ofxFiducial has... width recognition, Jitter filtering, frame flashing on/off filtering, rotation speed, rotation acceleration, motion speed, motion acceleration, motion speed x, motion speed y, a draw and drawCorners feature, getter functions which include getangle in radian and degree(compensated), = operator overloading ( I don't think any more is needed but if requested I will), point inside fiducial helper function check, fiducial collision helper function check, get distance from fiducial helper function :) |
| ||
 The top-left image shows the live tracking of "fiducial" objects. The filled blue circle under the mouse pointer denotes the pointer location is inside a fiducial object. In this example I was just pointing the camera at the screen, which was displaying a PDF of known fiducial symbols. Very interesting :-) Note : ofxFiducialFinder is GPL'd... (this seems to be quite common in this area of visualisation. On the other-hand, OpenCV itself is BSD). |
| ||
| I've added ofxCvHaarFinder support. The example shows face recognition - that is, recognising that there is a face in an image rather than recognising *who* is in the image. The routines are a bit of a resource-hog, so you may find it runs slow-ish on your computer. You could also test it by feeding it a photo which contains people's faces - it should be able to detect (and highlight) them. (see haarfinder example in the ofxOpenCv module) Fun fun... |
| ||
| Very neat. Thank you. :) *back to play with ofx* |
| ||
| Oh, cool! So far I've managed to do 2 things I've wanted to do: - Track a rolling golf ball - Use any surface such as a wall, floor etc as a touch screen of sorts I'll post some examples once I mature them a bit more. |
| ||
| @wmaas, openCV has some functions for tracking objects, which aren't currently implemented in openFrameworks. They may make tracking much easier to implement - given that they utilise all kinds of algorithms to do it. But, since I've only started touching on the surface of this project, I imagine there's all kinds of cool stuff to use. |
| ||
| I'm sure you are right. It would be interesting to see what I could do with more robust tracking functions, but I'm pleased with what I have been able to do so far with what you have implemented. |
| ||
| Brucey, I'm a bit new to the concept of Types and Methods. I would like to incorporate OpenFrameworks with the likes of LeadWerks Engine and MiniB3d. Would you be able to give me a few pointers as to how to structure the code? Typically everything goes on in a main loop but I am unsure how to reconcile the methods in OpenFrameworks with this. |
| ||
| I've got several fun projects going with this but here is something fun I knocked together with this stuff. http://www.youtube.com/watch?v=lZLs3BXiTak http://www.youtube.com/watch?v=LMXJIGMlePM |
| ||
| Hi Brucey What's the current statut of ofx module ? Do you still plan to finish it or is this module ready enough to be consistent ? Thanks |
| ||
| I've just checked in support for keyboard and mouse. This is demonstrated in the movieplayer example :  I've decided not to convert keys to OF key mappings, since it's just an extra conversion that we don't really need, as we already have a working solution in the keycodes module. (unless someone prefers otherwise?) Example (bmx): Method keyPressed(key:Int) Select key Case KEY_F frameByframe = Not frameByframe fingerMovie.setPaused(frameByframe) Case KEY_LEFT fingerMovie.previousFrame() Case KEY_RIGHT fingerMovie.nextFrame() Case KEY_0 fingerMovie.firstFrame() End Select End Method Example (c++) :
void testApp::keyPressed (Int key){
switch(key){
Case 'f':
frameByframe=!frameByframe;
fingerMovie.setPaused(frameByframe);
break;
Case OF_KEY_LEFT:
fingerMovie.previousFrame();
break;
Case OF_KEY_RIGHT:
fingerMovie.nextFrame();
break;
Case '0':
fingerMovie.firstFrame();
break;
}
}
As you can see, I think it maps fairly well as it is. |
| ||
| Nice! |
| ||
| Nice change :) I just read from here and there that KEY map is sometimes different from on platform to another. And that's maybe true since only from Windows to Linux functions keys are different. I can't say much about MacosX since I can only build the binaries and not really test this at regular speed. I should really think about a MacMini with Leopard if I want to support that platform but it's still pretty expensive for me at the moment. |
| ||
| Also checked in support for ofImage. Added a small "Processing" example port to show some interesting image use of a lens effect :  Of course, ofImage has much the same functionality as TPixmap/TImage, but, since it's part of the framework... |
| ||
| Keep it coming Brucey. |
| ||
| I don't know why but my webcam display in every example is up side down ? Which is not the case with a regular webcam display program. It wasn't the case few version before (I can't tell you exactly when but at least 3 months). By the way Brucey, what can you tell us about the Module status ? Do you exactly know what kind of ofx modules you are about to implement ? Would be nice to know in order to read a bit about every modules features and see what's currently doable. I'm still only playing with your examples for the moment. Thanks EDIT MoviePlayer example is also reverse while the black and white dot matrix one is correctly displayed. |
| ||
| I don't know why but my webcam display in every example is up side down That's completely my fault. Seems I lost my head when working on the ofImage support (which draws them upside down) and thought the best place to fix it would be in ofTexture - which it wasn't :-p I'll undo that "fix" and try to get ofImage working properly instead. |
| ||
| Okay, I've hacked ofImage to y-flip the FreeImage bitmap before loading it into the ofImage... Funnily enough, I also do this for pixmaps that are loaded by BaH.FreeImage. The included "processing" examples are quite interesting, I think. I've been porting some of these Java examples to openframeworks. Since openframeworks is meant to share some common ground with Processing, the porting process isn't too hard - although there is actually a lot more functionality in Processing. Hopefully this will change over time. |
| ||
| Nice :) I didn't know that ofx is capable to reproduce Processing examples. By more functionality over time you mean the official ofx framework roadmap or your ofx modules ? Would be great to be able to do exactly the same Processing stuff with ofx ! I just wonder how fast ofx rendering is compared to Max2d. If full realtime is still possible with game running in parallel |
| ||
| Have the dependencies changed at all for Linux? I'm getting these errors: /media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp:19:22: error: avformat.h: No such file or directory /media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp:20:21: error: swscale.h: No such file or directory Build Error: failed to compile (1) /media/system/blitzmax/mod/ofx.mod/core.mod/libs/openFrameworks/video/ofUCUtils.cpp Based on 'ofUCUtils.cpp', the includes have something to do with ffmpeg and Jaunty. EDIT: As suggested by the comments above the includes, commenting the first two made it compile.. EDIT2: Had to get all the Poco libs because they were imported in ofx.core. Now it says it can't find '-lGLee', which I cannot find. EDIT3: The second edit pertains to the example code, not the modules. The modules seem to build OK, but when trying to compile and run an example it freaks out about missing libraries. |
| ||
| Have the dependencies changed at all for Linux? Yeah, my fault. I'd fixed this in the previous update, but hadn't noted it anywhere ;-) Noted in core.bmx now - so I'll remember for next time! (hopefully) |
| ||
| Still can't get the example to work. I can't find Glee anywhere in Synaptic. This is the error message: /usr/bin/ld: cannot find -lGLee collect2: ld returned 1 exit status Build Error: Failed to link ~/blitzmax/mod/ofx.mod/ofxmsafluid.mod/examples/example_01 |
| ||
| Brucey, got any wisdom on how to get this to play nice with MiniB3d or other engine? |
| ||
| Since it's just a standard GL context it should play fine. In fact, if you look at the ofxglmax2d module, you'll see how I've implemented it. By default, openframeworks uses GLUT, but since we already have our own stuff in BlitzMax for dealing with GL, I thought it was more compatible for use to use the routines that are already available to us. (I realise it's a bit bare in there still, but the most useful methods are implemented :-) |
| ||
| Cool, I'll take a peak at ofxglmax2d and see what I can figure out. |
| ||
| Still can't get the example to work. I can't find Glee anywhere in Synaptic. I'll get there sometime. Applied yet another "fix" to it which seems to be more in the right direction. I'll take a peak at ofxglmax2d and see what I can figure out Note that there are already a couple of functions posted somewhere that let you switch between 2D and 3D view that I think Klepto made. (possibly search for max2d in the minib3d forum) |
| ||
| This looks promising... http://www.blitzbasic.com/Community/posts.php?topic=69542#784036 |
| ||
| Brucey, I updated to the latest, rebuilt everything. I had to go get bah.random to build core and now I'm getting this with my projects. Any idea? C:/Program Files/BlitzMax/mod/bah.mod/random.mod/random.debug.win32.x86.a(random.bmx.debug.win32.x86.o): multiple definition of `__bb_random_random' Warning: .drectve `/DEFAULTLIB:"LIBC" /DEFAULTLIB:"OLDNAMES" ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:uuid.lib ' unrecognized Warning: .drectve `-defaultlib:uuid.lib ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Warning: .drectve `-defaultlib:LIBCMT ' unrecognized Warning: .drectve `-defaultlib:OLDNAMES ' unrecognized Build Error: Failed to link C:\Program Files\BlitzMax\mod\ofx.mod\ofxopencv.mod\examples\myGui_7.debug.exe |
| ||
| Applied yet another "fix" to it which seems to be more in the right direction. Was r46 intended to work? You didn't add 'ofx.mod/core.mod/libs/GLee/lib/linux/libGLee.a' to the repository.EDIT: Tossed the library in there from http://nuicode.com/projects/tbeta/repository/revisions/169/changes/trunk/tbeta/Linux/libs/glee/libGLee.a and it worked. |
| ||
| multiple definition of `__bb_random_random I suppose I could substitute Bah.Random for BRL.Random - although the official Random module is not consistent across platforms (tsk tsk). It's part of the new perlin noise functionality - which I've implemented in BlitzMax, as opposed to C++. Maybe I should split it out into a separate module - yeah... probably better, since it's not really part of the original core framework. The other warnings, you get anyway. But they are just warnings. |
| ||
| You didn't add... Funnily enough, I added it to my project, then checked everything in. However, I forgot (as I often do) that .a files are by default not imported by subversion, so it wasn't actually picked up by the commit, and didn't show up in the project as a change. Quite annoying the .a thing. You have to specifically "add" them to the project. Oh well... sorry for the long, dragged out, round about process. My Linux is playing up, which I think doesn't help my temperament... |
| ||
| No worries Brucey, I backed out to the last working version I had. And I think I made some progress on the 3D integration by using MiniB3D with Oddball's stuff at the post below: http://www.blitzbasic.com/Community/posts.php?topic=69542#784036 Anyway, check it out.
Framework ofx.ofxopencv
Import ofx.ofxglmax2d
Import brl.glmax2d
Import "minib3d.bmx"
Graphics3D 1024,768
Global camera=CreateCamera()
Global light=CreateLight()
Global world=CreateSphere(12)
tex = LoadTexture("brick.jpg")
EntityTexture world,tex
Global myapp:TTestApp
PositionEntity camera,-5,0,-10 '***
PositionEntity world,0,0,0
PointEntity camera, world '***
myapp = New TTestApp
myapp.setup()
While Not KeyHit(KEY_ESCAPE) '***
TurnEntity world,0,1,0
RenderWorld
TGlobal.BeginMax2D
'SetBlend AlphaBlend
'SetAlpha(0.5)
'SetColor 255,0,0
'DrawText("Hello",50,50)
'SetBlend SolidBlend
myapp.update()
myapp.draw()
TGlobal.EndMax2D
Flip
Wend
End
Type TTestApp Extends ofBaseApp
Field vidGrabber:ofVideoGrabber
Field colorImg:ofxCvColorImage
Field grayImage:ofxCvGrayscaleImage
Field grayBg:ofxCvGrayscaleImage
Field grayDiff:ofxCvGrayscaleImage
Field contourFinder:ofxCvContourFinder
Field learnBackground:Int
Field threshold:Int
Method setup()
vidGrabber = New ofVideoGrabber
vidGrabber.setVerbose(True)
vidGrabber.initGrabber(320,240)
colorImg = New ofxCvColorImage
colorImg.allocate(320,240)
grayImage = New ofxCvGrayscaleImage
grayImage.allocate(320,240)
grayBg = New ofxCvGrayscaleImage
grayBg.allocate(320,240)
grayDiff = New ofxCvGrayscaleImage
grayDiff.allocate(320,240)
contourFinder = New ofxCvContourFinder
learnBackground = True
threshold = 80
End Method
Method update()
Local newFrame:Int = False
vidGrabber.grabFrame()
newFrame = vidGrabber.isFrameNew()
If newFrame Then
colorImg.setFromPixels(vidGrabber.getPixels(), 320,240)
grayImage.copy(colorImg)
If learnBackground = True Then
grayBg.copy(grayImage) ' copies the pixels from grayImage into grayBg
learnBackground = False
End If
' take the abs value of the difference between background and incoming and then threshold:
grayDiff.absDiff(grayBg, grayImage)
grayDiff.threshold(threshold)
' find contours which are between the size of 20 pixels and 1/3 the w*h pixels.
' also, find holes is set to true so we will get interior contours as well....
contourFinder.findContours(grayDiff, 20, (340 * 240) / 3, 10, False) ' find holes
End If
If KeyHit(key_escape) Then
End
End If
End Method
Method draw()
Local numBlobs:Int
Local blobArea:Float
Local centerx:Float
Local centery:Float
' draw the incoming, the grayscale, the bg And the thresholded difference
'ofSetColor(0xffffff)
colorImg.draw(20,20)
grayImage.draw(360,20)
grayBg.draw(20,280)
grayDiff.draw(360,280)
SetColor $33, $33, $33
DrawRect 360,540,320,240
' we could draw the whole contour finder
'contourFinder.draw(360,540)
' or, instead we can draw each blob individually,
' this is how to get access to them:
For Local blob:ofxCvBlob = EachIn contourFinder
numBlobs = numBlobs + 1
blob.draw(360, 540)
blobArea = blob.getArea()
blob.getCentroid(centerx, centery)
Next
SetColor 255, 255, 255
DrawText numBlobs, 10, 10
'DrawText blobArea, 10, 20
DrawText centerx, 10, 30
DrawText centery, 10, 40
End Method
End Type
|
| ||
| I lost track of that module ! Lately I wanted to learn a bit of processing but remember about something similar in BlitzMax. What is the statut of ofx.mod ?? I try to build the fluid example but fail, most of the core examples still run correctly. Last news on the official website is from 2009 :s Maybe better to stick to real Processing ? |
| ||
| The OpenCV examples fail. |
   |