Face detect and UDP

Blitz3D Forums/Blitz3D Userlibs/Face detect and UDP

ZJP(Posted 2009) [#1]
Hi,
You probably know Johnny Lee, a researcher at Carnegie Mellon who has fun with the wiimote. See here. http://www.youtube.com/watch?v=Jd3-eiid-Uw

I make a small webcam/application that after detection of a face moves the position of it via UDP.
Here http://www.zinfo972.net/avatar/FACE_UDP.RAR
Need help for the Blitz test ;-)

JP
PS: it's not a..."trojaner" topic. ;-)



;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Graphics3D 800,600,32,2
AppTitle "Face UDP Detect 3D "
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; Face UDP (c) zjp+++++++++++++++++++++++++++++++++++++++++
Global recu$
Global UDP_Serveur
Global VX=160
Global VY=120
Global VR=50
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
; UDP read face detect
Function recu_UDP()
	;recu$=""
	IP_Client = RecvUDPMsg(UDP_Serveur)
	If IP_Client Then 
		recu$ = ReadLine(UDP_Serveur)
		; parce
		; found X
		i=Instr(recu$,"X(",1)+2
		C$=Mid$(recu$,i,4)
		VX=C$
		; found Y
		i=Instr(recu$,"Y(",1)+2
		C$=Mid$(recu$,i,4)
		VY=C$
		; found R
		i=Instr(recu$,"R(",1)+2
		C$=Mid$(recu$,i,4)
		VR=C$
	End If 
End Function
UDP_Serveur = CreateUDPStream(26000) ; <<<<<<< Quake UDP PORT ;-)
If UDP_Serveur <>0 Then 
	Print "Server Start."
Else
	Print "Can't start Server"
	End
End If
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
;++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
light=CreateLight()
cam=CreateCamera()

Type box
	Field xpos#,ypos#,zpos#
	Field spinspeed#
	Field entity
	Field alpha#,r,g,b
End Type

CubeType=CreateCube()
For x= 1 To 6 
	For y = 1 To 6
		For z = 1 To 6
			b.box=New box 
			b\entity = CopyEntity(CubeType)
			b\xpos#=x*4
			b\ypos#=y*4
			b\zpos#=z*4
			rc=Rnd(10,200)
			gc=Rnd(10,205)
			bc=Rnd(10,205)
			EntityColor b\entity,rc,gc,bc
			EntityAlpha b\entity,.8
			PositionEntity b\entity,b\xpos#,b\ypos#,b\zpos#
		Next
	Next
Next
;
MoveEntity cam,12,10,0
HideEntity CubeType 
;
Repeat
	; move the cam. Fx to build here
	RotateEntity cam,vy/6,-vx/6,0 ; dummy test
	;++++++++++++++++++++++++++++++++++++++++++
	UpdateWorld
	RenderWorld
	recu_udp ; get Face position
	Text 10,10,"Face x "+VX+", Face Y "+VY+", Face Radius "+VR
	Flip
Until KeyHit(1)=1
End



Charrua(Posted 2009) [#2]
Hi

Are you doing head-tracking based on image recognition with a web cam?

I supose you are evaluating differences with the actual frame with tespect of the previous.

Why don't use the wiimote?

I'm not at home now, let me download, do a test with my webcam and will see if I can help in some way

edit
hey, cool!
works fine, let me test different conditions like light, contrast, distance.
Are you testing some feature in particular?


best regards


Juan


ZJP(Posted 2009) [#3]
Hi,
"...Are you doing head-tracking based on image recognition with a web cam?.."
> Yes

Need some help to build a "fantastic" demo with Blitz3D. I'm not a ... Blitz3D guru ;-)

JP


SytzeZ(Posted 2009) [#4]
It doesn't work on my webcam :(
It's all black in the webcam window


Nate the Great(Posted 2009) [#5]
it works here! very cool... maybe you could make it so that it can do that perspective thing that guy on youtube did with the wii remote

edit oops didnt see that :p good work

perhaps this should go in the codearchives?


Charrua(Posted 2009) [#6]
is easiest to do with the wiimote, it sends 100 packets of coord's info each second (position of the led's inside the sensor bar). The code johnny gives is so simple to translate to blitz, the math part very short the rest is easily done with the 3d blitz engine.

I'm geting poor frames per second but i'm testing your face detect in my laptop (an ancient one), i'll try nex week in a newer (obiously not mine!)

best regards

Juan


ZJP(Posted 2009) [#7]
"...It doesn't work on my webcam :(..."
This program use an UDP/port : 26000. Check your firewall

"..I'm geting poor frames per second.."
Updated. Speed of detection has improved.

The aim of the program is not to replace the wiimote. Wiimote does not detect faces.You can use it to detect your "sleep" ;-))

JP


Charrua(Posted 2009) [#8]
I try to simulate a face with a ball, and other mimics and your soft didn't fail!
If one put a hand or other thigs in the face, the soft stop detecting the face, that sort of things was the first I try to do, but with the intention of test seriously your work. I supose you are watching some particullar aspects of the face... (2 eyes, 1 nose, .. and the like)

about speed, i supose that was not the focus of your post.

and of course, face detection nothing has to do with the wiimote, great job.

I'll continue to see, what are the important aspects in your face recogn.

Juan


Nate the Great(Posted 2009) [#9]
i made a pong game you control with your face :)

here is the code.. very basic and messy tho :( I am trying to make a callibrator for each kind of camera. you may have to tweak this code to make it work for your camera..



can someone test this to see if it works the same on every computer. Just move your face left and right and it moves the pong paddle :)

Need some help to build a "fantastic" demo with Blitz3D. I'm not a ... Blitz3D guru ;-)



@ZJP - i will help if you want me to... wow i have made at least 3 of my previous games head controlled :) this is fun


Guy Fawkes(Posted 2009) [#10]
can u guys add the camera part to this so it works for my webcam via udp?


Nate the Great(Posted 2009) [#11]
what do you mean by adding a camera part?


Guy Fawkes(Posted 2009) [#12]
so that it actually turns on my webcam, and reads my face, then moves the platform w/ my face movement, as u said above?


Nate the Great(Posted 2009) [#13]
oh you must download and run the camera program in the first post in this thread. When you exit the window, it makes itself a process and keeps your camera on as well as doing facerecognition constantly

Here it is
http://www.zinfo972.net/avatar/FACE_UDP.RAR


Guy Fawkes(Posted 2009) [#14]
can u release the source so i can try to mod it if needed? if u need my email, ill be glad to give it to u :)


Nate the Great(Posted 2009) [#15]
oo im with darkshadowwing on this one. any chance you will release the source?


SytzeZ(Posted 2009) [#16]
It doesn't work with my Webcam :(
Some random dots in random colors appear in the screen.
I don't know what kind of webcam it is, because it's built in to my laptop screen.


Guy Fawkes(Posted 2009) [#17]
u need to download the file in the 1st post on this topic, THEN compile the source. i really wish he will release this source. it would be a great contribution to the community. besides, we might be able to help make it better! :)


ZJP(Posted 2009) [#18]
"...u need to download the file in the 1st post on this topic, THEN compile the source. i really wish he will release this source. it would be a great contribution to the community. besides, we might be able to help make it better! :) ..."

This is not a BlitzBasic source. Build in C++
I'm working on extra options. Webcam number ,ip number (local or not), udp port. It can detect a face via Wan or Lan

JP


Guy Fawkes(Posted 2009) [#19]
could u turn it to a dll THEN we could create the userlib for it, and create it? :)


cyberyoyo(Posted 2009) [#20]
AHAHA
and the first person you test on is francky vincent
ahahaha :))


ZJP(Posted 2009) [#21]
;-)


Panno(Posted 2009) [#22]
zjp how to capture a frame while the app isnt in the front ?

avicap stopps if he window invisible ! say me the trick please


Guy Fawkes(Posted 2009) [#23]
can u release the source ZJP? I have a friend who can change it from C++ to BlitzBasic


Ross C(Posted 2009) [#24]
I dare say blitzbasic would be too slow to do this.


Guy Fawkes(Posted 2009) [#25]
maybe. but i wanna see if i can edit it to run w/ both tcp OR udp. It IS a bit slow, but I don't mind the slowness.


ZJP(Posted 2009) [#26]
Hi,
Need OpenCV ;)

//**********************************************************************
//**********************************************************************
#include "cv.h"
#include "highgui.h"
#include <stdio.h>
// SOCKET **************************************************************
#include <winsock.h>
#define PORT 26000 // Quake port
#define IP "127.0.0.1"
SOCKET sock;
char result[100];
int nb_octet;
int nb_detect;
// version freeware *********************************
int nb_test1=110;
int nb_test2=145;
// version freeware *********************************
struct sockaddr_in si_client = { 0 };
// OPENCV **************************************************************
//**********************************************************************
static CvMemStorage* storage = 0;
static CvHaarClassifierCascade* cascade = 0;
IplImage* gray;
IplImage* small_img;
CvPoint pt1;
CvFont font;
int line_type = CV_AA; 

void detect_and_draw( IplImage* image );
//**********************************************************************
const char* cascade_name = "FACE.DAT";
/*  "haarcascade_frontalface_alt2.xml";*/
int main( int argc, char** argv )
{
    CvCapture* capture = 0;
    IplImage *frame, *frame_copy = 0;

    cascade = (CvHaarClassifierCascade*)cvLoad( cascade_name, 0, 0, 0 );

    if( !cascade )
    {
        return -1;
    }
    storage = cvCreateMemStorage(0);
    capture = cvCaptureFromCAM( -1 );

    cvNamedWindow( "FACE/UDP", 0 );
    cvInitFont( &font, CV_FONT_HERSHEY_COMPLEX, 0.3, 0.3, 0.0, 1, line_type );
	nb_detect=0;

	// INIT SOCKET *********************************************************************
	// INIT SOCKET *********************************************************************
	WSADATA WSAData;
	int erreur = WSAStartup (MAKEWORD (2, 0), &WSAData);

 	sock = socket (AF_INET, SOCK_DGRAM, IPPROTO_UDP);
	if (sock != INVALID_SOCKET)
	{
      /*Initialisation de la structure contenant l'adresse de l'hote et le port utilise */
      si_client.sin_family = AF_INET;
      si_client.sin_port = htons (PORT); //conversion du numero de port
      //conversion de l'adresse ip
      si_client.sin_addr.s_addr = inet_addr (IP);

      printf ("IP ET PORT %s:%d\n", IP, PORT);
   }
   else
   {
      printf ("ERREUR SOCKET\n");
      /*Fermeture de la socket */
      (void) shutdown (sock, 2);

       cvDestroyWindow( "FACE/UDP");
	   cvReleaseCapture( &capture );

      return -1;
   }
	// INIT SOCKET FIN ******************************************************************
    for(;;)
    {
       if( !cvGrabFrame( capture ))
          break;
       frame = cvRetrieveFrame( capture );
       if( !frame )
       break;
       if( !frame_copy )
          frame_copy = cvCreateImage( cvSize(frame->width,frame->height),IPL_DEPTH_8U, frame->nChannels );
       if( frame->origin == IPL_ORIGIN_TL )
          cvCopy( frame, frame_copy, 0 );
       else
          cvFlip( frame, frame_copy, 0 );
 
       detect_and_draw( frame_copy );

       if( cvWaitKey( 60 ) >= 0 )
         break;
    }
	cvReleaseImage( &gray );
	cvReleaseImage( &small_img );
    cvReleaseImage( &frame_copy );
    cvReleaseCapture( &capture );

    cvDestroyWindow("FACE/UDP");
   /*Fermeture de la socket */
   (void) shutdown (sock, 2);
   WSACleanup ();

    return 0;
}
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// DETECTION DU VISAGE ++++++++++++++++++++++++++++++++++++++++
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
void detect_and_draw( IplImage* img )
{
    static CvScalar colors[] =
    {
        {{0,250,155}},
     };

    double scale = 2.5; // Sensibilite 1.0 a 3.0 par exemple.
    int i;
	if( !gray )
    	gray = cvCreateImage( cvSize(img->width,img->height), 8, 1 );
	if( !small_img )
    	small_img = cvCreateImage( cvSize( cvRound (img->width/scale),cvRound (img->height/scale)), 8, 1 );

    cvCvtColor( img, gray, CV_BGR2GRAY );
    cvResize( gray, small_img, CV_INTER_LINEAR );
    cvEqualizeHist( small_img, small_img );
    cvClearMemStorage( storage );

        CvSeq* faces = cvHaarDetectObjects( small_img, cascade, storage,1.1, 2, 0,cvSize(30, 30) );
		pt1.x=0;
		pt1.y=235;

		cvPutText( img, "(c)zjp@...", pt1, &font, CV_RGB(150,150,150));

        if (faces->total)
        {
		 	nb_detect++;
			if (nb_detect> nb_test1*nb_test2)
			    return;
			i=0;
            CvRect* r = (CvRect*)cvGetSeqElem( faces, i );
            CvPoint center;
            int radius;
            center.x = cvRound((r->x + r->width*0.5)*scale);
            center.y = cvRound((r->y + r->height*0.5)*scale);
            radius = cvRound((r->width + r->height)*0.25*scale);
            cvCircle( img, center, radius, colors[i%8], 3, 8, 0 );

			/*Envoi du message */
			sprintf( result,"X(%d) Y(%d) R(%d) (c) zjp@...);
			nb_octet = sendto (sock, result, strlen (result) + 1, 0,(struct sockaddr *) &si_client,sizeof (si_client));
 			if (nb_octet > 0)
		       {
			   		// printf ("%d octets envoyés\n", nb_octet);
			   }
			else
			   {
			       //printf ("sendto() : erreur !!!\n");
			   }
        }
    cvShowImage( "FACE/UDP", img );
}
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
//+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




Guy Fawkes(Posted 2009) [#27]
what about highgui? or does that come w/ cv?


ZJP(Posted 2009) [#28]
Yes ;)
http://opencv.willowgarage.com/documentation/highgui.html


Guy Fawkes(Posted 2010) [#29]
how EXACTLY do you compile this in Dev-C++ 4.9.9.2 (beta) ?


ZJP(Posted 2010) [#30]
http://opencv.willowgarage.com/wiki/DevCpp
http://qevan.blogspot.com/2006/04/howto-setup-opencv-rfc-cambridge-wiki.html

By me from a french website.( http://www.developpez.net/forums/d368580/c-cpp/bibliotheques/opencv/informations-utiles-opencv/ ) Sorry. ;) ;)
Un rappel de la procédure a suivre pour configurer Open CV sous DEV C++


OpenCV Version 1.0 avec DevCPP

0) Lancez le programme d'installation (OpenCV_1.0.exe) que vous venez de télécharger. Acceptez toutes les options par défaut ; cela va installer OpenCV dans C:\Program Files\OpenCV.

1) Ajoutez les répertoires C:\Program Files\OpenCV\bin et C:\Program Files\OpenCV\otherlibs\highgui à votre variable d'environnement PATH.
Cela va indiquer à Windows où trouver les fichiers DLL.
Pour ce faire, allez dans "Menu démarrer" -> "Panneau de configuration" -> "Système" -> "Avancé" -> "Variables d'environnement". Dans la boîte de dialogue tout en bas, cherchez la variable nommée PATH.
Editez la. Ajoutez-y "C:\Program Files\OpenCV\bin;C:\Program Files\OpenCV\otherlibs\highgui" (n'oubliez pas les points-virgules pour séparer). Validez.

2) Ouvrez Dev-C++.

Le fichier de test. Allez dans le menu "Fichier" -> "Ouvrir projet ou fichier" et ouvrez C:\Program Files\OpenCV\samples\c\contours.c.
Créez un nouveau fichier ("Fichier" -> "Nouveau" -> "Fichier source"). Copiez-collez le code de contours.c dans ce fichier et sauvegardez le en tant que test.cpp.

3) Allez dans "Outils" -> "Options du compilateur". Il devrait s'ouvrir avec l'onglet "Compilateur" sélectionné. Cochez le bouton "Ajouter ces commandes à la ligne de commande de l'éditeur de liens" et tapez la ligne suivante dans la boîte de saisie :
-lhighgui -lcv -lcxcore -lcvaux -lcvcam

Cela indique à l'EDI de lier votre programme avec les bibliothèques d'OpenCV.

4) Rendez-vous maintenant dans l'onglet "Répertoires", et dans le sous-onglet "Bibliothèques" ajoutez aux répertoires le suivant :
C:\Program Files\OpenCV\lib

5) Passez à l'onglet "En-têtes". Ajoutez :

C:\Program Files\OpenCV\cv\include
C:\Program Files\OpenCV\cvaux\include
C:\Program Files\OpenCV\cxcore\include
C:\Program Files\OpenCV\otherlibs\highgui
C:\Program Files\OpenCV\otherlibs\cvcam\include

6) Faites de même pour l'onglet "En-têtes C++".

7) Compilez.



atv(Posted 2016) [#31]
ZJP do you have an example of how to use this in blitzmax? I'm specifically interested in how to put the return value of the cvshowimage pointer on a existing window in blitzmax. I dont want to create a new window of it's own.


ZJP(Posted 2016) [#32]
Hi,

Sorry, no example for or from BlitzMax.
For the pointer (to or from an Wndows Handle) maybe : https://www.google.fr/#safe=strict&q=opencv+windows+handle

// get its handle
HWND wnd = (HWND)cvGetWindowHandle("windowname");
Source : http://code.opencv.org/issues/1189

Good luck an Happy New Year. ;)