kmando human computer interface

System Software

Source (link to git-repo or to original if based on someone elses unmodified work): Add the source-code for this project on opencode.net

3
Score 63.3%
Description:

A software for camera-projector interaction was developed by Juan Roldan, Ushakiran Soutapalli, Julien Faucher, and Jan Wedekind at the Microsystem & Machine Vision Laboratory (MMVL) at Sheffield Hallam University, UK.

The software makes use of a low cost off-the shelf webcam that is calibrated against a standard projector screen (note that at the moment the new video for linux standard version 2 (V4L2) is not supported yet). The webcam is used to determine the position of physical pointer (e.g. a pen) which is then used to virtually move the X11 pointer. Point-and-click functionality has also been implemented.

The wizard allows you to take a reference picture showing the pointer. You then have to select the region in the image for generating a reference histogram. If you are using a projector, make sure that the desktop has a bright background. Otherwise the pointer will be illuminated differently and it will not be recognized any more.

Donnie Bhayangkara has built a package of mando-1.6 for openSuSE 10.3 which can be downloaded here:
http://software.opensuse.org/search?baseproject=openSUSE%3A10.3&p=1&q=mando

For a video see
http://video.google.com/videoplay?docid=4201589281466176805

Mando now has a new project page at Sourceforge:
http://mando.sourceforge.net/

New releases at: http://sourceforge.net/project/showfiles.php?group_id=220882

Git access: git clone git://mando.git.sourceforge.net/gitroot/mando mando

Discussion Forum: http://sourceforge.net/forum/?group_id=220882
C

wedesoft

13 years ago

The artefacts caused by the scanning electron beam used to make calibration difficult. Newer versions of kmando are using a white pattern on a black background for calibrating (instead of black on white) and it is much more stable. I have just now successfully tested kmando on my LCD monitor, which runs at only 60Hz.

Report

trevi55

13 years ago

Cool... When will you release it? :P
I'd like to test it :P

PS: i've an ubuntu package if you want, but if you release soon...

Report

C

wedesoft

13 years ago

There is a misunderstanding here ;)
All newer releases of kmando already do have white-on-black patterns.
Great to hear that there'll be a package for Ubuntu as well!

Report

trevi55

13 years ago

Sorry for misunderstanding, but I can't make it work on my laptop's LCD...
I've the same refresh rate here, but maybe I do something of wrong...

I can't find good docs about using kmando...

I use the searchpattern and projectedPattern.png you provided, but I can't understand which slide I should choose...

What's wrong?

Bye!

Report

C

wedesoft

13 years ago

I don't understand what you mean with "slide".
You need to click "Configure" to set up the application first.
Do you get a camera image already?

Report

C

wedesoft

13 years ago

> I meant that there are some images generated durning the calibration, with some square positioned in different positions of the screen (angles and center)... I can't understand wich one should I choose...

You don't need to choose anything for the screen calibration. You just have to make sure, that the camera can "see" the
whole screen. You can then check (by moving the scrollbar), wether each of the 5 positions was recognised successfully (i.e. location of red cross).
Only then you can grab an image with the pointer in it. Then you have to select the pointer in the image by mouse.

> Just an idea... Could you make possible to select change an offset?

That's a good idea. I may implement it, when I get the time to do it.

> Another thing... Could be disabled the "simulated click", using a keyboard key (or shortcut) instead.

Under X11 one can press "Shift+Num Lock" and then control the mouse using the arrow-keys. The "+" and "-" key become mouse-buttons.

> Ops... Sorry for this huge-multiple-posting...
LOL! A lot of people are doing this on kde-apps recently.

Report

trevi55

13 years ago

I meant that there are some images generated durning the calibration, with some square positioned in different positions of the screen (angles and center)... I can't understand wich one should I choose...

Btw, now I got something better... It seems to work, not so well but my mouse cursor follow quite well my pointer...

Just an idea... Could you make possible to select change an offset? I mean, the cusor is some pixel top-bottom than the pointer, maybe a manual resetting coul help...

Another thing... Could be disabled the "simulated click", using a keyboard key (or shortcut) instead?

Report

trevi55

13 years ago

Ops... Sorry for this huge-multiple-posting... Unfortunately my session-saver did this on restore (it saves post data... :/)

Report

trevi55

13 years ago

I meant that there are some images generated durning the calibration, with some square positioned in different positions of the screen (angles and center)... I can't understand wich one should I choose...

Btw, now I got something better... It seems to work, not so well but my mouse cursor follow quite well my pointer...

Just an idea... Could you make possible to select change an offset? I mean, the cusor is some pixel top-bottom than the pointer, maybe a manual resetting coul help...

Another thing... Could be disabled the "simulated click", using a keyboard key (or shortcut) instead?

Report

trevi55

13 years ago

I meant that there are some images generated durning the calibration, with some square positioned in different positions of the screen (angles and center)... I can't understand wich one should I choose...

Btw, now I got something better... It seems to work, not so well but my mouse cursor follow quite well my pointer...

Just an idea... Could you make possible to select change an offset? I mean, the cusor is some pixel top-bottom than the pointer, maybe a manual resetting coul help...

Another thing... Could be disabled the "simulated click", using a keyboard key (or shortcut) instead?

Report

trevi55

13 years ago

I meant that there are some images generated durning the calibration, with some square positioned in different positions of the screen (angles and center)... I can't understand wich one should I choose...

Btw, now I got something better... It seems to work, not so well but my mouse cursor follow quite well my pointer...

Just an idea... Could you make possible to select change an offset? I mean, the cusor is some pixel top-bottom than the pointer, maybe a manual resetting coul help...

Another thing... Could be disabled the "simulated click", using a keyboard key (or shortcut) instead?

Report

sirjoe

13 years ago

Having a little bit of a problem here
can't wait to get this working :)

basically I go through the 5 lines in the terminal. but when i write "make" it doesn't like me:
"no targets specified and no make file found. stop.

Not quite sure.
I'm pretty much totally beginning with all with linux stuff.
love it tho

any help would be much appreciated

Joe

Report

sirjoe

13 years ago

Having a little bit of a problem here
can't wait to get this working :)

basically I go through the 5 lines in the terminal. but when i write "make" it doesn't like me:
"no targets specified and no make file found. stop.

Not quite sure.
I'm pretty much totally beginning with all with linux stuff.
love it tho

any help would be much appreciated

Joe

Report

C

wedesoft

13 years ago

This is the sequence of commands:

tar xjf mando-*.tar.bz2
cd mando-*
./configure
make
./mando

If "make" fails this means, that the preceeding "./configure" step has failed and didn't create the makefiles. You need to post us the output of the configure-script here. Most probably some software packages are missing on your system.

Report

martijn

13 years ago

Hello.

I am trying to get this program to work on my TFT screen. Installation went fine (on Arch Linux with the latest source from this site) and when I start Kmando I go to 'Configure'. I can see the image of my webcam and I point it to see my whole screen. I calibrate the screen and it correctly detects the 5 centerpoints (red cross). When I press next again it asks me to "Capture a reference image for color segmentation".
I can grab an image but I can't press next. Why?

Thanks in advance!

Report

martijn

13 years ago

Hello.

I am trying to get this program to work on my TFT screen. Installation went fine (on Arch Linux with the latest source from this site) and when I start Kmando I go to 'Configure'. I can see the image of my webcam and I point it to see my whole screen. I calibrate the screen and it correctly detects the 5 centerpoints (red cross). When I press next again it asks me to "Capture a reference image for color segmentation".
I can grab an image but I can't press next. Why?

Thanks in advance!

Report

C

wedesoft

13 years ago

Sorry, I didn't document this properly. You need to select the image of the pointer before you can proceed:
http://vision.eng.shu.ac.uk/mediawiki/index.php/Image:Interactive2.jpg

Report

martijn

13 years ago

Thanks that did the trick, great app btw!

Report

C

wedesoft

13 years ago

I've added an explanation to the graphical user-interface because many users were stopped by this problem.

Report

martijn

13 years ago

Hello.

I am trying to get this program to work on my TFT screen. Installation went fine (on Arch Linux with the latest source from this site) and when I start Kmando I go to 'Configure'. I can see the image of my webcam and I point it to see my whole screen. I calibrate the screen and it correctly detects the 5 centerpoints (red cross). When I press next again it asks me to "Capture a reference image for color segmentation".
I can grab an image but I can't press next. Why?

Thanks in advance!

Report

dutler

13 years ago

hi, looks pretty awesome, cant wait to try it (need projector first).

can kmando use two cameras so the presentator can move around more with out blocking the view?

what kind resolution of movement can it detect?

many of the pict show a stylus with a bright orange tip. is it needed?

Report

C

wedesoft

13 years ago

Sorry about responding so late. I missed your posting due to loads of spam.
kmando can not use two cameras.

We are using a Logitech Quickcam 4000 with a resolution of 640x480. I don't remember the noise values, but the accuracy was sufficient to click on buttons with a size of 30x30 pixels (the screen resolution was 1280x1024). It depends on the camera resolution and how well you can position the camera.

We are using colour segmentation to detect the pointer. Therefore you need a pointer with a colour which is NOT present in the image. Recognition of the neon-colours of the marker-pen proved to be very robust. Maybe it is possible to use a laser-pointer as well. Unfortunately I didn't have the time to try this.

You don't need a projector. I also managed to make the application work with a TFT-monitor and a pencil (my finger unfortunately does not have a distinct colour). But on a small screen it is not so comfortable to use. Also it is difficult to position the camera. With a projector however it works much more stable.

Report

dutler

13 years ago

NO apology is needed for me and thank you for answering all my questions. I do have a follow up with the stereo vision and resolution, but first, why the questions.

at my school we use quite a few "smart boards" - touch sensitive marker boards - in the class room. they are junk and proprietary (a redundancy?)

to replace with a solution such as kmando, we need to be able to stay out of the shadow and high enough res to use handwriting recognition.

i understand kmando may not be a drop in solution (free as in freedom not free as in beer), but i trying to decide if it could be a solution.

the stereo seemed like a cheap (hardware) idea to take care of the shadow but it may increase the resolution of control too. Is the architecture incompatible with stereo, or the current version does not support it?

Report

C

wedesoft

13 years ago

Of course a touch sensitive marker board does not have the problem of a person blocking the view of the sensor. Also kmando cannot detect, wether you are touching the screen or not. Mouse-clicks are issued when the cursor is not moving for an adjustable period of time. At the moment the application has two modes: one for issuing mouse-clicks and one for drag-and-drop. I think one can use kmando for giving presentations. But if you want to write on a screen like on a chalk board, the current system is not sufficient (believe me, I tried).

The problem with using two webcams is that most Linux drivers are only tested with a single camera. Capturing from two Logitech cameras didn't work. However capturing from a bttv card (which has 8 channels) did work. So our grabbing software seems to be capable of grabbing from multiple cameras. It also may be possible already to launch two instances of kmando. But ideally you would want a single application with an algorithm for sensor-fusion. The current version does not support this.

My colleague had the idea to use a Linux-compatible laser pointer with usb-remote (for example see http://www.lasersale.com/catalog/display_products.asp?cat=89) . But as I said, so far I didn't test laser-pointers at all. If anyone tries a laser-pointer: Please let me know, wether it works.

Report

tibssy

13 years ago

checking for boost... yes
checking for OpenGL... yes
checking for GL... no
checking for GLU... no
checking for glut... no
configure: error: Check for OpenGL failed.

Report

12345678910
2
product-maker schleby Mar 07 2011 9 excellent
product-maker jrepin Nov 29 2010 9 excellent
product-maker Base: 4 x 5.0 Ratings
Pelandritus
Aug 15 2010
Tids
Jun 29 2009
carterson2
Apr 21 2009
File (click to download) Version Description PackagetypeArchitectureRelease Channel Downloads Date Filesize DL OCS-Install MD5SUM
*Needs pling-store or ocs-url to install things
Pling
0 Affiliates
Details
license
version
1.7
updated Jun 29 2009
added Oct 11 2006
downloads 24h
0
mediaviews 24h 0
pageviews 24h 3
System Tags app software