Edited by author 07-20-2006 05:09 PM
We just released the beta version of the labeling tool. Please try it out and let us know what you think about it! Things we've changed since the last version:

-Added a zoom feature so it's easier to annotate smaller objects
-Made the labeling tool compatible with Firefox, IE, and Safari
-Changed the way that the outlines are drawn
-Added capability of removing individual line segments
-Made the layout more intuitive

Is there a delay between labelling an object and the database being updated? I labelled some giraffes yesterday but a search today on "giraffe" doesn't show the ones I labelled. I wanted to check that everything was working before I did more images, but now I'm not sure.

Yes, there is a delay between labeling an object and when you are able to search for the object through the online search tool. However, the annotations should be recorded. You can verify this by revisiting the image that you annotated (the giraffe annotations should be there).

To make sure that everything is ok, did you label the following image?

http://labelme.csail.mit.edu/tool.html?col...811_144650_GM6A0142

Thanks for the reply - yes, that's the image I annotated, and the giraffes also now show up in a search on "giraffe".

I suggest the possibility to specify occlusions. It could be done like in Illustrator or other programs where you can order objects back or forward stepwise. In the end there would be an ordered list and if two regions overlap and one appears before the other in the list then it occludes it.

Daniel, thanks for your feedback and suggestion. While the tool does not explicitly provide a mechanism for labeling depth orderings, one can achieve this by appending to the label field of an annotation "depthXX". During post-processing with the Matlab toolbox, you can then extract this information and develop functions to display the depth information.

Also, there is quite a bit one can do with the annotations already to determine depth ordering. We will soon release a Matlab function that suggests a depth ordering for the polygons in an image based on some simple heuristics. This function seems to work quite well in practice.

We agree that depth information is very important and may try to include explicit labeling instructions in a future version of LabelMe.

Again, thanks for your comments!

Hello, in the first place I would like to say congratulations, this is a great idea. I'm working in image segmentation, and I'm concerned about the lack of quantitative results in this field. However there aren't many labeled databases, particularly for segmentation. I know the aim is slightly different than object recognition, but some tools like this one can be re-utilized. I was wondering if you ever thought about that, and if you have any ideas about this.
Thanks in advance.
Juan

I'm experiencing a strange behaviour. As I label objects the counter advances, but after I change the image, it is back to the original number.
Is that normal, and only due to some delay?
Are my labels being stored?
Thanks.
       Juan

Juan, thank you for your feedback and for reporting the bug. Yes, all of your annotations are being stored safely. There was just a small bug with the counter. We have since fixed this problem, so hopefully the counter will not decrease anymore.

As for using the dataset for segmentation evaluation, we agree that this could be an interesting way to use the dataset. We have not given this much thought and have not pursued this direction. Thanks for raising this issue!

Ok. Thank you very much for your quick response. By now im annotating images just for fun, but as soon as I get some time I plan to give that idea a deeper thought. It has to be possible, since the worst part of evaluating a segmentation is also getting the ground truth.
Thanks again.

How about putting up the PASCAL:VOC 2007 images. Are there any copyright issues? I intend to segment extra images for my entry and they might as well be publicly available.

Alastair, thanks for you interest in the LabelMe project. Unfortunately we cannot post the PASCAL VOC 2007 images on LabelMe at this time.

I need to know if you guys are still working on Labelme or not. I had read about Label Me in some academic paper and we were working on more or less similar project in Nayang Tech in singapore [Called SYGY]. But I doubt if Label Me is still alive or the project is ended.
I checked the tool, and it works fine, but I really don't know if still people are welcomed to contribute?
And one more.
How can I find out if I have got response on this discussion page? there is no notifier , seems I have to get back sometime later.
Alireza Kashian @ Singapore

Thank you for your interest in the LabelMe project. Yes, LabelMe is alive and well. You are more than welcome to contribute images, annotations, or code. If you want immediate correspondence, please contact us by e-mail directly as only messages can be posted to this discussion space.

Hello! great idea of color of this siyte!

Hey , i wanted to some details abt ur labelme..
Firstly, wich language are u ppl using ....
Secondly, How can i query
Thirdly do we have to label eac image for querying.... ??

The answers for your questions can be found via links from the main LabelMe page (http://labelme.csail.mit.edu). More specifically, check out the (1) tool source code and (2) search objects/scenes links for answers regarding source code and object queries.

I can't seem to figure out how to use MatLab toolbox to be able to query the images using WordNet. Any help would be appreciated. Thanks!

Nevermind, I found the demoWordnet.m file.
This is great!

I just labeled one image. It's a nice interface. Suggestions: (1) support for Safari (Mac OS X), (2) you could also just show an image + existing contours (without labels) to the user (either created by previous users or computed via a segmentation algorithm) and ask them to either reject the contour or provide their own labels for it. (3) you could do the reverse too: show them just the image + a label (obtained from a previous user), and ask them to either state that no such object exists in the image, or contour that object... at the end, you can score the contour using an edge detector (or similar)... this way, you could perhaps improve/reject the badly drawn contours, and get rid of "maliciously" drawn objects which don't actually exist in the image.

Thanks for contributing labels and for your suggestions. In response to your suggestions:

(1) We are close to having Safari support but have not completed this yet. If you urgently need Safari support, then we encourage you to download the annotation tool source code: http://labelme.csail.mit.edu/Documentation/sourceCode.html

(2,3) Quality control in LabelMe is certainly an important issue. However, we have found that users generally provide good labels (c.f. our paper on the LabelMe database: http://people.csail.mit.edu/brussell/resea...IM-2005-025-new.pdf).

Hi! Nice site!

Hi! Is there any way we can obtain a list of images that contain an object of interest. For example, a query which gives me the list of all images on the database that contain at least one car.

Yes, you may query using the Matlab toolbox:

HOMEANNOTATIONS = 'http://labelme.csail.mit.edu/Annotations';
D = LMquery(LMdatabase(HOMEANNOTATIONS), 'object.name', 'road');

The structure "D" will have the list of folders and images containing at least one car.

Thanks! This is great work!

Edited by author 01-31-2008 01:46 PM
Partialy visible object labeling

This question about labeling probably has been answered already, but you may want to post clear instruction how to draw the lines around the object if it is only partially visible (eg. chair only legs are seen, or plate or car when only quarter is seen, there is a person in front of the car, or only half of fridge is seen in the kitchen - creates triangular fridge). This creates weird geometric shapes and may may not really help you all that much. Garbage in garbage out, you know the old saying.... But other than that, keep up the great job.

Q: Should you label only what is visible even if it is partially occluded, or should you label the object as it should be seen.

Thanks for your question and feedback.

We give general labeling guidelines here:

http://labelme.csail.mit.edu/guidelines.html

In general, we prefer that you label objects as if they were not occluded, whenever possible. When you do this, you can reliably infer the depth-ordering of the objects (c.f. one way of doing this is outlined in our IJCV paper: http://people.csail.mit.edu/brussell/resea...IM-2005-025-new.pdf).

Hello!

I would like to obtain the source code of your annotation tool for my project, and I have filled in the code registration form on 2008. feb. 19., but have not received a reply.

I submitted my data again. Please let me know if I will receive the code soon. Your help would be kindly appreciated.

Gyula Vörös

My apologies for not sending you the code sooner. I just sent it to you. Please let us know if you have not received it.

Thank you for the code very much, though I cannot use it properly.

I enabled server side includes and perl scripts in Apache the following way (I use Ubuntu Linux):

Options +Includes +ExecCGI
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
AddHandler cgi-script .cgi .pl

The image appears in the annotation tool, but I am unable to draw polygons. In Firefox, at Tools->Page Info, "text/html" appears instead of "application/xhtml+xml".

I am sure that server side includes are enabled, because without them, the image does not appear at all.

Could you please help me?

Please send me e-mail (brussell AT csail DOT mit DOT edu) to resolve this. Once we debug this, I'll post the solution on the installation instructions page.

How do I specify which image I want to annotate with your web-based version of the LabelMe Annotation Tool?

You may browse all of the images on LabelMe here:

http://labelme.csail.mit.edu/browseLabelMe/index.html

and then select the one you wish to label.

We succeed installing the LabelMe code in Windows.
Here is a general guideline of what we did:

1) Downloaded the Labelme code package (the authors will send that).
2) Installed "Apache" for windows
3) Installed "Active Perl"
4) Installed Cygwin
5) Make sure that all of the above (except LabelMe) works OK first.
6) We change the config file of Apache (httpd.conf) by adding the following:
------------------------------------------------------------- ------
# Added by Juan from line 194
<Directory "C:/POSTDOC/LabelMe">
 Options FollowSymLinks SymLinksIfOwnerMatch Indexes +Includes +ExecCGI
        AllowOverride AuthConfig
 AllowOverride All
 Order allow,deny
 Allow from all
</Directory>



    # Added by Juan in line 325

Alias /LabelMe/ "C:/POSTDOC/LabelMe/"

# Added and changed by Juan (line 338)
    #ScriptAlias /cgi-bin/ "C:/Program Files/Apache Software Foundation/Apache2.2/cgi-bin/"
    ScriptAlias /cgi-bin/ "C:/POSTDOC/LabelMe/"

# Changed line 406
    AddHandler cgi-script .cgi .pl

# Added and changed in line 417
    AddType text/html .shtml
    AddHandler server-parsed .shtml
    AddOutputFilter INCLUDES .shtml


-----------------------------------------------------

7) We added the module "mod_rewrite" to the Active Perl version.
8) We changed and added to the file "Makefile" the following:
------------------------------------------------------------- ---------
#IMAGES_DIR=/afs/csail/group/vision/www/data/LabelMe/Test/Images
#ANNOTATIONS_DIR=/afs/csail/group/vision/www/data/LabelMe/Test/Annotations

IMAGES_DIR=/cygdrive/c/POSTDOC/LabelMe/Images/
ANNOTATIONS_DIR=/cygdrive/c/POSTDOC/LabelMe/Annotations/
 
---------------------------------------------------------- --------------
9) We run the command "make" on Cygwin, under the folder "LabelMe"
10) We changed all the "#! /usr/bin/.." by "#!c:/Perl/bin/perl.exe" in all the scripts under LabelMe
11) We add/changed the scipt "fetch_image.cgi". See below all the file after the changes
-------------------------------------------------------------- ------------

#!c:/Perl/bin/perl.exe

# Get STDIN:

read(STDIN,$collection,$ENV{'CONTENT_LENGTH'});

$fname = "DirLists/$collection.txt";

# Juan add:
if(!open(xFP,$fname)) {
    print "Status: 404\n\n";
    return;
}
$numlines = 0;
@lines = readline(xFP);
foreach $i (@lines) {
    $numlines = $numlines + 1;
};
close(xFP);
# end Juan add

if(!open(FP,$fname)) {
    print "Status: 404\n\n";
    return;
}

# Juan remove:
#open(NUMLINES,"wc -l $fname |");
#$numlines = <NUMLINES>;
#($numlines,$bar) = split(" DirLists",$numlines);
#close(NUMLINES);
# end Juan remove

$line = int(rand($numlines))+1;

for($i=1; $i < $line; $i++) {
    $garbage = readline(FP);
}

$fileinfo = readline(FP);
($im_dir,$im_file) = split(",",$fileinfo);
$im_file =~ tr/"\n"//d; # remove trailing newline

close(FP);

# Send back data:
print "Content-type: text/xml\n\n" ;
print "<out><dir>$im_dir</dir><file>$im_file</file></out>";

--------------------------------------------------------- -----------

12) Last things: Open your browser (we use Firefox) and enter the following url:
http://127.0.0.1/LabelMe/tool.html

Good luck!
Thanks Antonio and the rest for this great application!
Mathias Kolsch and Juan Wachs

Have tried several times to submit the form to receive the source code for your LabelMe project and the form always fails to submit correctly.
Is this a temporary problem?
If so, when do you expect it to be fixed?

Will download another browser and try again.

Nope, still a 500 Internal Server Error.

Thank you for pointing out that the source code download page is down. We have fixed the problem, so please try again. Our apologies for this bug.

Thank you, it worked this time.

Just a little bug I've noticed.

When you edit the name of an image that has already been annotated, a point is created somewhere near (1,1) after you click OK.

Can you send me an e-mail describing the browser you were using and the exact actions you took to produce the bug? I cannot reproduce it on my side.

Thanks!
--Bryan

Not sure what the LabelMe email is so I will post the information here.

I am using the Firefox 3.0.1 browser.

To produce the point around (1,1) all you need to do is either change the name of an already annotated object or delete the name of an already annotated object.

Thank you very much for providing the information. We'll look into fixing this bug.

Itis cool

I downloaded all 254 folders using LMinstall. and query about 'person'
 [Dj,j] = LMquery(Dlabelme, 'object.name', 'person');

However, it shows only 37 matches out of 815. I'm expecting much more examples of persons in the dataset. am I missing any part of the dataset, or making any mistakes?


Thanks for help.

Oh, I also found many folders after run LMprogressiveinstall are empty..
How to fix this?


Thanks.

There are more than 11,000 persons annotated in LabelMe.
How did you created the structure Dlabelme?

Did LMinstall produced empty folders?

--Antonio

QT - emily wrote:
< replied-to message removed by QT >

Edited by author 09-04-2008 06:58 AM
I have developed a new game for my Master's thesis called "Name It Game". It's a combination of LabelMe and Peekaboom, where you choose an object from a list of objects and you'll have to select that objects in the image. The other player has to guess what object you are revealing.

I have used a limited set of images from LabelMe as a base to test the game on quality of the selections. I use LabelMe because you have a very large set of high quality selections.

At the moment I'm looking for players for the game in order to get results for my thesis. Who can help me out?

The game is located at http://www.name-it-game.com

Some image annotations seem to be corrupted in the database.
For e.g., 'boston_static_street\0021.xml' downloaded from LabelMe website is different from what's downloaded from the boosting demo webpage 'http://people.csail.mit.edu/torralba/short...sting/boosting.html'.
Simple check revealed that the 'boston_static_street\0021.xml' from demo webpage is correct, while 'boston_static_street\0021.xml' from LabelMe website using urlwrite commmand is not correct.
Could anybody helo taking a look?
Thanks!

Is this the matlab command that you tried?

urlwrite('http://labelme.csail.mit.edu/Annotations/b...0021.xml','test.xml')
This seems to work for me.


Also the xml file seems to be correct:

http://labelme.csail.mit.edu/Annotations/b...tic_street/0021.xml
Can you give more information about the error you are getting?


QT - joe wrote:
< replied-to message removed by QT >

Yes. Did you take a look at the ''boston_static_street\0021.xml' from boosting demo website?

Here are first few lines of the first object 'CarSide' from boosting demo website.
- <pt>
  <x>98</x>
  <y>108</y>
  </pt>
- <pt>
  <x>100</x>
  <y>108</y>
  </pt>
- <pt>
  <x>101</x>
  <y>101</y>

Here are first few lines of the first object 'CarSide' from urlwrite .

- <pt>
  <x>3</x>
  <y>2</y>
  </pt>
- <pt>
  <x>1</x>
  <y>281</y>
  </pt>
- <pt>
  <x>324</x>
  <y>265</y>


The points' coordinates from urlwrite exceed the image size, which is 179x239x3.

Thanks. I understand the problem now. The image in the boosting demo website has been scaled so that the car has a canonical scale.
Therefore, the image size has changed and you can not use the annotation file from the LabelMe website.

The original image in LabelMe, has a size of 640*480

http://labelme.csail.mit.edu/Images/boston_static_street/0021.jpg
Antonio


QT - joe wrote:
< replied-to message removed by QT >

Hi,
 Are there any statistics as to how many *unique* labels are there in the labelme dataset?

The 242,xxx labels statistic counts how many objects are labeled but we are looking into how many unique labels are there.

Thanks

Umm... also, the link to download annotations is not working!

This link...
http://people.csail.mit.edu/torralba/resea...rap_annotations.cgi
referenced on
http://people.csail.mit.edu/torralba/resea...s/instructions.html

Gives a 404 error.

Thanks

In order to download the annotations you should visit this page:

http://labelme.csail.mit.edu/instructions.html

The page you were using is old and we do not support that method of download anymore.

Antonio


QT - James wrote:
< replied-to message removed by QT >

The number o unique labels increases every week. So, in order to know hoe many unique labels you have in your copy, you can run the next commands:
[names, counts] = LMobjectnames(D);
% where D is the index struct generated with LMdatabase.

length(names)
% this will give you the number of unique labels.



QT - James wrote:
< replied-to message removed by QT >

Just wanted to say couple of bugs happen when running LabelMe on Firefox 3.0.3.

First, when editing or deleting an annotation a point is automatically created somewhere around the top left corner.

Also, after about 10-15 minutes of use the browser seems to crash because of what appears to be a memory overload.

Which version of MATLAB do I need to run LabelMe? I have a 6.0 and in the office a 7b. With 6.0 you cannot initialize the database:

Dlabelme=LMdatabase(HOMEANNOTATIONS);

Nfolders =

   151

1/151,
 Total images:0, annotation files:0 (with 0 empty xml files)
2/151, 05june05_static_indoor
??? Error using ==> subplot
Too many input arguments.

Error in ==> C:\MATLAB6p5\toolbox\LabelMeToolbox\LMdatabase.m (plotbar)
On line 226 ==> ha = subplot(2,1,1, 'parent', fig); cla(ha)

Error in ==> C:\MATLAB6p5\toolbox\LabelMeToolbox\LMdatabase.m
On line 194 ==> plotbar(Hfig,f,Nfolders,i,N);

these errors occur both using the data online and after having downloaded the entire database. Does someone know why is this happening?

Cheers

It is said that there are 5.8 Gbytes for training.tar.gz in train in Spain and test in the rest of the world (http://people.csail.mit.edu/torralba/benchmarks/). But when I download them, this package is only 1.8 Gbytes. When I unzipt it, it fails and pops up the information "it is not complete". Could you please have a check and upload the full 5.8 Gbytes images. Thanks

Hey LabelMe gurus,

I've been using LabelMe for quite some time for my research and I like to make sure I'm using the most recent version of the Labelme Toolbox at all times. Periodically re-downloading it seems kind of silly. CVS/svn would be awesome, but just putting the timestamp of the most recent version next to the link would work too.

Thanks,
Tomasz

Thanks for the suggestion. CVS will be the best thing to do. We will try to set this up (maybe after the CVPR deadline...)

The toolbox is being updated very often but we always try to make sure that it is backwards compatible.

Antonio


QT - Tomasz wrote:
< replied-to message removed by QT >

Sir
Ive gone through your annotation tool.It is an excellent project and i downloaded(received thru mail) your source code.When i am trying to
configure it on my ubuntu system ,After running the make file it is showing some error in the 1st two lines (i.e, near Images and Annotations).Now if i run tool.html my screen is hanging up @ loading image......No image is being displayed .Please assist me in solving the problem asap!! its a kinda urgent!!!

Thanq!!

Hi,

Can you e-mail us directly what the Makefile error is? Also, make sure to check that your site is generating XHTML pages. If not, then you need to configure Server Side Includes (SSI) properly.

Bryan

Sir ,
Thanq for replying so fast.
Iam getting an error as follows:

------------------------------------------------------------- ------------------------------------------------------------- if [ -a "Images" ]; then echo "IMAGES_DIR all set."; else ln -s
/var/www/labcopy/lab/LabelMe1/Images Images; fi

[ : 1:Images: unexpected operator


if [ -a "Annotations" ]; then echo "ANNOTATIONS_DIR all set."; else ln -s /var/www/labcopy/lab/LabelMe1/Annotations Annotations; fi

[ : 1:Images: unexpected operator
--------------------------------------------------------------- --------------------------------------------------------
/var/www/labcopy/lab/LabelMe1 is my path ..iV created Images and Annotaions directories mannually . I also created directory figs in both images and annoations folders .I placed my images in he figs folder in Images directory.

I also dunno exactly about SSI .Can you please send me a link or something regarding it.

Thanks in advance


On Fri, Nov 14, 2008 at 10:27 PM, QT - LabelMe <
qtopic-37-E4xRZ7fZZhh@quicktopic.com> wrote:

>
< replied-to message removed by QT >

Sir,

In the page properties it is showing text/html ..not any xhtml or so!!

Hi,

Since it looks like the Images/ and Annotations/ folders are in the same directory as the annotation tool code, you can change the "all" line in the Makefile to read:

all: dirlists counter logs

Then, try running "make" again. If the rest of the commands do not work, then it looks like your system is not set up to properly execute the bash commands in the Makefile. My suggestion here is to do a Google search to resolve this.

Regarding SSI, I would suggest a web search since this depends on how your system is configured.

Best,
Bryan

Sir,

After editing "all" line as you said,Makefile is working properly without giving any errors
but when I am trying to open tool.html ..the screen is once again hanging @ Loading image
labelme.txt is showing all the images correctly.But in the page properties it is showing text/html.Please help me out!!

Thanks in advance

On Fri, Nov 14, 2008 at 11:00 PM, QT - LabelMe <
qtopic-37-E4xRZ7fZZhh@quicktopic.com> wrote:

>
< replied-to message removed by QT >

Sir

Now i have enabled SSI in my system. It is working fine as i have tested it by running a test file in the browser.Even then tool.html is hanging @ Loading image.... and the pageinfo is not showing xhtml and stuff. What might be the reason??

Thanks in advance

Please help me asap.. its kinda urgent!!!

thanx in anticipation