"In particular, we show high quality retrieval
results without any consideration of the geometric layout
of visual words within the frame" - section 2

Although they make the argument that the geometry checks can be used on the top n matches, I think this method could be really improved if the geometry was somehow tied into the tree.

On the other hand, their results look very impressive!

In Section 5 [Implementation of Scoring], the paper mentions forward files and fully expanded forward files. What are these and how do they differ from the inverted files they describe?

What is an MSER, and why do they warp an elliptical region around these regions into a circle for their feature extraction?

"What is an MSER, and why do they warp an elliptical region around these regions into a circle for their feature extraction?"

MSER (Maximally Stable Extremal Region) is a high quality detector created by Matas (et al.) (Google it). The detector finds regions of interest by applying a segmentation algorithm based on a watershed algorithm to the image. A key region, as opposed to a key point, has the excellent quality that it is automatically affine covariant - the detector does not have to "construct" this ability. On the contrary the region should be affine normalized before use and this is done by warping an ellipse around the region and normalizing it into a circle.

1. " With our vocabulary tree apporach, the representation of an image patch is simply one or two integers, which should be contrasted to the hundreds of bytes or floats used for a descriptor vector."

then on the next page...

" For D-dimensional descriptors represented as char the size of the tree is Dk^L bytes. With our current implementation, a tree with D = 128, L= 6, and k= 10, ..., uses 143MB of memory."

I am confused of what the descriptor vectors consist of. I thought it was only 1 or 2 integers per image patch. Does dimension of the descriptor vector (e.g D = 128) mean the number of pixels in the image patch, and the 128 elements of descriptor vector take the value of one of the 2 integers.. ? Thank you for clarifying.

Image retrieval results are sensitive to the particular distance metric used. It would be interesting to know if the authors have any intuition why L1 norm works better than L2 norm.

It's also interesting that in table 1 the best image retrieval performance for vocabulary size 10K is for a non-hierarchical tree. I guess for performance the hierarchy makes a lot of sense though.

They use the Term Frequency Inverse Document Frequency for 'document matching'. I know thats a really hot and large area of research in and of itself. How does the TF-IDF compare to the state of the art systems in NLP (or is it state of the art)?

The hierarchical tree is a good idea to distribute quantized patches and retrieve them in a faster way. Although I wonder how do they check redundancy of words and how can they insert new objects on the fly if they have to compute entropy for all the words in the tree (what if they are millions?). Is the quantization of patches enough to claim this?

I'm not sure I completely understand how entropy works, but why is it better to "use entropy relative to the root of the tree and ignore dependencies within the path" than to assign entropy relative to the parent node, even though assigning entropy based on the node above seems more intuitively correct?

 "By using a vocabulary adapted to the likely distribution of data, we use a much smaller tree, resulting in better resolution while maintaining a compact representatin."

When the authors pick a vocabulary to adapt to the likely distribution of the data, are they doing that for every on-the-fly inserted object? I was thinking that each time a new object is inserted into the tree on-the-fly, they can check its distribution first to find the tree that has the vocabulary most like it.

Edited by author 09-28-2006 03:01 PM
The graph on branck factor shows that the performance increse with the factor 'k', but does it stop increasing after a certain k? what is the trade off there?

I was impressed by the paper's continual focus on implementation issues, whether it be the tree structure for efficiency, the ability to "on-the-fly" add new images to the database or even the use of real video data (not all lab photos!) in their results.
I am not really sure what the entropy is talking about in the paper. But it got me thinking (uh-oh). In information theory, when you are trying to maximize the compression of a symbol alphabet, you are trying to minimize the entropy. As the symbols become less likely to occur the number of bits assigned to describe the symbol increases. Doing this ideally (say Huffman Encoder) will then minimize the entropy of the alphabet and maximize the compression. My idea, is why not be keeping track of the probabilities of object matches for each of the nodes in the search tree and have variable length branches. The most frequently found objects would then have very short branches and the least frequently found objects would have much longer search branches. This would reduce the average search for each object in the tree.
The paper may have addressed this issue with the "stop lists" but I didn't quite follow it.

In this paper, they extract elliptical patches while in many papers rectangular patches seem to be more the norm. Intuitively it seems like elliptical patches may yield slightly better performance (corners of rectangles seem likely to be noisy while circular patches catch the content best) while rectangular patches may yield slightly better efficiency (in terms of indexing). What factors should one consider in general when making this decision?

Also a comment along the lines of Paul's. It seems as though they looked at a wide variety of methods in their search for the best approach (23 different settings reported in Table 1) covering a wide range of performance (70.1 to 90.6). It seems like a deeper coverage of *why* certain settings yielded the best performance would be helpful, particularly where some decisions seem quite fundamental.

Alert for all travellers to North America: Abuse of Human Rights and Privacy Violations:

Racially intolerant white canadian cops and security and their henchmen claim to be despots; following parasitically in the footsteps of their american counterparts, and wilfully engage in their racial profiling of non-whites, in racial harassment of non-whites, and in racially dehumanizing attempts to racially harass non-whites through intimidating physically, mentally, and spiritually; portraying their racial hatred of non-whites through causing wilfull and dehumanizing disturbance to non-whites through using illegal wall-see-through technologies and audio-bugs on non-whites' homes; through listening and watching through the walls of non-whites' rented and owned homes, and through their internet and private telephones. The perpetuators of these evil deeds do this from their cars using illegal equipment slyly given to them by the unworthy cops, and then
accelerating their cars loudly and intimidatingly near non-whites' homes and driving intimidatingly in presence of non-whites on streets, making threatening u-turns, driving intimidatingly right up and over sidewalks when a non-white is on the sidewalk, and throwing their ugly bullying weight around, in their shameless acts of cowardice. It is all done slyly, supposedly smartly, however, they cannot fool all the people all the time. The cops also participate themselves to wail their sirens abusively everytime non-whites move and talk inside their rented and owned homes in daily routine living, in addition to having their henchmen, and often, using their non-white gutless henchmen in cars, transport, shopping centers, neighborhoods, etc, to commit these ugly harassing racially profiling
deeds at all times day and night. Using non-whites to engage in racial harassment of other non-whites is an obnoxiously evil sinister humanely disgraceful intelligent move of the whites well-known for their ugly divide and rule tactics through their non-white henchmen.

It's a shameful disgrace when the so called protectors of law turn into abusers of law themselves and throw the weight of their uniforms and law around as cowards. So, they and their henchmen, appear to be very law respecting on the outside; however, they network cowardly to commit sly acts of provocation to non-whites all the time, which is supposed to
be legally acceptable. Is watching through walls of non-whites homes, bugging their homes, working in networking syndicates against them, committing human rights and privacy violations against them, supposedly lawful for the whites? Who makes those laws that favor
only the whites? The law itself has racism in its clauses. The ugly inner dirt of the perpetuators of these evil deeds of racism do not deserve to step into religious institutions for their ugly deeds - such as, if you ain't white, you ain't right? Oh! Really? Nicely dressed, beautiful people, magnificient concrete jungles, clean roads and lawns, sweet polite talkers on the outside, full of ugly stench in their souls, that is the
cause of these racist policies that are outrightly biased against non-whites. What a shame!

Most of these ugly acts of dehumanizing racial profiling depict the cowardice of the doers of these deeds in the real sense, and are done at the behind the scenes insistence of the racially intolerant white cops through their frontline stooges. However, without physical evidence, the white cops, security, societies, and their henchmen are laughing sinisterly
at their heinous deeds and the legal system seems to support this evil through its inability to take action without physical evidence. Their racial profiling penetrates public transport systems, shops and stores to do all they can to make the non-whites feel unwelcome in their dehumanizing acts of racial profiling against non-whites and those who don't conform to their nonsense. The white cops, security, and white communities use their
henchmen who do just as they are told and from behind the safety cushion of their oil-guzzling, pollution creating, often dark-glassed vehicles to intimidate and harass non-whites in obnoxious racial profiling that reflects the immoral, despotic, and cowardly behaviour of racially intolerant white cops, security, communities and their dumb henchmen
who do just as they are told, fuelled as they are in their racial frenzy, thanks to the racially manipulative corporate controlled media.

For more information, visit:

http://www.yourluckytoday.blogspot.com

Volunteers are welcome to circulate this information to all they know to put an end to this abuse and violations of human rights committed by immorally misbehaved white cops, security, white communities, public transports, shops, stores, etc, and their dumb henchmen who do just as they are told in their racial frenzy.

Save this information on your computers before any cowards remove it from the websites.

Racism is immoral and dehumanizing behaviour that reflects the "incapable to perform humanely" quality of those who are racist and are being watched from God's court above in ways they cannot be expected to be capable to perceive yet.

It's a shame when obnoxious stench of racism comes from people in so called rich countries. It's even more of a shame when words are twisted by media to influence young minds with lies. It's even more of a shame when so called authorities perpetuate racism and behave racistly and enforce racist policies and behaviour through intimidating means amidst outer sweet and polite talks. Racism seems to be prominent among so called white people in rich countries who cannot bear non-whites from other countries of origin. Planet Earth belongs to people of Earth. Highly educated people of high intellectual calibres, rich bosses and CEOs, etc, of rich countries are a blotch on humanity and their material levels when they
haven't yet evolved to basic human concepts of all humans have red blood irrespective of race.

Racism stems from social attitudes that are perpetuated by racist societies, the media, the authoritarians, and the peers. It's time to say, shame on all those who perpetuate racism and racist attitudes.

Thank you.