In Figure 13, it took 4 fields to recover that tracking failure. Can we get any idea how long it will take to track the correct object or it just gauranteed to recover for a very short time. And in Figure5, at diffuse step, how to split the elements from t-1 to t?

This method doesn't address initialization of the object model. It's assumed to be known in advance.

I won't go into much detail tomorrow about the specific parameterization that they use, but for their examples they use B-splines. However, rather than allowing arbitrary changes to the splines they restrict the shape-space to a small subspace of possible changes. The purpose of this is to disallow unrealistic changes such as a rigid box turning into a sphere. One way to determine a good subspace is to use PCA.

p(x) is assumed to be known before the algorithm starts, so clutter doesn't really affect it. What clutter affects is accurate feature extraction. With lots of clutter there will be lots of detected features that don't belong to the object being tracked and in this case the tracker might not know which features to follow or not. Having a multi-modal probability distribution allows the tracker to follow both the object and false features caused by clutter. If a Gaussian distribution is used, the tracker will often lose track of the object because the false features will cause the Gaussian to move away from the object.

As for the observation independence, I think what they mean is that the conditional observations p(z_i|x_i) are independent.

Sorry this isn't more clear, hopefully this clarifies things

I've noticed that for all their given examples, a hand-drawn template was provided for initialization. Is this absolutely necesary for the tracking to work or is it just a convenience?

Also, how much variablity is allowed in the curves from frame to frame? I've noticed some warping of the curves in the movie example with the hyperactive kid dancing (http://www.robots.ox.ac.uk/~misard/images/dancemv.mpg), and it seems like scaling is dealt with quite nicely also.

Sunny.

btw more examples abound @ (http://www.robots.ox.ac.uk/~misard/condensation.html)

I'm glad someone is presenting this paper, because I have never been able to understand anything other than the equations in Sections 1-3 of this paper. In general, the descriptions in words of the equations are almost meaningless to me, and I'm glad someone will be presenting these, Here are a couple of the more specific questions I have from these sections:

What is the relationship between assuming a p(x) is Gaussian and clutter? I could see that there would be a relationship between p(x|z) or p(z|x) being Gaussian and clutter, but not the prior p(x).

To me, "observations z_t are assumed to be independent, both mutually and wrt the dynamical process" is not implied by the corresponding equation (equation 2). It should never be the case that z_t and z_{t-1} are independent in a tracking application.

The author mentioned that the tracker must be supplied a set of sample values to initilise, so I guess it cannot handle very well when the camera "lose sight", which can be considered starting a new sequence. BTW, as the task is difficult even to human, it would be really impressive if this algorithm can do.

Also, it is interesting to know how far the algorithm can go using other cues rather than just contour.

You're probably right. The algorithm would search for something that looks like a leaf and since there's a bush full of leaves it would probably find another leaf.

Maybe I should redefine "lose sight of the leaf". If the camera were to turn away for a while (the leaf is not visible) and then return to its original position, I have a feeling that one of the outlier leaves would probably be picked up instead.

The CONDENSATION algorithm handles occlusion fairly well because it combines image features with a stored model of the object being tracked. When measured features in the image become unreliable (as in occlusion), the model compensates. If occlusion persists for a long time, then probably the algorithm would fail however.

The camouflage section is neat. Despite the fact that many of the leaves are potential outliers, the Condensation algorithm performs rather well. However, it does make me wonder what would happen if the camera were to ever lose site of the leaf and then try to reacquire it. I can imagine that this might cause some problems.