Hey Neil,

What do we do in the cases where the thresholds fail? Do we just set u and v to 0 at those places? Your explanation is still really fuzzy to me. Also, how many thresholds are we supposed to have? Your response to Ali made it sound like we need to check if lambda1 is large, lambda2/lambda1 is less than but close to 1, lambda2 is large, and lambda1/lambda2 is small but not smaller than 1. Looking at my lambda values for the corridor images, those 4 thresholds will never hold together, so I'd pretty much create a u,v field of all zeros. Is that what you want? Sorry, this isn't too clear to me.

neil:
how many layers should the gaussian pyrimad have? 3,4,5?

Ryan,

If you are comparing windows and one of the thresholds fail, it means you shouldn't update your flow field at that position because you will get a bad result (A'*A is either ill-conditioned or the conditioning is due to noise).

--------------

Ali,

It sounds like you understand what's going on. The best optical flow estimates will come from regions with lots of texture; in this case lambda1 will be large and lambda2/lambda1 will be close to 1. Basically, the higher lambda2 is the better and the lower lambda1/lambda2 is the better (where lambda1 > lambda2).

--------------

Yuzhe,

Suppose you are computing the optical flow at position (x,y) and the current optical flow estimate is given by u(x,y) and v(x,y). Then you would use windows of (window size = [5 5]),

W0 = I0(y-2:y+2,x-2:x+2);
W1 = I1(y-2+v(x,y):x+2+u(x,y));

So basically, the optical flow at the center pixel is used to shift the entire window the same amount.

-Neil

Hi everyone,

While implementing the algorithm, shall we shift the window pairs in a "pixel by pixel" manner, or should we divide the whole image into blocks and then figure out the velocity for each block? I think according to our assumption, pixels within a certain window are supposed to have the same velocity. So, if we do it in a pixel by pixel fashion, is it a contradiction to our assumption? I got confused here.

Thanks!

Edited by author 03-11-2006 09:30 PM
Neil:
I still dont get why we need setting thresholds for lambda1 and lamda2. Well I do understand when lamda1 and lamda2 are small it means low textured region(i.e. constant region) so there wont be much motion in those regions. I do also understand that when lamda1 is much larger than lamda2 we have an egde and gradients will point most in one direction and could cause matrix A to become singular. And also when both lamda1 and lamda2 are large it refers to a high textured region. So what do the thresholds that you talked about in the assignment mean? and how could they help us
Thanks

Neil,

So, I understand a bit more about the eigenvalues of A'A. However, I still do not understand what to do once I find that, for example, lambda1/lambda2 is too large. I understand that if lambda1/lambda2 is large then this corresponds to the gradient being the same throughout the whole window, but how does knowing this help modify A'A? Thanks

The free program FineView (scroll down on the following page: http://www.snapfiles.com/Freeware/gmm/fwviewers.html) will do it as well.

Ryan,

I used my ACDSee 7 to convert those files into .bmp format. Let me know if you still need them.:)

Just as a note to all, Matlab 6 (Release 12) DOES NOT support .ppm or .pgm images. Can someone save these images as a different file extension? If not, I guess I will have to find a newer version of Matlab...

Hey Ryan,

Here's one way to think about it. What kind of image windows will yield poor optical flow estimate? One is where the spatial gradients of every pixel point in the same direction. lambda1 / lambda2 indicates these type of regions (if this number is high, then the rank of A'*A is close to 1). Another case where the estimation will fail is if the spatial gradient is close to zero at every pixel (ie, rank(A'*A) is close to 0). Because of noise, such a region may be well conditioned (ie, lambda1 / lambda2 close to 1). That's where the check on lambda2 comes in.

Neil,

I vaguely remember the discussion in class about the eigenvalues of A. Could you explain a little more about why they would help us with teh Lucas-Kanade algorithm and how we can use them as thresholds?

OK, I apologize for the confusion that may be caused by my example. I based it on my implementation and thought the example had an unambiguous solution, but it turns out there are multiple paths with the same lowest cost. As Alex figured out the hard way, if given a choice for the "best" path (ie, more than one of left, up, or upleft have the same path cost) I choose upleft over left and left over up.

-Neil

Just thought I'd point this out for anyone who's not getting Neil's newsgroup example exactly, like if you're getting one too many -2's on the end. Here's a fix a few of us have found.

In the pseudocode in the paper, you'll see something like the following:

if(min1 == cmin) Mij = 1
if(min2 == cmin) Mij = 2
if(min3 == cmin) Mij = 3

If you code this directly, then you'll get situations where ties between min1 and min3 go to min3, because Mij gets overwritten. Essentially, ties go to the occlusion. Change these lines to an if-else clause so that ties go to min1 instead. Then you'll get the right answer. Shouldn't change your picture results much, if at all.

Hope this helps someone,
-Alex