could someone post any notes they took at today's Q & A session?

When we refer to the two windows, (the second of which is shifted by the previous result in MSLK)

The gradient of the first window is used in producing A,
The difference of the two windows is used in producing B...

Is this correct?

OK, seems every time I make a post I end up confusing things for people. The optical flow results I posted for the corridor are from images 1 and 2 instead of 0 and 1. So if you're wondering why the optical flow looks funny that's why.

-Neil

Brian,

Yeah, you probably don't need to explicitly apply a Gaussian filter, but I was just making sure...

----

Yuzhe,

In my implmentation, if a window goes outside the image, I truncate it so it only contains valid portions of the image (an equivalent technique would be to zero pad the image). I found this helped a lot because otherwise the edges of the image don't get an optical flow estimate at low-res. I recommend doing this if you haven't already, but it's not really stated anywhere that you should do this so consider it optional.

Hi Neil,

For the sphere image, you used lambda2 > .01 & lambda1/lambda2 < 100, windowsize = 15x15, 6 levels. Well, the sphere image is 200*200, then the pyramid should be 200, 100, 50, 25, 12, 6. Then how could we use the 15*15 window on a 6*6 image on the 6th level? I think the window has fixed size at each level, right?

Thank you!

Is explicit gaussian smoothing necessary? If we use imresize(), isn't it supposed to do that automatically?

From the matlab doc:

When the specified output size is smaller than the size of
the input image, and METHOD is 'bilinear' or 'bicubic',
IMRESIZE applies a lowpass filter before interpolation to
reduce aliasing. The default filter size is 11-by-11.

Oh, one more thing I should mention: I'm using Gaussian smoothing at each level of the pyramid..

-Neil

OK, I guess image tags don't work in this discussion board...here's links to the images:

sphere result

corridor result

Ali,

In my implementation I used 5-6 levels (at least for the sphere and corridor images).

-----------------------------

Ramsin,

When the threshold test fails, you should just keep your previous estimate of the optical flow at that pixel. One of the advantages of multi-scale optical flow is that it can work in regions with little texture (without a huge window) because the optical flow is propagated from the low-res estimation.

Also I think you're confused about the thresholds..there should only be two thresholds. You should check that lambda2 > th1 and that lambda1/lambda2 < th2. I found that values of th1 = .01 and th2 = 100 worked fairly well.

----------------------------

Nikhil,

Below are two results from my implementation. For the corridor image I am using lambda2 > .1 & lambda1/lambda2 < 100 with a window size of 9x9 and 7 levels. For the sphere image, I am using lambda2 > .01 & lambda1/lambda2 < 100, windowsize = 15x15, 6 levels.

<img src="http://www.neilalldrin.com/shared/files/of_sphere.jpg" />

<img src="http://www.neilalldrin.com/shared/files/of_corridor.jpg" />

Neil,
How well do we expect the MSLK to work for the sphere and corridor images?
Thanks

Hey Neil,

What do we do in the cases where the thresholds fail? Do we just set u and v to 0 at those places? Your explanation is still really fuzzy to me. Also, how many thresholds are we supposed to have? Your response to Ali made it sound like we need to check if lambda1 is large, lambda2/lambda1 is less than but close to 1, lambda2 is large, and lambda1/lambda2 is small but not smaller than 1. Looking at my lambda values for the corridor images, those 4 thresholds will never hold together, so I'd pretty much create a u,v field of all zeros. Is that what you want? Sorry, this isn't too clear to me.

neil:
how many layers should the gaussian pyrimad have? 3,4,5?

Ryan,

If you are comparing windows and one of the thresholds fail, it means you shouldn't update your flow field at that position because you will get a bad result (A'*A is either ill-conditioned or the conditioning is due to noise).

--------------

Ali,

It sounds like you understand what's going on. The best optical flow estimates will come from regions with lots of texture; in this case lambda1 will be large and lambda2/lambda1 will be close to 1. Basically, the higher lambda2 is the better and the lower lambda1/lambda2 is the better (where lambda1 > lambda2).

--------------

Yuzhe,

Suppose you are computing the optical flow at position (x,y) and the current optical flow estimate is given by u(x,y) and v(x,y). Then you would use windows of (window size = [5 5]),

W0 = I0(y-2:y+2,x-2:x+2);
W1 = I1(y-2+v(x,y):x+2+u(x,y));

So basically, the optical flow at the center pixel is used to shift the entire window the same amount.

-Neil

Hi everyone,

While implementing the algorithm, shall we shift the window pairs in a "pixel by pixel" manner, or should we divide the whole image into blocks and then figure out the velocity for each block? I think according to our assumption, pixels within a certain window are supposed to have the same velocity. So, if we do it in a pixel by pixel fashion, is it a contradiction to our assumption? I got confused here.

Thanks!

Edited by author 03-11-2006 09:30 PM
Neil:
I still dont get why we need setting thresholds for lambda1 and lamda2. Well I do understand when lamda1 and lamda2 are small it means low textured region(i.e. constant region) so there wont be much motion in those regions. I do also understand that when lamda1 is much larger than lamda2 we have an egde and gradients will point most in one direction and could cause matrix A to become singular. And also when both lamda1 and lamda2 are large it refers to a high textured region. So what do the thresholds that you talked about in the assignment mean? and how could they help us
Thanks