well since I happen to have strong views on this, I guess I will make the first noise. Let me start my saying that I personally think that the multi-scale optical flow enterprise is a pipe-dream and it is time that people working on it realized the reality of the combinatorial structure of the problem. The general problem of scene matching does not allow for any of the assumptions that the people who use direct methods make. The brightness constancy equation is barly valid in anything but the most restricted of the circumstances, and a search enterprise which is based on it somehow does not make too much sense to me.

We are now in a time and place where there is ample computation power available to us (that is not to say that we should abuse it) to be able to be look at the combinatorial structure of the problems before us and design algorithms which explicitly deal with it.

The only thing which the direct methods people can claim over the feature based methods is that they use the whole image where as the feature based enterprise only uses "corner features". This makes a lot of sense when you are dealing with objects in the scene which have smooth boundaries e.g. an ellipse with a large enough radius of curvature.

I would have loved to make the first noise because my current work on face modeling has made me a big fan of feature based method. It is nearly impossible to do dense stereo matching in surfaces as smooth as the face unless we are using active vision ( like projecting patterns ). Almost all methods of dense stereo fail to recover a different depth value for the nose.

I think there is a flaw in the reasoning given in direct method paper. They say in page three, start of para 3

" In other words, the direct method use information from all pixels, weighting the contribution of each pixel according to the underlying image structure around the pixel"

Now it is clear from the above statement that the pixels in smooth regions ( where we can't detect corners ) are unreliable and are automatically given less weight. Are they not contradicting their own statement that they use all pixels instead of a selected few. They are using all pixels but only a selected few dominate the result.

Furthermore I think it is very difficult(impossible??) to do wide baseline stereo using direct method.

However I do feel that direct methods should be applied in situations where we have very dense sampling for the space .. for example a video sequence..

I am also a strong proponent of the feature based methods, but I think the direct methods definately still have a place in matching work.

First of all, for short baseline pairs and for frames with differential motion, direct methods blow feature based methods out of the water. I have yet to see any feature-based method solve the flower garden sequence (which happens to be basically a toy problem for the direct people).

That said, if coarse-to-fine processing really makes direct methods on par with feature based methods for large disparity, where is a successful piece of research doing just that?