Brian,

Yeah, you probably don't need to explicitly apply a Gaussian filter, but I was just making sure...

----

Yuzhe,

In my implmentation, if a window goes outside the image, I truncate it so it only contains valid portions of the image (an equivalent technique would be to zero pad the image). I found this helped a lot because otherwise the edges of the image don't get an optical flow estimate at low-res. I recommend doing this if you haven't already, but it's not really stated anywhere that you should do this so consider it optional.