Very interesting paper. In part 3, where illumination problems are dealt with, basis images are mentioned. Rest of the lighting approach relies on these basis images, yet there is nothing about them. What are the basis images made of?

Is there an abuse of notation in equation (3)? Shouldnt the mu for each row be different ; mu1,mu2 etc. If we assume a constant mu, doesnt it mean that all regions are undergoing the same motion which essentially means that the camera is moving wrt the scene.

This is a nice application of optical flow since in a sequence at this high framerate the motion is guaranteed to be small. It seems that because of this, its not very likely to have more than a pixel or two wide region of occlusion between two frames. So the only occlusion being dealt with is that when considering two fairly widely separated views. Does this mean that the occlusion will not really effect the tracking?

I think the paper is great in dealing with different issues related to tracking. Tracking has traditionally done using some kind of adaptive filtering (ex. Kalman Filtering ). Again the idea remains the same. The parameters involved in tracking are not calcuated from scratch but are iteratively updated based on the previous information. All adaptive algorithms use a steps for tracking:

1) Predict: From previous data the new parameters are predicted.
2) Update: In this stage a search is performed in the neighborbood of the predicted parameters to search for the best match.

The algorithm presented in this paper essentially does the same. All the kinds of motion described can be modeled in Kalman Filter also. It would be nice if we can know how it compares with tracking using some kind of adaptive filters?

The inclusion of illumination effects and occlusion etc would definitely make this algorithm superior to conventional tracking but I was interested to know if it is indeed superior when there is no occlusion and illumination changes?