The Seitz and Dyer paper assumes the availability of dense correspondence besides the availabiity of the projection matrices and yet they claim they do not require structure. If you have those two things correctly, you infact have the structure of the scene. It is a simple calculation away. Hence the whole thing about how they are able to do this interpolation without recovering structure is quite bogus.