Instead of presenting a single idea, this paper presents two fairly distinct ones. Although the results from scene context are used for object recognition, there doesn't seem to be any backwards link. I wonder if it would be helpful to not reconstruct the scene as an intermediate step, but rather use results from the object detector training to find homogeneous regions, using the same features.

I imagine that this would take a lot more data and might be computationally too expensive. Even more so, maybe the reason people think of the world in that kind of 3D-way is the best, but I think it would be good to try.

Ploease post your thoughts on Geometric Context from a single image here