I guess I'm still a bit confused by the toy example results in figure 2. Probably because I'm trying to visualize something higher dimensional in a low-D space, but I still have trouble seeing how a linear combination of the three components of degree 2 could uniquely specify a point. It seems like the middle one, with the concentric circles, would constrain a radius from the center of the circles, the top would constrain a point in a cross-shaped shell, and the bottom one would constrain it in another cross-shaped shell, but rotated about 45 degrees. Perhaps I'm off base and these could uniquely constrain a point?

The other thing i found curious is that the one which supposedly picks up the variance in the noise seems to determine how far the points lie from the parabola in a direction normal to the parabola, while the original noise was vertical and had no actual horizontal component.