1. Regarding to Brandyn's question: I guess it is a common knowledge that PCA somehow assumes Gaussians, as Javier pointed out last week. But since I am quite new to this feild, can anyone try to explain the relationship between PCA and Gaussian sources? I thought the relationship was this: PCA rotates your data so that they become indepepdent if the sources are gaussian. So if you don't care too much about
independence I don't see why we should chose "gaussianfying" kernels. I mean, no matter what the distribution it is (in the feature space), PCA will still give you maximal variance at the first eigenvector, and that's what's so useful about PCA, no? Please forgive me if this is a silly argument.

2. The reproducing kernel section is confusing to me. I can see that for each kernel a map can be constructed from the input space (X) to the RKHS, but I don't quite follow how the RKHS can be interpreted as the feature space, since elements in RKHS are functions.

3. This is a completely irreponsible suggestion (meaning that I have no idea how this can be done, and I am not even going to try), but maybe it is worthwhile to ask this question: is it possible to characterize the set of kernels that always have pre-image? If we restrict
ourselves to this set of kernels, how much would it reduce to power of kernel PCA?