I think it is a wonderful paper. However at the end of the paper its a bit discouraging to find that the algo is not gauranteed to converge to an optimum solution.
I have the following questions/doubts:

1) Near the end of section 4 we see that an extra term \epsilon*B_F() has been added to the loss function to force the solution to a local minima ( or saddle point). I understand that the proof might be rigorous but can we have an intuitive insight into why adding such a term would make theta(t) bounded? Is such a thing also used in other optimization problems when there is a possibility of the algorithm diverging to a degenerate solution at infinity?

2) we have got two sets of parameters : a and v. To me it seems like a's represent the eigen values and v's represent the eigen vectors of the subspace onto which we are projecting our original data( am I right?). Now the assumption is that the two sets are independent of each other ( otherwise we cant do the minimizations separately). How far is the assumption valid? ( Pls ignore the question if I am not making sense. I can explain the question in class).