I just got done skimming "The information bottleneck method", which is the original paper in which they actually propose the idea of the information bottleneck. I think the formulation given there is much more clearer, informative and cleaner.

The principal idea being that you just do not want compression of data, you want compression while taking care of the fidelity. The fidelity measurements require that you define a distortion function, which is a bit of a pain, since choosing a distortion function is equivalent to choosing apriori what features of X are more interesting than others.

In the original paper a very clean variational formulation is given to this problem in terms of mutual information as a measure of distortion.

Excellent idea and method, I just wish that we did the original paper instead of the one we are assigned to read.