I too thought this paper addressed an interesting approach to the marketing problem. I particularly enjoyed the introductory material, which I felt did a good job of giving background.

However, I found some of the experimental set-up and experimentation sections a bit vague. I had a hard time figuring out exactly what data was used for training and testing. Also, the paper compares a simplistic repeated version versus the batch learners. Is the simplistic approach simply trained once and then repeatedly applied? Is there extra information available after a campaign has been done? Do these systems relearn between campaigns? If not, I think the simplistic approach may be at a disadvantage, but that is just my intuition (and I don't claim to perfectly understand everything :).

Although the use of a simulation model is nice in that it provides a form of evaluation for testing methods, I wished the paper had analyzed the effects of testing using a simulation. For example, is there any real data availabe for which the simulation can be validated? What affect will the simulation simplification have on the results? Does the simulation model favor any of the algorithms?

Finally, just a style note, I think Figure 6 should probably be eliminated. The information presented in it is also presented in 7 and I don't think that it is that beneficial to have BatchRL(sarsa) shown by itself.