Overall, I found the paper interesting, particularly with the use of 'virtual reinforcement learning'. Often game playing agents neglect to analyze their opponent's strategy, and this paper deals with the issue by cleanly incorporating the other player's decisions.

However, I was a little disappointed that the algorithm did not explicitly deal with sequences of moves over time.
A related game is the 'Prisioner's Dilemma' where players select whether to coordinate or defect against one another and get points based on a payoff matrix. In this game, one of the best strategies is 'tit-for-tat', where each agent replies with the last move his opponent made. This notion of time, or sequences of moves would be interesting to analyze with the Ultimatum Game.