I find it facinating that an agent, knowing only the rules of the game, can build an effective neural network based on an evaluation of the game state alone. This defies intutition since the agent does not incorporate any kind of strategies or previous knowledge. The success of reinforcement learning on neural nets is known (InfoSpiders), but it is still surprising to see weights of a network evolve into a killer game playing strategy.

It seems odd that the selection of the next move is based on a one-ply min-max search only. It may be interesting to compare results of what would happen with a deeper search depth. I'm a little surprised that the authors did not justify this decision, or at least a write note for why the search depth is only one layer deep.