I was able to relate myself to this paper more readily than the other ones since its topic was managing traffic where I take a selfish role on some days while polite ones on other days. :)

I had a minor question about the performance function P (equation 1) on page 3. The paper states that it wants to 'maximize equation 1' but I am failing to see why one would want to do this. I think I understand the motivations behind the choice of two terms that compose P. In order to maximize P, one would like to maximize the first term, which is the sum of squared errors, and minizmize the second term, which is sum of number of lane changes. However maximizing the first term contradicts the motivation behind the choice of this term-- smaller squared errors is desired since it means that cars are more or less driving at speeds that they want.

Yohan,
That's actually an error in the paper. It is correct in one version of the paper that they published, but not the one that we are reading. The '-' should be a '+' and then we want to minimize that equation.

Edited by author 04-18-2002 06:13 AM
This was an interesting paper. I like the way the authors simplified the problem by just wanting to keep the driving speed constant for each driver and by wanting to minimize the number of lane changes. Although I think the author correctly points out at the end that wanting constant speeds is a big assumption, given that people may want to alter their speeds, especially younger drivers. Also practically it does seem that poor traffic usually happens the most when encountering on and off ramps, and not because of drivers having differing desired speeds.

In the beginnning it also seems like this problem is formulated as an optimization problem (read: Figure 1) and not a learning problem. By that I mean early he phrases the problem as though there is 'one' best arrangment of cars and lanes that all the drivers should strive for.

I think there is an absence of details in the paper about the learning algorithms. Each is described at a high level but detail is omitted, which is bad. Also it seemed as though the system was hacked together, employing many different learning strategies and there really wasn't a central idea other than that ai can be applied to this problem. Is he using reinforcement learning, ANN's, local search, intelligent heuristics? It seems like a hodgepodge of learning algorithms.

However he does get good performance, which is promising. Also though this paper seems a bit dated, with no reference being later than 1997. I know this problem is currently being looked at in EE/ECE depts, but I don't think ML people have touched it in a while. Why is that?

Edited by author 04-18-2002 07:05 AM
In response to the fact that “governments want high traffic throughput” I think it would have been interesting for the authors to include results describing the traffic throughput in their evaluation. Intuitively it seems that minimizing the difference from desired speed also maximizes total traffic throughput (unless cars are speeding up for traffic).

Personally, if I where own one of Moriarty and Langley’s new Mercedes, I would opt to turn off the "lane selection feature"

I agree with Joe about there being far too little detail about their approach. They only give one equation, and naturally their learning method will be better at optimizing it if the learning method is the only method trying to optimize it. Additionally, it is very unclear how the populations of driving strategies change, and perhaps learn from each other. The heuristics they employ (seeding with intelligent knowledge, selfish strategy, and polite strategy) are unclear. This is a very application-oriented paper.



The results in figures 10 and 11 are interesting, which show that having a small amount of "smart cars" in traffic contributes to improving the overall flow of traffic (according to their metric).

This paper is different than most papers we read because it was created in response to a new problem. While a lot of papers I read build strongly upon past research, this paper was more a first stab at trying to solve a unique problem. I think that leaves it open to many criticisms and much improvement. For example, there is not much reasoning about which learning algorithm to use, and the SANE RL method seems arbitrary and overly complicated for a first attempt. In addition, while it seems the authors went through some effort to keep their learning algorithm from making simplifying observations, the algorithm learned in an oversimplified simulated traffic environment.

I think the problem addressed in this paper is interesting, and there is much room for improvement in this algorithm. I would be interested to see how this algorithm actually works in real traffic. It seems like it would be pretty harmless to test it in actual traffic, particularly since it is meant only as advice to the driver. I would also be interested to see how real customers react to the advice given. There have been some suggestions in the AI lab that such "backseat driver" advice would be annoying.

I have to agree with the previous comments about the lack of detail in the paper.

I do think that traffic control is an important problem to address considering our congested highways, however i did not find the approach considered here too useful for several reasons.

The learning system proposed in the paper was a bit of an overshoot for the problem and was not formally justified in the paper.

Parts of the domain knowledge were dangerous and i believe against the most traffic laws.

The assumptions made by the simulator were far from reality, hence the usefulness of results is highly questionable.

If i'm not mistaken after a few simple calculations one can see that the most severe traffic conditions tested are mildest one in reality. 400 cars in 3 (or 4) lanes over a stretch of 13.3 miles assuming a uniform distribution amounts to roughly 160 meters or 1/10 miles with no cars in front of a given car on average.

While the lane change results show that the policy learned results in more stable driving strategy, the policy was not so successful in achieving the ultimate goal of reducing travel time. From Figure 5a) one can see that the difference between the "errors" of the the learned polity and polite policy are negligible (i.e.: in the case of 400 cars with the learned policy one goes on average 6 miles "bellow" their desired speed, and with the polite policy one goes only 8 miles "bellow" their desired speed. I hardly consider this such a big a gain. You may get to work 1 minute earlier if you are a far-commuter.

Furthermore, I'm not sure how one would like to take directions from a car when people hardly take the advice of other people sitting in the same car.

I think the paper presents an attempt to a smooth transition from manual cars to truly computer guided cars. I truly think that the solution to the ultimate goal of increasing throughput on our highways lies in co-operations between cars.

I think the main idea of this paper is to apply the first author's PhD thesis to a real problem. The author leaves many algorithmic details in his thesis, and put emphasis on the formulation of the problem and the experimental results.

As many pointed out, the central algorithm (SANE) is really a complicated one and maybe a overkill, especially the neural network part. It would be nice if the authors rephrase briefly the algorithm to make the paper self-contained.

Formulation the problem in terms of symbiotic RL is interesting. Is this problem related to Ecology systems? Is there similar works in recent RL literature?