| Hsin-Hao Yu
|
5
|
 |
|
10-23-2001 05:43 AM ET (US)
|
|
1. This is probably an idiotic suggestion, but I can't resist proposing the following experiment - since the boosting procedure can be applied to ANY learning device, we can certainly apply it to human learners. Let's train a series of human experts on a classification task (eg. identifying obscure insects, for example) according to the principles of AdaBoost, and see if this is more efficient than training one single expert. Of course "human nature" will make the mathematical arguement in the papers invalid, but what the heck.
2. Maybe it's because I didn't really follow the mathematics of the second paper, but I am quite skeptical about training a community of experts to vote. I read an example of AdaBoost somewher, where the trained individual expert solve the problem incorrectly (eg. they capture the underlying distribution of data very wrong), but the non-linear voting procedure somehow made the net result quite accurate. Yes, this is the whole point of community machine, but intuitively, if each expert simply doesn't "get it", we cannot expect that the voting procedure will magically make the problem solvable.
A paper by Jeff Elman (the "starting small" paper) showed that in order to solve a problem (especially when you are using recurrent nets), sometimes the network need to be trained on a much easier task. Without it the network is not able to solve the problem. It seems that there are a lot of un-explored possibilities of re-assigning the distributions of the training data, possibily involving making the problem easier in some non-trivial way, instead of AdaBoost's "making it harder" strategy.
|