What self-respecting terrorist is going to go to the library now to get info on how to make bombs? None! Or if they do, you can bet they'll bypass the super-high security forms that you have to fill in when you get your library card.
Laws generally restrict those who obey the laws more than criminals. After all, by definition, criminals already disobey the law.
Does that mean that the laws need to go? No. I think some laws are necessary. The question is really whether the 'law' in question actually does what it's supposed to do. I don't think this one does.
Also, keep in mind that the FBI has wanted the ability to secretly access your library records for a LONG TIME. They wanted this long before 9/11, and until 9/11 they've, to the best of my knowledge, never claimed that it was to prevent terrorism.
The Patriot Act was a clearing house for all of those police powers that law enforcement has been wanting all along.