In case it's not immediately obvious to you why you wouldn't want to walk around in public broadcasting your financial and/or medical information to anyone with an RFID reader, Bruce Schneier spells it all out for you in a great post on the Bush administration's
plans to push for RFID-based passports. Here's a nice chunk of it, but you should read the whole thing.
These chips are like smart cards, but they can be read from a distance. A receiving device can "talk" to the chip remotely, without any need for physical contact, and get whatever information is on it. Passport officials envision being able to download the information on the chip simply by bringing it within a few centimeters of an electronic reader.
Unfortunately, RFID chips can be read by any reader, not just the ones at passport control. The upshot of this is that travelers carrying around RFID passports are broadcasting their identity.
Think about what that means for a minute. It means that passport holders are continuously broadcasting their name, nationality, age, address and whatever else is on the RFID chip. It means that anyone with a reader can learn that information, without the passport holder's knowledge or consent. It means that pickpockets, kidnappers and terrorists can easily--and surreptitiously--pick Americans or nationals of other participating countries out of a crowd.
It is a clear threat to both privacy and personal safety, and quite simply, that is why it is bad idea. Proponents of the system claim that the chips can be read only from within a distance of a few centimeters, so there is no potential for abuse. This is a spectacularly naive claim. All wireless protocols can work at much longer ranges than specified. In tests, RFID chips have been read by receivers 20 meters away. Improvements in technology are inevitable.