"The thing that worries me about this push towards encryption and anonymity is that it does afford nefarious elements a lot of safe harbor"
I have yet to see any evidence that this is true; all I see are anecdotes from law enforcement agencies who are pushing for less crypto, and even those anecdotes only tenuously describe "nefarious" elements. You cannot encrypt a blood stain or a fingerprint, nor can you use an anonymity system to hide the fact that a known terrorist group is increasing the volume of data it sends and receives.
So what nefarious elements do you think are going to avoid prosecution by using crypto? A few years ago, there was a case of a group of child molesters using PGP and anonymous remailers (and possibly other privacy technologies) to communicate and exchange photos of themselves abusing children over Usenet. They were caught, arrested, and prosecuted following a well-coordinated investigation -- otherwise known as "good police work." Only a handful managed to avoid identification or prosecution, and it is worth noting that this was the case with investigations of criminal organizations long before good encryption was widely available. It should hardly be surprising that such a group was ultimately caught: they were sending each other photographic evidence of their own crimes.
It is also worth pointing out that anonymity systems are used by the police to catch criminals. In the case described above, the police used the very anonymity system that the child abusers were using, and were thus able to observe their messages while maintaining the secrecy of their investigation. Similar scenarios have played out with Tor. Criminals who use the Internet will eventually figure out which IP addresses belong to the police; it will be critical for the police to use anonymity systems to mask their IP addresses (this, in fact, is closely related to the reason the Navy created Tor in the first place). It is not so much that society's nefarious elements are more empowered now; it is more that the nature of the game has changed, that new tactics and strategies will be developed by both criminals and the police to utilize and cope with these systems.
I would argue that crypto and anonymity systems have benefited society more than they have harmed it. The same crypto and anonymity technologies used by the group I mentioned above have been used by political dissidents and activists to protect themselves from abusive governments; human rights activists have used such systems to protect the people they work with. Whistleblowers have used such systems in the past, and will have to do so in the future. For people who do not have an army at their command, these sorts of systems are necessary for their protection. If you were going to report Mafia activity to the police, would you rather use Tor/remailers/etc. or would you walk into the police station in person?
There is no such thing as a world of transparency and openness, because crime is part of human nature and because political ambition is part of human nature, and both of these behaviors exploit openness and transparency. I am all for an open government, but even I acknowledge that the government will need to keep some things secret -- military plans, investigations of dangerous criminal enterprises, the locations of witnesses to crimes, and many other things must be kept secret for society's benefit. Open and transparent government does not mean "secret free," it means "secrecy when it is absolutely necessary;" it is the responsibility of citizens to ensure that their representatives in the government are not declaring too many things to be secret, and it is the job of journalists to report to the citizens what is being kept secret from them. The sooner society realizes that, the better.
Thanks for the detailed response - first of all, I don't see all uses of crypto as problematic by any means - and I'm very glad that good police work and investigation can, and does, track down malicious behaviour even when enabled by the latest technology. Ultimately all these things are about human networks at the end of the day, and we're all fallible, and I generally trust that there's more sunlight than darkness.
I also completely agree that pretty much every technological innovation throughout history - weapons, communication mediums, etc - are used by both 'police and thieves', or whichever actors fit in the white/black boxes in the given situation.
The problem in my mind is more general - it's that crypto and concealment are just an evolution of the status quo - the arms race continues, with more 'secrets whispered in the woods', yet average people are already losing their day-to-day privacy, in vast numbers, and crimes still occur of course.
Opening everything is clearly radical and/or impossible, and it's a long-term idea/concept rather than anything feasible in the near future. Despite my ramblings, I'm a realist, and I've worked on migrating many, many legacy systems, so I'm familiar with the challenges, but this is the only process I've reasoned about so far regarding disruption of the arms race itself, as opposed to just evolution of arms, so I'm hunting for counter-arguments and these are good ones.
Enabling dissidence is a very good point, and I think it highlights the problems with hierarchy/power itself and information disparity. If neither party was able to operate without the knowledge of the other, then dissidence could take place openly without fear of hidden/unknown retribution. Trusting that the system is really 'open' and that you can see all the communications is definitely a challenge though (unsolvable?)
Regarding your final point, I'd say that although crime is part of a darker aspect of human nature (which can be encouraged, manipulated or instilled), we have been controlling our own education and evolution for generations through society and religion and choice of partners - so why can't we see this as an optimization problem, and try to guide ourselves towards a less criminal and violent nature?
(PS: the final point is a bit rhetorical - I think we have been for a long time already - cannibalism isn't hugely popular for example)
I have yet to see any evidence that this is true; all I see are anecdotes from law enforcement agencies who are pushing for less crypto, and even those anecdotes only tenuously describe "nefarious" elements. You cannot encrypt a blood stain or a fingerprint, nor can you use an anonymity system to hide the fact that a known terrorist group is increasing the volume of data it sends and receives.
So what nefarious elements do you think are going to avoid prosecution by using crypto? A few years ago, there was a case of a group of child molesters using PGP and anonymous remailers (and possibly other privacy technologies) to communicate and exchange photos of themselves abusing children over Usenet. They were caught, arrested, and prosecuted following a well-coordinated investigation -- otherwise known as "good police work." Only a handful managed to avoid identification or prosecution, and it is worth noting that this was the case with investigations of criminal organizations long before good encryption was widely available. It should hardly be surprising that such a group was ultimately caught: they were sending each other photographic evidence of their own crimes.
It is also worth pointing out that anonymity systems are used by the police to catch criminals. In the case described above, the police used the very anonymity system that the child abusers were using, and were thus able to observe their messages while maintaining the secrecy of their investigation. Similar scenarios have played out with Tor. Criminals who use the Internet will eventually figure out which IP addresses belong to the police; it will be critical for the police to use anonymity systems to mask their IP addresses (this, in fact, is closely related to the reason the Navy created Tor in the first place). It is not so much that society's nefarious elements are more empowered now; it is more that the nature of the game has changed, that new tactics and strategies will be developed by both criminals and the police to utilize and cope with these systems.
I would argue that crypto and anonymity systems have benefited society more than they have harmed it. The same crypto and anonymity technologies used by the group I mentioned above have been used by political dissidents and activists to protect themselves from abusive governments; human rights activists have used such systems to protect the people they work with. Whistleblowers have used such systems in the past, and will have to do so in the future. For people who do not have an army at their command, these sorts of systems are necessary for their protection. If you were going to report Mafia activity to the police, would you rather use Tor/remailers/etc. or would you walk into the police station in person?
There is no such thing as a world of transparency and openness, because crime is part of human nature and because political ambition is part of human nature, and both of these behaviors exploit openness and transparency. I am all for an open government, but even I acknowledge that the government will need to keep some things secret -- military plans, investigations of dangerous criminal enterprises, the locations of witnesses to crimes, and many other things must be kept secret for society's benefit. Open and transparent government does not mean "secret free," it means "secrecy when it is absolutely necessary;" it is the responsibility of citizens to ensure that their representatives in the government are not declaring too many things to be secret, and it is the job of journalists to report to the citizens what is being kept secret from them. The sooner society realizes that, the better.