We have some notes on using differential privacy to slow down over-fitting here: http://www.win-vector.com/blog/2015/10/a-simpler-explanation...

Oh I hadn't considered the statistical advantage here.

You do lose out on a lot of human bias in the research process, but you also create blind errors that are hard to validate.

I know in my work there is plenty of times I run analysis and go back and manually check some entries as a sanity check - pros and cons here!

The thresholdout method [0] for preventing overfitting on a test set is an interesting application of this.

Here's a talk on differential privacy applied to the overfitting problem [1]

[0] http://andyljones.tumblr.com/post/127547085623/holdout-reuse

[1] https://www.youtube.com/watch?v=9mqXjdnZA18