The underlying problem in tech hiring is that it is difficult to assess candidates in an interview format. Many of the skills and personality factors that make someone a success are dependent on context. Transplanting an engineer to a new context is uncertain, even for putative rock stars.
Many of us have presumably seen our share of brilliant people flame out because they could not adjust to the ambient value system, and figure out how to get the right things done. Often these people flourished in a prior context: school or another company with more or fewer constraints.
Assessing that context match is hard. The most successful way is to have a mutual trial period (ex: internship). Failing that, in interviewing for a full-time position, the closest you can get seems to be by trying to simulate what it'd be like to work together. That's why you try to throw them at coding exercises or difficult problems. "Brain teasers" are silly but coding challenges where you see how far they can go with some direction are actually pretty useful. They give you a large dynamic range within which to quicky hone in on someone's current skill level. More interesting than their current skill level is how quickly you think they can progress, and so that's what you try to simulate in an interview.
That is definitely an underlying problem. Interviews are too short and too stressful to accurately show how good a candidate is.
I'm not particularly good in interviews because I get nervous and stumble over my words. It is not at all like a real working environment, so I think any conclusions you draw from one are necessarily flawed. I recently read about Heroku's interview process, and I think it's great that they can invest so much in evaluating their candidates. They give each new candidate a project to work on and make a hiring decision after it's been completed. (See http://www.craigkerstiens.com/2011/12/02/how-heroku-works-hi...). I also think it's unsustainable as their company grows but I would love to be proven wrong.
Many of us have presumably seen our share of brilliant people flame out because they could not adjust to the ambient value system, and figure out how to get the right things done. Often these people flourished in a prior context: school or another company with more or fewer constraints.
Assessing that context match is hard. The most successful way is to have a mutual trial period (ex: internship). Failing that, in interviewing for a full-time position, the closest you can get seems to be by trying to simulate what it'd be like to work together. That's why you try to throw them at coding exercises or difficult problems. "Brain teasers" are silly but coding challenges where you see how far they can go with some direction are actually pretty useful. They give you a large dynamic range within which to quicky hone in on someone's current skill level. More interesting than their current skill level is how quickly you think they can progress, and so that's what you try to simulate in an interview.