Really, this is no different from the offline coding test, except that since the code is open source it might be useful to someone later.
I'd never do this Github interview for an open source project that was not very widely used, for the same reasons I'd never work on the company's closed source for an interview: exploitation.
In that sense, working on a toy problem is a big feature to me in an interview process, I know I'm not being cheaply exploited.
Also, other things equal, I prefer to do a code problem at interview because I know it will have helped filter out the fluff from getting into the company I'm applying to.
That's why it should be a cool, fun project. It's hard for me to see how one might get exploited by contributing as much time as he wants to an open-source project. Remember, the prospective employee uses the same experience to gauge what it's like to work with his prospective employer.
Assuming this company has fun work to do (you'd hope so), there's nothing stopping them just giving you their work.
It doesn't matter that it's open source if you're not going to use it and no-one else apart from them is (a very real possibility).
In essence, I'd happily agree with the article after changing this:
> "You come up with a cool idea of an open-source project"
with this:
> "You find a _popular_ open-source project"
Now you can be certain (not just optimistic) that the wider community will benefit from your work.
(edit: I'm thinking along the lines of jQuery, backbone.js, nodejs, django, memcached, sqlite, Nginx, Hadoop, Linux, ..., not some dev's/company's lonely github project)
It's true that you'll have more impact by contributing to a well-known project, but on the other hand the range of activities you can take part in is limited. Well-established projects usually don't call for high-level design activities - the sort that a tech co-founder would need to do. They're more about contributing patches and perhaps a new feature (after you've learned the project, which is usually quite big). Ultimately I think you get the same amount of karma if you work on a pet project and not a well-established one. And there's absolutely no reason why the company's sandbox project can't fill a real need and become popular with time.
I'd never do this Github interview for an open source project that was not very widely used, for the same reasons I'd never work on the company's closed source for an interview: exploitation.
In that sense, working on a toy problem is a big feature to me in an interview process, I know I'm not being cheaply exploited.
Also, other things equal, I prefer to do a code problem at interview because I know it will have helped filter out the fluff from getting into the company I'm applying to.