I interviewed with Jeremy's team at Sailthru using this process.
Result: Take Home -> Data Day -> No Offer.
It was an overall great experience. Afterwards I felt like I had a really good idea of the "fit". I understood the kinds of projects I'd be working on, I understood how their team functions, and most importantly I understood that I likely wouldn't be a good fit.
If I have a differentiating skill; it's Data Mining. A skill that I wasn't really able to show off.
From the article (regarding the sample data set):
"... I would suggest keeping this sample reasonably clean to ensure that they don’t waste valuable hours on munging that would otherwise have gone into analysis or modeling."
That pretty much kills my wow-factor. Most of the problems I work on don't start with a structured data set to build from, generally the biggest challenges I face (and enjoy) involve identifying and structuring the data needed to answer a question.
Hence the understanding that I wouldn't be a good fit.
Data day was a lot of fun, I certainly didn't consider it a waste of time; even knowing right off the bat I wouldn't be able to really shine. As the candidate, I garnered much insight about the job's nature and got to meet some awesome people.
Advice to other companies considering implementing a similar system:
1. Do it.
2. Tailor Data Day to select for the kind of skill set you need the most i.e If you want a candidate like me, be more withholding with your data :)
Great feedback and I really appreciate your positive words on the process despite the "no offer" outcome.
I will think more about expanding the variety of data included in the data day (we certainly have a ton of it!). But I worry about candidates getting lost in the complexity or pursuing too many rabbit holes. Having a narrower dataset increases the chance that the candidate can produce something meaningful in a fixed amount of time. It also increases the comparability across data days.
But I agree that a wider and less structured set of data would be a more realistic experience.
