Good call. This post itself is a great example of getting arbitrary results when you don't handle experimental groups identically and when all your results are coming from a black box that you don't understand (in this case indeed's recommender weighting system).
When you say "arbitrary results when you don't handle experimental groups identically", do you mean there's an issue with entity resolution? Sounds like it to me.
Why use quotes when Indeed's recommended terms returned considerably more results?
I agree with you that enclosing terms in quotes returns additional search results. It's a good idea. It's less clear to me that we can say something substantive about these additional search terms.
You probably noticed that a data analyst is 1.46% of postings without quotes and 0.136% of postings with quotes on the end of June 2016.
By "right" I think he only means "the 'right' thing to use to achieve your goal," whether that goal is analyzing the labor market, finding a job, or whatever.
