Well, sure, but if you're going to take that tack, then you might as well say "it's all just statistical learning anyway." Which is true, the main reason why ML and statistics are different things is because they've historically grown from different scientific disciplines and have yet to fully merge, not because they're inherently different or because you have to choose just one. But that's not very informative when trying to explain the real differences in attitude and approach between these different communities.
Bayesian inference is actually case in point. If you read any of the work on MCMC and probabilistic programming, machine learning hardly ever gets mentioned because the scholars pushing MCMC don't identify with that community. Why? No reason why, that's just how it is. The only time I have seen MCMC explicitly mentioned as an ML technique is in http://www.mbmlbook.com/, where it's part of a conscious attempt by the author to win over people used to more mainstream ML techniques like random forests and SVMs.
