DataTaunew | comments | leaders | submitlogin
Gradient Boosting playground (github.io)
15 points by axelr 1730 days ago | 6 comments


2 points by niklosb 1721 days ago | link

Where does the option to rotate the trees come from? I've never seen this technique used, but it seems to make a huge difference. Is there an option for this in the XGBoost package or is this something you added?

-----

2 points by axelr 1721 days ago | link

That's my addition. As I explained in the demo, this is an alternative way of randomizing tree structure.

Such techniques are rarely used (Rotational Forest is the most known example), and in practice may work well only in the case when you have many measurements of the same nature (temperatures / pressures / exchange rates).

If you have nice domain-specific feature engineering, rotating trees only spoils the model.

-----

2 points by axelr 1730 days ago | link

Also, compare this with neural networks http://playground.tensorflow.org on a spiral dataset.

-----

2 points by tfturing 1728 days ago | link

Thanks axelr. Perhaps you could have a blog post in the future on how you made the GB playground using TF playground? It would be really helpful for people learning and teaching concepts on machine learning!

-----

2 points by axelr 1728 days ago | link

Hi, I wasn't using tensorflow demo code - it is too demo-specific and written in typescript (which is nice, but ...).

As of programming, I coded gradient boosting in an hour or so - as I've done that many times before in python, this was an easy part. Later ~2 hours were spent on adding parameters / features to GB.

Most of the time was spent on proper visualization (different plotting libs are slow or too complicated, so I finished up with doing everything with canvas using some tricks), some code optimization and correct layout + support of mobile devices (so demo should be working on most mobile devices). And this took a week or even more.

To sum up, time is usually spent on something ridiculous (wrong event handling, wrong plotting, wrong layout), not on math or machine learning :)

-----

2 points by axelr 1730 days ago | link

In demo you can play with parameters of gradient boosting and see how it works (or does not work).

-----




RSS | Announcements