DataTaunew | comments | leaders | submitlogin
1 point by TheCartographer 3270 days ago | link | parent

Oh god, are you me? The len() / length() issue is non-trivial for me; for some reason (and even now) I cannot remember which goes with which and have to pound the keyboard before I can figure it out.

My own knowledge of R predates my knowledge of python by a few years, so I've been loathe to switch to pure python for that reason, and because ggplot2 produces such pretty charts and is very easy to use.

Generally though, I think that most pre-processing and raw data handling is best done in python, and I will usually use it to do something like troll through a directory of raw sensor data, strip out the metadata and values, and import them into a PostgreSQL database. Python's syntactic sugar - list and dictionary comprehension mostly - let's you batch process raw text tables in a minimum amount of code.

Using constraints in postgres are the fastest and easiest way to ensure a proper qa/qc of the data. If postgres starts barfing errors back at you, it's pretty trivial to either adjust your python coding to identify and handle specific problem cases, or to except the error and insert the problem row or values into a text table.

To my mind, though, R is the only way to go for visualization and/or statistical analyses. ggplot2 is just too easy to use and too powerful - I have yet to find anything that can compare, particularly in the quality of charts it produces and the ease of handling multivariate data. Any sort of pre-processing or formatting of data in R is an absolute bear though - something about using tapply() and sapply() ties my poor brain in knots. I find any sort of complex, functional, or iterative programming in R is a nightmare.

The other thing that I wish R did better was handle and plot spatial data. To date, I have yet to find a good package for making maps. ggplot2 is fine for simple point maps, but complex polygons are a friggin nightmare.



1 point by lamlink 3270 days ago | link

Have you tried ggmaps? What about bokeh in python?

-----

1 point by TheCartographer 3270 days ago | link

ggmaps I have tried. I don't remember what the exact issue was but I will update the package and take another look. :-P

bokeh I haven't heard of, but I will definitely check it out.

Thank you for both the recommendations!

-----

1 point by Lofkin 3270 days ago | link

For exploratory plotting in python, there is also seaborn... though bokeh is likely poised to play this role as well in the future.

Sure!

-----




RSS | Announcements