DataTaunew | comments | leaders | submitlogin
2 points by aflyax 3085 days ago | link | parent

Oh the privilege of dealing with datasets measuring KB’s (apparently) and being able to use read.csv().

For the “real-world” data, one is forced to use fread. Sometimes passing to it custom-written awk statements that filter the required rows and columns from a csv.



1 point by IndianAstronaut 3083 days ago | link

I am a big fan of Python's "with open" for streaming through large files.

-----

1 point by aflyax 3065 days ago | link

You can also chunk with pandas in Python. But some unfortunates are forced to work with R (e.g., because the Python community didn’t get around creating an equivalent of Shiny).

-----




RSS | Announcements