Working with large datasets in Python

How can you work with large datasets in Python -- millions, billions, or more records? Here is a great answer to that question: https://www.quora.com/Can-Python-Pandas-handle-10-million-rows-What-are-some-useful-techniques-to-work-with-the-large-data-frames Pandas is memory hungry, you may need 8-16GB of memory or more to load your dataset into memory to work efficiently. You can use large / extra large AWS cloud systems … Continue reading Working with large datasets in Python