with open("C:\Anaconda3\FalkParameters\AllModels.csv","r") as f:
reader = csv.reader(f)
listofModels = list(reader)
AllModels is a huge csv file (2.4 GB). I am not sure how many rows are in there, because I can't open it with any application (notepad, notepad++, excel, etc.).
The code above runs okay, but if I try to print "listofModels", the console starts printing, but at some point I get a "ValueError: I/O operation on closed file". Printing stops at different rows each time.
I was wondering if there is a way I can deal with huge csv files in python.Answer1:
Don't convert it to list. Rather use generator instead. print one value/line at a time.
for line in reader: print line
Hope this helps !