14636

Unexpected results feeding Django File upload object to Python CSV module

Question:

I have no problems getting the file to upload and if I save it to disk, all formatting is intact.

I wrote a function to read in the the file within Django using:

data = csv.reader(f.read())

where f is the Django file object that I get from 'form.cleaned_data['file']' and yes the file is already bound to the form.

When I try to read the file using

for row in data: logging.debug(row)

I get an unexpected result in that it appears to be producing small packs of the data almost as if its reading some buffer. for example, for my float fields I get this when I log each row:

['0'] ['.'] ['0'] ['5']['', ''] ['0'] ['.'] ['2'] etc etc... where each item between the square bracket is actually from one row (ie. a newline)

csv.reader requires the object it takes to support the iterator protocol which I believe the Django File object <a href="http://docs.djangoproject.com/en/dev/ref/files/file/" rel="nofollow">does</a>

So what am I doing wrong?

Answer1:

You're actually passing the wrong iterable to csv.reader(). Try changing that line to:

data = csv.reader(f)

What you're doing is passing the whole contents of the file to the csv.reader() function, which will cause it to iterate over every individual character, treating each of them as a separate line. If you pass the actual file object to the function, it will iterate over the lines in that file, as you expect.

Answer2:

What happens if you try reading in the whole file, logging its first few bytes, and then passing it into the parser? Perhaps the file isn't being uploaded as you expect? An http form-encoding issue, perhaps?

Recommend

  • Specific Dynamic nested Dictionaries, Autovivification implementation
  • JSON.stringify not converting array.object correctly
  • Problem with form in Django
  • django inline_formset - form.empty_permitted = False doesn't work
  • Adding a new field along with username and password in django's login form
  • Python Django Getting user input
  • Intellij Idea Terminal shortcut not working
  • What is RSL (Runtime shared library ) used for in flash?
  • Calling C function from lua
  • Changing a global variable in C
  • How gzip file gets stored in HDFS
  • Connecting bluetooth device to windows phone 8 application
  • Error processing multiple files
  • Detecting null parameter in preprocessor macro
  • Azure webjobs output logs indexing taking very long
  • CXF JAXB JAXBEncoderDecoder unmarshalling error : unexpected element when having qualified elements
  • Unexpected token ILLEGAL while running node.js mocha test
  • ThreadStatic in asynchronous ASP.NET Web API
  • TextToSpeech.setEngineByPackageName() triggers NullPointerException
  • Using a canvas object in a thread to do simple animations - Java
  • How to determine if there are bytes available to be read from boost:asio:serial_port
  • Installed module is empty
  • Time complexity of a program which involves multiple variables
  • Can I check if a recipient has an automatic reply before I send an email?
  • Reading JSON from a file using C++ REST SDK (Casablanca)
  • Getting last autonumber in access
  • Perl system calls when running as another user using sudo
  • php design question - will a Helper help here?
  • Unanticipated behavior
  • using conditional logic : check if record exists; if it does, update it, if not, create it
  • How to disable jQuery.jplayer autoplay?
  • Unit Testing MVC Web Application in Visual Studio and Problem with QTAgent
  • AngularJs get employee from factory
  • What are the advantages and disadvantages of reading an entire file into a single String as opposed
  • -fvisibility=hidden not passed by compiler for Debug builds
  • How to CLICK on IE download dialog box i.e.(Open, Save, Save As…)
  • IndexOutOfRangeException on multidimensional array despite using GetLength check
  • Authorize attributes not working in MVC 4
  • Can't mass-assign protected attributes when import data from csv file
  • Unable to use reactive element in my shiny app