7798

Weird characters while reading file content

Question:

I'm not sure what is wrong:

for line in open(textfile, 'r'): print(line)

Output:

abcd

The file was created using textpad++ using Unix EOL and UTF8 encoding.

Now it works properly using Encoding with UTF-8 without BOM option on notepad++ . But why? I mean how could I convert all sent files to UTF-8 to avoid weird chars?

Answer1:

Specifying <a href="https://docs.python.org/3/library/codecs.html#standard-encodings" rel="nofollow">encoding</a> will solve your problem.

for line in open(textfile, 'r', encoding='utf-8-sig'): print(line)

<a href="https://docs.python.org/3/library/codecs.html#module-encodings.utf_8_sig" rel="nofollow">utf_8_sig</a>: UTF-8 codec with BOM signature

Answer2:

You must set the encoding of your file while reading it, using UTF-8.

Add a third parameter to your code, setting its enconding. From:

for line in open(textfile, 'r'): print(line)

to:

for line in open(textfile, 'r', encoding='utf-8-sig'): print (line)

Recommend

  • How can i fetch recursive data from Bill of materials tables
  • What is the appropriate character encoding for a git repo?
  • Retrieve JSON data returned from GOOGLE FINANCE API URL in android
  • Phpexcel display sheet(2) in html
  • Bash: How to do a variable expansion within an arithmetic expression?
  • Does Python have an equivalent of Perl's qq?
  • RichFaces 4 autocomplete is not showing suggestions
  • How do I split this string into an array?
  • C# Networkstream reads nothing
  • Xamarin PCLCrypto SHA256 give different hash
  • TSQL Rolling Average of Time Groupings
  • How to extract text from a PDF and decode characters?
  • Lua: Line breaks in strings
  • Synchronize windows folders
  • Splash Screen will not display
  • Easiest way to get current unix timestamp via XSL
  • Xamarin Android | Layout style
  • How can I include If-None-Match header in HttpRequestMessage
  • Sending cookie value via httpget but not getting the desired response
  • How to convert SOAP response with xsi values to json in WSO2esb
  • Is there a way to save the selected text and highlight it again once the page is refreshed?
  • Create Instant using a negative year
  • JBoss External Properties Files in Classpath
  • Android - Material Design - NavigationView - How to put vertical scroll?
  • How do I access an unhandled exception in an MVC Error view?
  • Why Encoding.ASCII != ASCIIEncoding.Default in C#?
  • Jquery UI tool tip close icon
  • D3 nodes and links from JSON with nested arrays of children
  • Google Custom Search with transparent background
  • Encrypt data by using a public key in c# and decrypt data by using a private key in php
  • How to recover from a Spring Social ExpiredAuthorizationException
  • SSO with signing and signature validation doesn't work
  • ILMerge & Keep Assembly Name
  • Deserializing XML into class C#
  • vba code to select only visible cells in specific column except heading
  • Large data - storage and query
  • WOWZA + RTMP + HTML5 Playback?
  • How to format a variable of double type
  • costura.fody for a dll that references another dll
  • To Get the radio button value in ruby on rails