Replace string that contains CRLF?


I'm reformatting a file, and I want to perform the following steps:

<ol><li>Replace double CRLF's with a temporary character sequence ($CRLF$ or something)</li> <li>Remove all CRLF's in the whole file</li> <li>Go back and replace the double CRLF's.</li> </ol>

So input like this:

This is a paragraph of text that has been manually fitted into a certain colum width. This is another paragraph of text that is the same.

Will become

This is a paragraph of text that has been manually fitted into a certain colum width. This is another paragraph of text that is the same.

It seems this should be possible by piping the input through a few simple sed programs, but I'm not sure how to refer to CRLF in sed (to use in sed 's/<CRLF><CRLF>/$CRLF$/'). Or maybe there's a better way of doing this?


You can use sed to decorate all rows with a {CRLF} at end:

sed 's/$/<CRLF>/'

then remove all \r\n with tr

| tr -d "\r\n"

and then replace double CRLF's with \n

| sed 's/<CRLF><CRLF>/\n/g'

and remove leftover CRLF's.

There was an one-liner sed which did all this in a single cycle, but I can't seem to find it now.


Try the below:

cat file.txt | sed 's/$/ /;s/^ *$/CRLF/' | tr -d '\r\n' | sed 's/CRLF/\r\n'/

That's not quite the method you've given; what this does is the below:

<ol><li>Add a space to the end of each line.</li> <li>Replace any line that contains only whitespace (ie blank lines) with "CRLF".</li> <li>Deletes any line-breaking characters (both CR and LF).</li> <li>Replaces any occurrences of the string "CRLF" with a Windows-style line break.</li> </ol>

This works on Cygwin bash for me.


<h3>Redefine the Problem</h3>

It looks like what you're <em>really</em> trying to do is reflow your paragraphs and single-space your lines. There are a number of ways you can do this.

<h3>A Non-Sed Solution</h3>

If you don't mind using some packages outside coreutils, you could use some additional shell utilities to make this as easy as:

dos2unix /tmp/foo fmt -w0 /tmp/foo | cat --squeeze-blank | sponge /tmp/foo unix2dos /tmp/foo

Sponge is from the <em>moreutils</em> package, and will allow you to write the same file you're reading. The <em>dos2unix</em> (or alternatively the <em>tofrodos</em>) package will allow to convert your line endings back and forth for easier integration with tools that expect Unix-style line endings.


This might work for you (GNU sed):

sed ':a;$!{N;/\n$/{p;d};s/\r\?\n/ /;ba}' file


Am I missing why this is not easier?



sed -e s/\s+$/$'\r\n'/ < index.html > index_CRLF.html


remove CRLF... go unix:


sed -e s/\s+$/$'\n'/ < index_CRLF.html > index.html



  • How to scale a fixed-width website to an iPad viewport
  • Pause Google ARCore in Unity
  • Docker build takes too long
  • What happens if I manually delete one of the VMs that Dataflow created?
  • Why can I create an instance of a class without storing it to a variable and still have everything w
  • How to use fmt.Sscan to parse integers into an array?
  • Compressing “n”-time object member call
  • Python : Finding an item in a list where a function return a minimum value?
  • C++ DLL Wrapper in vb.net Passing byte array of a byte array?
  • how to remove a div with same ids but display='block' and display='none' in JAVa
  • Creating My Symmetric Key in C#
  • garbled css name when styling within UiBinder
  • How to open html table in xls on click of a button
  • Use sed with regex and (
  • import scipy.sparse failed
  • In-place sed command not working
  • Find group of records that match multiple values
  • Get specific string
  • Differences in dis-assembled C code of GCC and Borland?
  • Android application: how to use the camera and grab the image bytes?
  • Unable to install Git-core+svn by MacPorts
  • Django simple Captcha “No module named fields” error
  • With Hadoop, can I create a tasktracker on a machine that isn't running a datanode?
  • Could not find rake using whenever rails
  • Read a local file using javascript
  • Sails.js/waterline: Executing waterline queries in toJSON function of a model?
  • Regex thinks I'm nesting, but I'm not
  • What is the “return” in scheme?
  • vba code to select only visible cells in specific column except heading
  • Redux, normalised entities and lodash merge
  • Do create extension work in single-user mode in postgres?
  • Do I've to free mysql result after storing it?
  • How can I estimate amount of memory left with calling System.gc()?
  • Confusion with PayPal's monthly billing cycle
  • Unanticipated behavior
  • Transpose CSV data with awk (pivot transformation)
  • How to disable jQuery.jplayer autoplay?
  • Flexbox equal height doesn't work
  • Can Visual Studio XAML designer handle font family names with spaces as a resource?
  • Sorting a 2D array using the second column C++