85984

bash: all combinations of lines

Question:

I have the following file (this is semicolon delimited; the real file is tab-delimited)

abc;173959;172730 def;4186657;4187943 ghi;4703911;4702577 jkl;2243551;2242259

and I want to combine each line with each, so that my output would be:

abc;173959;172730;def;4186657;4187943 abc;173959;172730;ghi;4703911;4702577 abc;173959;172730;jkl;2243551;2242259 def;4186657;4187943;ghi;4703911;4702577 def;4186657;4187943;jkl;2243551;2242259 ghi;4703911;4702577;jkl;2243551;2242259

The order is not important.

I came up with the following awk-solution:

awk '{ a[$0] } END { for (i in a){ for (j in a){if (i != j) print (i "\t" j) } } }' file

But this prints me the combinations in both directions, so for example

abc;173959;172730;def;4186657;4187943 def;4186657;4187943;abc;173959;172730

Because I am pretty unfamiliar with python or perl, I kindly ask for a solution using awk/bash etc.

Answer1:

In awk:

$ awk '{ a[$0] } END { for(i in a) { delete a[i] # new place for delete for(j in a) if(i!=j) print i ";" j # delete a[i] # previous and maybe wrong place } }' file def;4186657;4187943;ghi;4703911;4702577 def;4186657;4187943;abc;173959;172730 def;4186657;4187943;jkl;2243551;2242259 ghi;4703911;4702577;abc;173959;172730 ghi;4703911;4702577;jkl;2243551;2242259 abc;173959;172730;jkl;2243551;2242259

Unfortunately the order is random.

Another way that restores the order and doesn't modify the a while processing (see comments) is:

$ awk '{ a[NR]=$0 } # index on NR END { for(i=1;i<=NR;i++) for(j=i+1;j<=NR;j++) # j=i+1 is the magic print a[i] ";" a[j] }' file abc;173959;172730;def;4186657;4187943 abc;173959;172730;ghi;4703911;4702577 abc;173959;172730;jkl;2243551;2242259 def;4186657;4187943;ghi;4703911;4702577 def;4186657;4187943;jkl;2243551;2242259 ghi;4703911;4702577;jkl;2243551;2242259

Answer2:

This awk should work as well:

awk -F ';' 'NR==FNR{a[++k]=$0; next} {for (i=FNR+1; i<=k; i++) print $0 FS a[i]}' file{,} abc;173959;172730;def;4186657;4187943 abc;173959;172730;ghi;4703911;4702577 abc;173959;172730;jkl;2243551;2242259 def;4186657;4187943;ghi;4703911;4702577 def;4186657;4187943;jkl;2243551;2242259 ghi;4703911;4702577;jkl;2243551;2242259

Answer3:

Could you please try following one, it will give you same order as Input_file's field values only, by reading the Input_file once only.

awk '{a[FNR]=$0} END{j=1;while(length(a)>=++k){for(q=j+1;q<=FNR;q++){print a[j]";"a[q]}j++};}' Input_file

<strong>OR</strong>

awk ' { a[FNR]=$0 } END{ j=1; while(length(a)>=++k){ for(q=j+1;q<=FNR;q++){ print a[j]";"a[q] } j++ } } ' Input_file

Output will be as follows.

abc;173959;172730;def;4186657;4187943 abc;173959;172730;ghi;4703911;4702577 abc;173959;172730;jkl;2243551;2242259 def;4186657;4187943;ghi;4703911;4702577 def;4186657;4187943;jkl;2243551;2242259 ghi;4703911;4702577;jkl;2243551;2242259

Recommend

  • How to replace value of key in json file using sed
  • Example from three.js but how to implement using SceneKit?
  • Django: nested content blocks with the same name
  • unable to fetch the selected option from mat-form-field to generate the url dynamically in angular2
  • handling $http.get and $http.post errors in Angularjs
  • a simple question about lib folder in Eclipse
  • Local Notifications for Corona SDK (Android)
  • Writing a Build Script for a Play! Framework application
  • how to get DAYS absent from working days from given date range?
  • Grow Animation for Floating Action Button
  • Apple review objection Invalid directory name for Plugins folder (Child Browser Cordova)
  • Can wso2 data service return affected rows after SQL update?
  • Intellij Idea Terminal shortcut not working
  • Merge arrays by common column values in julia
  • Remove characters after a specific character in column
  • Perl keyword say is not working in version 5.14.4
  • Extract data between rows r
  • Recursion Control flow
  • Extracting individual digits from a float
  • import scipy.sparse failed
  • In-place sed command not working
  • perl, mysql - fasting way to upload a csv file into mysql?
  • During installation of Django, why do I keep getting ImportError: No module named django?
  • where do I find the xml.dom python package for the python-2.6.0-8.9.28 and I have a suse/x86_64 vers
  • Python pickle not one-to-one: different pickles give same object
  • Get specific string
  • Appending Character to Character Array In C
  • Is there a perl module to validate passwords stored in “{crypt}hashedpassword” “{ssha}hashedpassword
  • Checking free space on FTP server
  • Excel - Autoshape get it's name from cell (value)
  • Check if a string to interpolate provides expected placeholders
  • How do you troubleshoot character encoding problems?
  • align graphs with different xlab
  • Return words with double consecutive letters
  • How to pass list parameters for each object using Spring MVC?
  • RestKit - RKRequestDelegate does not exist
  • Traverse Array and Display in markup
  • Transpose CSV data with awk (pivot transformation)
  • Reading document lines to the user (python)
  • Python/Django TangoWithDjango Models and Databases