70004

run a bash for loop in parallel

<h3>Question</h3>

I've got this script that does a credential lookup for each host, in an on-premise vault system, and then runs an ansible-playbook for it.

<pre class="lang-sh prettyprint-override">#!/bin/bash for host in `cat ~/.ansible/hosts` do SECRET=`/opt/vault/bin/get-admin-credential --tag=$host` HOST=`echo $SECRET | cut -d ';' -f1` LOGIN=`echo $SECRET | cut -d ';' -f2` DOMAIN=`echo $SECRET | cut -d ';' -f3` PWD=`echo $SECRET | cut -d ';' -f4` if [ -z "$DOMAIN" ]; then ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$HOST ansible_user=$LOGIN ansible_password=$PWD" --limit $host else ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$HOST ansible_user=$LOGIN@$DOMAIN ansible_password=$PWD" --limit $host fi done

This loops over each host sequentially, I've tried stuff with GNU parallel but haven't been able to do what I want, running the for loop with 5 in parallel.

Anyone point me in the right direction?


<h3>Answer1:</h3>

I don't have any <em>"ansibles"</em> or <em>"vaults"</em>, so this is completely untested but may get you close:

doit(){ host="$1" SECRET=$(/opt/vault/bin/get-admin-credential --tag=$host) HOST=$(echo $SECRET | cut -d ';' -f1) LOGIN=$(echo $SECRET | cut -d ';' -f2) DOMAIN=$(echo $SECRET | cut -d ';' -f3) PWD=$(echo $SECRET | cut -d ';' -f4) if [ -z "$DOMAIN" ]; then ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$HOST ansible_user=$LOGIN ansible_password=$PWD" --limit $host else ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$HOST ansible_user=$LOGIN@$DOMAIN ansible_password=$PWD" --limit $host fi } # Export doit function to subshells created by GNU Parallel export -f doit parallel -a ~/.ansible/hosts doit

Stylistically, there are maybe a few improvements. Firstly, shell variables consisting of upper case letters are reserved, so you shouldn't maybe use HOST, DOMAIN etc. Also, you can probably simplify all that unsightly cutting and echoing to extract the variables from the SECRET by using an IFS=';' and a read like this:

SECRET=$(/opt/vault/bin/get-admin-credential --tag=$host) IFS=';' read host login domain pwd <<< "$SECRET"

So, my best and final answer is:

doit(){ host="$1" secret=$(/opt/vault/bin/get-admin-credential --tag=$host) IFS=';' read host login domain pwd <<< "$secret" if [ -z "$domain" ]; then ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$host ansible_user=$login ansible_password=$pwd" --limit $host else ansible-playbook -i ~/.ansible/hosts ~/.ansible/windows.yml -e "ansible_host=$host ansible_user=$login@$domain ansible_password=$pwd" --limit $host fi } # Export doit function to subshells created by GNU Parallel export -f doit parallel -a ~/.ansible/hosts doit
<h3>Answer2:</h3>

You simply need to run ansible-playbook in the background using the & command terminator. Note, though, that the entire loop can be simplified and improved.

run_playbook () { ansible-playbook -i ~/.ansible/hosts \ -e "ansible_host=$2 ansible_login=$3 ansible_password=$4" \ ~/.ansible/windows.yml --limit "$1" } while IFS= read -r host; do secret=$(/opt/vault/bin/get-admin-credential --tag="$host") IFS=";" read -r shost slogin sdomain spasswd _ <<< "$secret" if [[ -n $sdomain ]]; then login="$slogin@$sdomain" fi run_playbook "$host" "$shost" "$login" "$password" & done < ~/.ansible/hosts

来源:https://stackoverflow.com/questions/60849931/run-a-bash-for-loop-in-parallel

Recommend

  • Angular and Firestore: FirebaseError: Missing or insufficient permissions. How add idToken while obs
  • VHDL indexed name issue
  • Best way to perform multiprocessing on a large file Python
  • Read Data From Http Response rarely throws BindException: Address already in use
  • AVLayerVideoGravityResize does not match on new devices, iOS 10?
  • How do you access the for loop of the parent component within a child component which has the inputs
  • Make a Powershell function/task run in the background
  • change placeholder and clear-icon color for ion-search bar not globally?
  • Linking LLVM causes gcov to fail
  • What is the difference between _mm512_load_epi32 and _mm512_load_si512?
  • python 2.7.3 while loop
  • Invoke MS Access Query which calls a VBA Function in a Module from Delphi ADO Component
  • Can not get clojure-contrib to load - FileNotFoundException
  • Autolayout collapsing space when removing intermediate views
  • Clojure: slurping structs from file fails with string attributes containing whitespaces
  • Need To Compile Keras Model Before `model.evaluate()`
  • Objective C read file wrong encoding
  • Access add new line to table
  • Write unittest for function with yield
  • Geofence triggering issues in Android
  • Attempting to combine Angular UI Bootstrap's Typeahead with Alert
  • Rails 4: Using PostgreSQL function in order causes error in query due to the includes table not bein
  • How can I get the value from the last td in the tr tag in jQuery?
  • How to find unmatched rows in oracle without using set operator and join & also Query the unmatc
  • Disappearing icons in WPF
  • How to access a bundled ES6 class in inline
  • Python: Why this error is coming?
  • Bash script to bring up and down an interface on loop
  • Isabelle matrix arithmetic: det_linear_row_setsum in library with different notation
  • Shrinking Bootstrap Navbar with logo on scroll
  • How to put an object in the air?
  • JavaMail connection problems [duplicate]
  • how can i get selectedRange.location value?
  • Getting the type of a “Type” in C# reflection
  • How to check if a database and tables exist in sql server in a vb .net project?
  • How do I add a mouse over tooltip to an Image using .DrawImage()
  • Using redis as an LRU cache for postgres