How to read config files with section in bash shell


I have the configuration file like this in sections

[rsync_includes] user data conf [rsync_exclude] tmp .pyc */vendor [javascript] utils data

I have the patterns which i want to exlude in rsync and other configuration data in that file

Now i am confused how can i use those patterns on command line

rsync -avz --exclude-from 'content from config file rsync exclude' source/ destination/

I am not sure how can read part of config file and then use on command line


To use --exclude-from you will have to isolate the relevant section of the config into a temporary file. This is easy to do with a bit of sed:

tmp_file=$(mktemp) sed -n '1,/rsync_exclude/d;/\[/,$d;/^$/d;p' config.file > $tmp_file rsync -avz --exclude-from $tmp_file source/ destination/

I am omitting error checking and cleanup for clarity.

Note that rsync can read the exclude pattern from the stdin for an - input, so this is even shorter:

sed -n '1,/rsync_exclude/d;/\[/,$d;/^$/d;p' config.file | \ rsync -avz --exclude-from - source/ destination/


<ul><li>The 1,/rsync_exclude/d excludes all lines up to the rsync_exclude section entry</li> <li>The /\[/,$d excludes everything from the start of the next section to the end of the file</li> <li>The /^$/d excludes empty lines (this is optional)</li> </ul>

All of the above extracts the relevant section from the config.


If your configuration file is in config.ini, then run a bash script:

rm rsync-filter while IFS= read -r line do case "$line" in \[rsync_includes\]) command=include ;; \[rsync_exclude\]) command=exclude ;; \[*) command= ;; *) [ "$command" -a "$line" ] && echo "$command $line" >>rsync-filter esac done <config.ini

After that runs, it creates rsync-filter which contains both the include and exclude rules and can be used with rsync as:

rsync -avz --filter='merge rsync-filter' source/ destination/

Separately, rsync offers the -F option which is equivalent to --filter='dir-merge /.rsync-filter'. This loads include/exclude rules from the file /source/.rsync-filter and, further, as rsync goes deeper into the directory tree, it will look for and load rules from .rsync-filter files that it finds and apply those rules to files in that directory and its subdirectories. This is a powerful way to keep and organize rsync rules.

Also, the order in which rsync reads include and exclude rules is important. With these filter files, you retain control over that order. That is an important advantage when you are trying to get rsync rules to work right.


I will admit that I'm not familiar with rsync, but I would format that data differently, myself.

# rsync-data-file+.txt rsync-includes:user rsync-includes:data rsync-includes:conf rsync-exclude:tmp rsync-exclude:.pyc rsync-exclude:\*\/vendor javascript:utils javascript:data

From there, you can do the following:-

#!/usr/bin/env bash set -x while read line do if [ $(echo "${line}" | sed -n '/rsync-includes/'p) ] then parameter=$(echo "${line}" | cut -d':' -f2) rsync "${parameter}" (other switches here etc) fi done < rsync-data-file+.txt

This way you can customise your command line depending on which group the parameter belongs to; so with parameters from the javascript group, you can log the operations to a different file, for instance.


#!/bin/sh typeset -A Nconfig # init array typeset -A Oconfig # init array , u can declare multiple array for each section.s while read line do if [ "$line" = "[SECTION1]" ] then SECTION1=1 SECTION2=0 continue fi if [ "$line" = "[SECTION2]" ] then SECTION1=0 SECTION2=1 continue fi if [ "$line" = "[SECTION3]" ] then SECTION1=0 SECTION2=0 continue fi if [ $SECTION1= 1 ] then if echo $line | grep -F = &>/dev/null then varname=$(echo "$line" | cut -d '=' -f 1) echo "Novar $varname" Nconfig[$varname]=$(echo "$line" | cut -d '=' -f 2) fi fi if [ $SECTION2 = 1 ] then if echo $line | grep -F = &>/dev/null then varname=$(echo "$line" | cut -d '=' -f 1) Oconfig[$varname]=$(echo "$line" | cut -d '=' -f 2) fi fi done < Config echo "SECTION1 FROM=${Nconfig[FROM]}" echo "SECTION2FROM=${Oconfig[FROM]}" [SECTION1] FROM=abc@pqr.com TO=abc@pqr.com SIZE=80 THRESHOULD=60 [SECTION2] FROM=xxxx@pqr.com TO=xxxx@pqr.com,yyyy@pqr.com SIZE=60 THRESHOULD=30 [SECTION3] FROM=AAAA@pqr.com TO=BBBB@pqr.com,yyyy@pqr.com SIZE=60 THRESHOULD=30 LOCATION=/mnt/device/user1/


  • How get client (user) machine name (computer name) with jQuery or JavaScript or server-side codes. (
  • In R:how to extract part of list of list
  • Change progress tracker to full span of parent container
  • Android - EditText not editable and not selectable
  • Linq, VB - Anonymous type cannot be converted to anonymous type
  • AccessViolation occurs in RichTextBox.ScrollToCaret
  • Nginx server_name regexp not working as variable
  • RecyclerView item layout_weight, sizes not updating
  • Embed URL Link in SMS via PHP
  • No console application in Visual Studio 2015?
  • Rails 4: Using PostgreSQL function in order causes error in query due to the includes table not bein
  • boto3 S3 Object Parsing
  • scipy.signal's convolve differs from calculated result
  • How to store confidential PDF documents (file system vs. SQL) if we only use forms authentication
  • python: Using ncurses when underlying library logs to stdout
  • Identify xml text elements with Schematron
  • Set selected using jQuery
  • Parsing a complicated array with GetJSON Jquery
  • Retrieving Application's DisplayName/UninstallString from Registry Using Powershell
  • Silverlight MVVM, stop SelectionChanged triggering in response to ItemsSource reset
  • How to delete first 7 characters of folder name by using batch script?
  • Python sum values in tuple in a list in a dictionary?
  • Use PHP to Replace HTML with HTML
  • $this->a->b->c->d calling methods from a superclass in php
  • How to create OLE Automation to be used with ClearExplorer
  • How to get the Owner of the ContextMenu (from Silverlight 4 toolkit)?
  • 'Edit' function for forum posts and such
  • select an input by value?
  • Add font awesome icon to custom add to cart button in Woocommerce 3
  • how to read to huge file into buffer
  • C++ STL stack pop operation giving segmentation fault
  • JQuery: Infinite input select
  • Drag and drop unicode TText in DelphiXe4
  • How to handle div that is created dynamically in a table
  • Google App Engine backend servlet not responding
  • Comma decimal separator is ignored by ASP.NET MVC model binder
  • ARKit code issue {unknown error -1=ffffffffffffffff error: Task failed with exit 1}