Tuesday, May 19, 2020

Linux Shell - What is IFS?



 IFS stands for internal field separator. This is the delimiter used when words are split.

The man page of bash tells :

IFS    The Internal Field Separator that is used for word splitting after expansion and to split lines into
              words with the read builtin command.  The default value is ``<space><tab><newline>''.

The default value of IFS is a space, a tab followed by a newline.
guru@unixschool:~$ echo "$IFS"
  

guru@unixschool:~$ echo "$IFS" | cat -tve
 ^I$
$
guru@unixschool:~$ 
  When we echoed IFS for the first time, we could not see anything becuase they are special characters. On using the tve options of cat, we can see a space, followed by a ^I which is a tab character and then followed by a newline.

1. Variable with space separated values:
guru@unixschool:~$ var="hi 25 hello"
We have declared a variable var with space separated values. Now, lets use for loop at the prompt itself to access the individual values:
guru@unixschool:~$ var="hi 25 hello"
guru@unixschool:~$ for i in $var
> do
>   echo $i
> done
hi
25
hello
guru@unixschool:~$ 
If you notice, the values got separated with the space as delimiter and the variable i got each value in every iteration.

2. Variable with tab separated values :
guru@unixschool:~$ var="hi      25      hello"
guru@unixschool:~$ echo $var
hi 25 hello
guru@unixschool:~$ echo "$var" | cat -tve
hi^I25^Ihello$
guru@unixschool:~$ 
guru@unixschool:~$ for i in $var
> do
>  echo $i
> done
hi
25
hello
guru@unixschool:~$ 
As shown above, the values got separated with tab as delimiter . And the value of i got each one in subsequent iterations. Please note when we print $var, we cannot see the tabs, only when you pipe with cat and use the tve option, we can.

3. Variable with values separated by a new line character:
guru@unixschool:~$ var="hi
> 25
> hello"
guru@unixschool:~$ 
guru@unixschool:~$ echo "$var"
hi
25
hello
guru@unixschool:~$ for i in $var
> do
>  echo $i
> done
hi
25
hello
guru@unixschool:~$ 
Like in earlier cases, the values got separated now with newline as delimiter.
   Basically, shell splits the value of variable as what is the value is IFS special variable. Since IFS has space, tab and newline, the variables will get split with any/all of them as  a delimiter.

4. Variable with comma separated values:
  Similarly, if we want to split a variable on the basis of comma, simply set it to IFS.
guru@unixschool:~$ IFS=","
guru@unixschool:~$ var="hi,hello,world"
guru@unixschool:~$ 
guru@unixschool:~$ for i in $var
> do
>   echo $i
> done
hi
hello
world
guru@unixschool:~$ 
As seen , the variables now got split with comma as separator the moment we set the IFS to comma.

5. Default IFS vs Custom IFS:
But there is a difference with the default IFS and the IFS value set by the user. In case of default IFS, the shell tries to separate taking into account sequence of spaces, sequence of tabs as one single separator, whereas it is not the case with user set separator.

5a. Variable with more spaces:
guru@unixschool:~$ var="hi 25     hello  "
guru@unixschool:~$ 
guru@unixschool:~$ for i in $var
> do
>   echo [$i]
> done
[hi]
[25]
[hello]
guru@unixschool:~$ 
If you notice, the shell considered the entire set of spaces as one delimiter. Square brackets are used to capture the variable value as is.

5b. Variable with multiple commas:
guru@unixschool:~$ var="hi,25,,,hello "
guru@unixschool:~$ IFS=,
guru@unixschool:~$ for i in $var
> do
>   echo [$i]
> done
[hi]
[25]
[]
[]
[hello ]
guru@unixschool:~$ 
As seen above, the shell took each comma as a separate delimiter unlike taking sequence of commas as it happened in case of default IFS. Due to this, we got empty some values, and also notice the space in the last value.

6. Reading a file with custom IFS:
Let us have a multi line file with a colon delimiter :
guru@unixschool:~$ cat user.txt 
guru:x:1001:1001:guru,,,:/home/guru:/bin/bash
oracle:x:1002:1002::/u01/app/oracle:/bin/bash
test:x:1003:1003::/home/test:/bin/bash
guru@unixschool:~$ 
 Now, we will have a script to parse the file using colon as IFS:
6a. Reading into multiple variables:
#!/bin/bash
  
IFS=":"
while read user home_dir shell
do
    echo "User : $user , Home Dir : $home_dir"
done < $HOME/user.txt
Output:
User : guru , Home Dir : /home/guru
User : oracle , Home Dir : /u01/app/oracle
User : test , Home Dir : /home/test
 Once the IFS is set to colon, the read started reading each value with colon as delimiter and read the individual values in user, home_dir and shell variables.

6b. Reading into an array:
  Instead of reading into multiple variables, we can also read into an array:
#!/bin/bash
  
IFS=":"
while read -a arr
do
    echo "User : ${arr[0]} , Home Dir : ${arr[1]}"
done < $HOME/user.txt

By using -a option of read, the values got read into arrays.

No comments:

Post a Comment