Thursday, August 16, 2012

15 examples of sort command in Linux



sort command is used to sort a file, arranging the records in a particular order. By default, the sort command sorts file assuming the contents are ascii. Using options in sort command, it can also be used to sort numerically. Let us discuss it with some examples:

File with Ascii data:
 Let us consider a file with the following contents:
$ cat file
Unix
Linux
Solaris
AIX
Linux
HPUX

1. sort simply sorts the file in alphabetical order:
$ sort file
AIX
HPUX
Linux
Linux
Solaris
Unix
All records are sorted alphabetically.

 2. sort removes the duplicates using the -u option:
$ sort -u file
AIX
HPUX
Linux
Solaris
Unix
The duplicate 'Linux' record got removed. '-u' option removes all the duplicate records in the file. Even if the file have had 10 'Linux' records, with -u option, only the first record is retained.

File with numbers:
 Let us consider a file with numbers:
$ cat file
20
19
5
49
200

3. The default sort 'might' give incorrect result on a file containing numbers:
$ sort file
19
20
200
49
5
In the above result, 200 got placed immediately below 20, not at the end which is incorrect. This is because the sort did  ASCII sort. If the file had not contained '200', the default sort would have given proper result. However, it is incorrect to sort a numerical file in this way since the sorting logic is incorrect.

 4. To sort a file numericallly:
$ sort -n file
5
19
20
49
200
   -n option can sort the decimal numbers as well.

5. sort file numerically in reverse order:
$ sort -nr file
200
49
20
19
5
'r' option does a reverse sort.

Multiple Files:
 Let us consider examples with multiple files, say file1 and file2, containing numbers:
$ cat file1
20
19
5
49
200
$ cat file2
25
18
5
48
200

6. sort can sort multiple files as well.
$ sort -n file1 file2
5
5
18
19
20
25
48
49
200
200
The result of sort with multiple files will be a sorted and merged output of the multiple files.

 7. Sort, merge and remove duplicates:
$ sort -nu file1 file2
5
18
19
20
25
48
49
200
-u option becomes more handy in case of multiple files. With this, the output is now sorted, merged and without duplicate records.

Files with multiple fields and delimiter:
 Let us consider a file with multiple fields:
$ cat file
Linux,20
Unix,30
AIX,25
Linux,25
Solaris,10
HPUX,100

8. sorting a file containing multiple fields:
$ sort file
AIX,25
HPUX,100
Linux,20
Linux,25
Solaris,10
Unix,30
As shown above, the file got sorted on the 1st field, by default.

 9. sort file on the basis of 1st field:
$ sort -t"," -k1,1 file
AIX,25
HPUX,100
Linux,20
Linux,25
Solaris,10
Unix,30
This is being more explicit. '-t' option is used to provide the delimiter in case of files with delimiter. '-k' is used to specify the keys on the basis of which the sorting has to be done. The format of '-k' is : '-km,n' where m is the starting key and n is the ending key. In other words, sort can be used to sort on a range of fields just like how the group by in sql does. In our case, since the sorting is on the 1st field alone, we speciy '1,1'. Similarly, if the sorting is to be done on the basis of first 3 fields, it will be: '-k 1,3'.

Note: For a file which has fields delimited by a space or a tab, there is no need to specify the "-t" option since the white space is the delimiter by default in sort.

 10. sorting file on the basis of the 2nd field:
$ sort -t"," -k2,2 file
Solaris,10
HPUX,100
Linux,20
AIX,25
Linux,25
Unix,30

11. sorting file on the basis of 2nd field , numerically:
$ sort -t"," -k2n,2 file
Solaris,10
Linux,20
AIX,25
Linux,25
Unix,30
HPUX,100

12. Remove duplicates from the file based on 1st field:
$ sort -t"," -k1,1 -u file
AIX,25
HPUX,100
Linux,20
Solaris,10
Unix,30
The duplicate Linux record got removed. Keep in mind, the command "sort -u file" would not have worked here becuase both the 'Linux' records are not same, the values were different. However, in the above, sort is told to remove the duplicates based on the 1st key, and hence the duplicate 'Linux' record got removed. According to sort, in case of a group of similar records, except the first one, the rest are considered duplicate.

13. Sort the file numerically on the 2nd field in reverse order:
$ sort -t"," -k2nr,2 file
HPUX,100
Unix,30
AIX,25
Linux,25
Linux,20
Solaris,10

14. sort the file alphabetically on the 1st field, numerically on the 2nd field:
$ sort -t"," -k1,1 -k2n,2 file
AIX,25
HPUX,100
Linux,20
Linux,25
Solaris,10
Unix,30

15. sort a file based on the 1st and 2nd field, and numerically on 3rd field on  a file containing 5 columns:
$ sort -t"," -k1,2 -k3n,3 file 

23 comments:

  1. Thank you very much, these examples are so helpful

    ReplyDelete
  2. how to sort on decimal numbers

    ReplyDelete
  3. very useful and helpfu :)
    But i didn't understand this (14. sort the file alphabetically on the 1st field, numerically on the 2nd field: )

    ReplyDelete
    Replies
    1. Since 1st field is alpha,we sort alphabetically, 2nd being numeric, it is sorted numerically..

      Delete
  4. Thanks for the clear explanation...

    ReplyDelete
  5. Hello I want to sort list of directories listed in a text file as

    12-15-2015.20.40
    12-01-2015.21.35
    12-14-2015.20.35
    12-03-2015.22.45

    I want to omit everthing after first "." delimiter. Any way to do this ?


    And expected result is
    12-01-2015.20.35
    12-03-2015.22.45
    12-14-2015.20.35
    12-15-2015.20.40

    ReplyDelete
  6. All the "sort" command examples are simple and powerful. Great job and thank you so much!

    ReplyDelete
  7. very helpful with good examples
    on 14th example sorting with alphabetical in first field and numerical in 2nd field
    result shows it was sorted only by first field alne...
    is that will sort only with alphabet neglecting 2nd field ?

    ReplyDelete
    Replies
    1. Sorting is done based on the 1st field. Sorting on 2nd field comes into picture only when 1st field value is same. In case of Linux in 1st field, you could see the 2nd field values are sorted numerically.

      Delete
  8. Thanks for the explanation. Works like a charm. Really Helpful.

    ReplyDelete
  9. for the sort if it contains delimiter for the 8th command it will also sort when we give "sort -d +1 -2 file" if it is possible then what is the use of command sort -t

    ReplyDelete
  10. hi, from the example provided, and the result. How could be just obtain the 1st field after sorting with 2nd field?

    $ cat file
    Linux,20
    Unix,30
    AIX,25
    Linux,25
    Solaris,10
    HPUX,100

    $ sort -t"," -k2,2 file
    Solaris,10
    HPUX,100
    Linux,20
    AIX,25
    Linux,25
    Unix,30


    Actual output needed:
    Solaris
    HPUX
    Linux
    AIX
    Linux
    Unix



    ReplyDelete
  11. You can pipe the sort command to cut or awk:
    sort -t"," -k2,2 file | awk -F, '{print $1}'

    ReplyDelete
  12. Still U haven't discussed about the '.' selection
    sort -t"," -k 2.2,2.4 file
    This starts the selection from 2nd character of 2nd field to 4th character of 2nd field itself.
    However I loved the stuff and helped me out .
    :-)

    ReplyDelete