Uniq Command in Linux

Introduction

uniq is a command used to detect and filter the duplicate lines in text files or strings. This command is very useful to do that.

Speak in an easy-to-understand way, this command will filter out the adjacent repeated lines from INPUT and write to OUTPUT.

Below is the guide on how to use the uniq command in Linux.

The structure of uniq command

$ uniq [option] [input[output]]

[option]

-c (count) display the number of times a line was duplicated

-d (duplicate) just only print the duplicate lines

-f N (skip-fields) skip N fields of a line then determine the uniqueness of a line

-i (ignore case) by default, the command is case identification but when using this option, it doesn’t do that.

-s N (skip-chars) skip N special characters

-u (unique) print only unique lines

uniq command with examples

1. -c (count)

Now we will try creating a file named count.txt by cat command then type the following lines:

$ cat > count.txt

Then we use uniq with -c option to count lines was duplicated:

$ uniq -c count.txt

Output:

The numbers before each line are the number of repetitions of that line.

2. -d (duplicate)

Now we will try creating a file named dup.txt by cat command then type the following lines:

$ cat > dup.txt

Then we use uniq with -d option:

$ uniq -d dup.txt

Output:

As you can see, only duplicate lines are printed out.

3. -f N (skip-fields)

Now we will try creating a file named fn.txt by cat command then type the following lines:

$ cat > fn.txt

Then we use uniq with -f N option:

$ uniq -f 2 fn.txt

Output:

This option is useful with the lines are numbered. N = 2 means the command will compare from the 2nd line onwards.

3. -i (ignore case)

Now we will try creating a file named ignore.txt by cat command then type the following lines:

$ cat > ignore.txt

Then we use uniq with -i option:

$ uniq -i ignore.txt

Output:

By default, the command is case identification but when using this option, it doesn’t do that.

4. -s N (skip-chars)

Now we will try creating a file named skip.txt by cat command then type the following lines:

$ cat > skip.txt

Then we use uniq with -s N option:

$ uniq -s 3 skip.txt

Output:

As you can see, N = 3 means the command omitted the first 3 characters and start filter out.

5. -u (unique)

Now we will try creating a file named unique.txt by cat command then type the following lines:

$ cat > unique.txt

Then we use uniq with -u option:

$ uniq -u unique.txt

Output:

The line “MUSIC” is unique and it was printed out.

Conclusion

You have just read a tutorial about how to use the uniq command in Linux.

Thanks for reading.