LangStop

uniq Command Builder — Linux uniq Examples

Linux uniq Command Builder (Filter Duplicate Lines)

The uniq command is used to filter adjacent duplicate lines in Linux. This interactive uniq Command Builder helps you generate precise uniq commands for deduplication and frequency analysis.

🚀 What is the uniq Command?

The uniq command allows you to:

  • Remove adjacent duplicate lines from sorted input
  • Count occurrences of each unique line
  • Show only duplicated or only unique lines
  • Compare lines while skipping fields or characters

⚙️ Basic Syntax

uniq [options] [INPUT [OUTPUT]]
sort file.txt | uniq [options]

🧠 How to Use This uniq Command Builder

This tool helps you:

  • Choose counting (-c), duplicate-only (-d), or unique-only (-u) modes
  • Ignore case with -i or skip fields with -f, -s, -w
  • Build the correct sort | uniq pipeline
  • Generate a ready-to-use uniq command

Workflow

  1. Sort your input so duplicates become adjacent
  2. Pick a display mode: default, -c, -d, or -u
  3. Add matching tweaks like -i, -f, -s, or -w
  4. Copy generated command

🔧 Common uniq Usage Patterns

Remove Duplicates (Sort First)

sort names.txt | uniq

Count Occurrences of Each Line

sort names.txt | uniq -c

Rank Lines by Frequency (Most Common First)

sort names.txt | uniq -c | sort -rn

Show Only Duplicated Lines

sort names.txt | uniq -d

Show Only Truly Unique Lines

sort names.txt | uniq -u

Case-Insensitive Dedup With Field Skipping

sort data.txt | uniq -i -f 1 -s 2 -w 10

⚠️ Important Notes

  • The #1 uniq pitfall is adjacent-only matching: uniq compares only neighboring lines, so unsorted input keeps scattered duplicates
  • Always sort first: the safe pattern is sort file | uniq, never bare uniq file on unsorted data
  • Counting: -c prefixes each line with its occurrence count
  • Filtering: -d keeps only lines seen more than once, -u keeps only lines seen exactly once
  • Fuzzy matching: -i ignores case, -f N skips N fields, -s N skips N characters, -w N compares only the first N characters

✅ Pro Tips

  • Pipe uniq -c into sort -rn for instant frequency rankings
  • Use -d to find duplicates before deciding what to delete
  • Use -u to isolate lines that appear exactly once
  • Combine -i with -w for case-insensitive prefix grouping