NODELOG by THIVELAB
Subscribe
Home/Engineer/Linux / Shell/grep · cut · sort · uniq · w…
Linux / ShellBeginnerUbuntuDebianCentOSmacOSgrepcutsort

grep · cut · sort · uniq · wc: Practical Pipelines

A practitioner’s roundup of production patterns that combine grep regex, cut field splitting, sort, uniq deduplication, and wc counts into pipelines.

NONodelogUpdated 0 views
Contents
  1. grep — Pattern Search
  2. cut — Field Extraction
  3. sort — Sorting
  4. uniq — Deduplication
  5. wc — Counting
  6. Practical Pipeline Combinations
  7. Deduplication — sort · uniq in Practice
Bash
# Basic search
grep "error" /var/log/syslog

# Case-insensitive
grep -i "ERROR" app.log

# Regex
grep -E "^(GET|POST) /api" access.log

# Inverse match (exclude the pattern)
grep -v "^#" /etc/nginx/nginx.conf

# Context (3 lines before and after)
grep -C 3 "FATAL" app.log

# Recursive search
grep -r "TODO" ./src --include="*.ts"

# Print filenames only
grep -rl "deprecated" ./src

# Show line numbers
grep -n "listen" /etc/nginx/nginx.conf

cut — Field Extraction

Bash
# Extract a field by delimiter (colon, 3rd field)
cut -d: -f3 /etc/passwd

# Multiple fields
cut -d: -f1,3 /etc/passwd

# Cut by character position
cut -c1-10 file.txt

# Second column from a CSV
cut -d, -f2 data.csv

sort — Sorting

Bash
# Basic sort
sort file.txt

# Reverse
sort -r file.txt

# Numeric sort
sort -n numbers.txt

# Sort by field (whitespace-delimited, 3rd field)
sort -k3 -n access.log

# Sort and unique
sort -u file.txt

# Human-readable size sort (1K, 2M, 3G)
du -sh * | sort -h

uniq — Deduplication

Bash
# Remove consecutive duplicates (use after sort)
sort file.txt | uniq

# Count occurrences
sort file.txt | uniq -c

# Print only duplicated lines
sort file.txt | uniq -d

# Print only unique lines
sort file.txt | uniq -u

wc — Counting

Bash
wc -l file.txt      # Line count
wc -w file.txt      # Word count
wc -c file.txt      # Byte count

# File count in a directory
ls /etc | wc -l

# Count of lines matching a pattern
grep -c "ERROR" app.log

Practical Pipeline Combinations

Bash
# Nginx logs — top 5 status codes
awk '{print $9}' /var/log/nginx/access.log \
  | sort | uniq -c | sort -rn | head -5

# Top 10 client IPs
cut -d' ' -f1 access.log | sort | uniq -c | sort -rn | head -10

# Unique error messages from error logs
grep "ERROR" app.log | cut -d']' -f2 | sort -u

# Users with bash as their shell in /etc/passwd
grep "/bash$" /etc/passwd | cut -d: -f1 | sort

# Top 5 processes by memory
ps aux | sort -k4 -rn | head -6 | awk '{print $4"%", $11}'

# File counts by extension in the current directory
find . -type f | grep -oE '\.[^.]+$' | sort | uniq -c | sort -rn

Deduplication — sort · uniq in Practice

uniq only removes consecutive duplicates, so always run sort first.

Bash
sort file.txt | uniq                 # Remove duplicate lines
sort file.txt | uniq -c              # Count of each line
sort file.txt | uniq -d              # Show only duplicated lines
grep "ERROR" app.log | sort | uniq -c | sort -rn   # Error frequency ranking
awk '!seen[$0]++' file.txt           # Deduplicate while preserving original order (no sort)
#grep#cut#sort#uniq#linux
Editorial note

AI tools may assist drafting. Check the evidence in each guide for whether commands were run and which environment was used. Confirm the current docs for your version before applying instructions, and email us about errors. email us

Topic-related official docs (auto-linked · not per-claim sources)Linux man pages ↗
How we write and review →Editorial policy →

Questions & answers

Ask a question about this guide. We'll review and reply.