Abc.web.id
Abc.web.id
Sed & Awk Command Cheatsheet

Sed & Awk Command Cheatsheet

Essential sed and awk commands for text processing

DevOps

Sed & Awk Command Cheatsheet

A comprehensive reference for sed and awk text processing.

Sed Basics

Substitution

sed 's/old/new/' file               # Replace first occurrence
sed 's/old/new/g' file              # Replace all occurrences
sed 's/old/new/2' file              # Replace 2nd occurrence
sed 's/old/new/gi' file             # Case-insensitive replace
sed '1,10s/old/new/g' file          # Replace in lines 1-10
sed '/pattern/s/old/new/g' file     # Replace in matching lines

Delete Lines

sed '5d' file                       # Delete line 5
sed '1,5d' file                     # Delete lines 1-5
sed '/pattern/d' file               # Delete matching lines
sed '/^$/d' file                    # Delete empty lines
sed '/^#/d' file                    # Delete comments

sed -n '5p' file                    # Print line 5
sed -n '1,5p' file                  # Print lines 1-5
sed -n '/pattern/p' file            # Print matching lines
sed -n '/start/,/end/p' file        # Print between patterns

Insert/Append

sed '5i\New line' file              # Insert before line 5
sed '5a\New line' file              # Append after line 5
sed '/pattern/i\New line' file      # Insert before match
sed '/pattern/a\New line' file      # Append after match

In-place Editing

sed -i 's/old/new/g' file           # Edit file in-place
sed -i.bak 's/old/new/g' file       # Create backup

Multiple Commands

sed -e 's/old/new/g' -e 's/foo/bar/g' file
sed 's/old/new/g; s/foo/bar/g' file

Awk Basics

awk '{print $1}' file               # Print first column
awk '{print $1, $3}' file           # Print columns 1 and 3
awk '{print $NF}' file              # Print last column
awk '{print $0}' file               # Print entire line

Field Separator

awk -F: '{print $1}' file           # Use : as separator
awk -F',' '{print $1}' file         # Use , as separator
awk 'BEGIN{FS=":"} {print $1}' file # Set separator in BEGIN

Pattern Matching

awk '/pattern/' file                # Print matching lines
awk '!/pattern/' file               # Print non-matching lines
awk '/start/,/end/' file            # Print between patterns
awk '$3 > 100' file                 # Print if column 3 > 100
awk '$1 == "value"' file            # Print if column 1 equals value

Built-in Variables

awk '{print NR, $0}' file           # NR = line number
awk '{print NF, $0}' file           # NF = number of fields
awk 'END {print NR}' file           # Print total lines
awk '{sum+=$1} END {print sum}' file # Sum column 1

Conditions

awk '$3 > 100 {print $1}' file      # If column 3 > 100
awk '$1 == "value" {print $2}' file # If column 1 equals
awk 'NR > 1' file                   # Skip first line
awk 'NR % 2 == 0' file              # Print even lines

BEGIN and END

awk 'BEGIN {print "Start"} {print} END {print "End"}' file
awk 'BEGIN {sum=0} {sum+=$1} END {print sum}' file

Sed Examples

Replace in Specific Lines

sed '10s/old/new/' file             # Line 10 only
sed '10,20s/old/new/g' file         # Lines 10-20
sed '10,$s/old/new/g' file          # Line 10 to end

Multiple Substitutions

sed 's/old/new/g; s/foo/bar/g' file
sed -e 's/old/new/g' -e 's/foo/bar/g' file

Remove Leading/Trailing Spaces

sed 's/^[ \t]*//' file              # Remove leading
sed 's/[ \t]*$//' file              # Remove trailing
sed 's/^[ \t]*//; s/[ \t]*$//' file # Both

Add Line Numbers

sed = file | sed 'N; s/\n/\t/'

Double Space File

sed G file

Remove Blank Lines

sed '/^$/d' file
sed '/^[[:space:]]*$/d' file        # Including whitespace

Awk Examples

Sum Column

awk '{sum+=$1} END {print sum}' file

Average

awk '{sum+=$1; count++} END {print sum/count}' file

awk 'NR==5' file                    # Line 5
awk 'NR>=5 && NR<=10' file          # Lines 5-10

Count Lines

awk 'END {print NR}' file

awk '!seen[$0]++' file

CSV Processing

awk -F',' '{print $1, $3}' file.csv
awk -F',' 'NR>1 {print $1}' file.csv  # Skip header

Format Output

awk '{printf "%-10s %s\n", $1, $2}' file
awk '{printf "%s,%s,%s\n", $1, $2, $3}' file

Combined Examples

Replace and Save

sed 's/old/new/g' input.txt > output.txt

Process Log Files

awk '/ERROR/ {print $1, $2, $NF}' log.txt
sed -n '/ERROR/p' log.txt

Extract Email Addresses

sed -n 's/.*\([a-zA-Z0-9._%+-]\+@[a-zA-Z0-9.-]\+\.[a-zA-Z]\{2,\}\).*/\1/p' file

Convert CSV to TSV

sed 's/,/\t/g' file.csv

Remove HTML Tags

sed 's/<[^>]*>//g' file.html

Quick Reference

Sed

Command Description
s/old/new/ Substitute
d Delete
p Print
i\text Insert
a\text Append
-n Suppress output
-i In-place edit

Awk

Variable Description
$0 Entire line
$1, $2, ... Fields
NR Line number
NF Number of fields
FS Field separator
OFS Output field separator

Best Practices

  1. Test before in-place editing
  2. Use -n with sed for selective printing
  3. Quote patterns to avoid shell expansion
  4. Use BEGIN/END in awk for initialization
  5. Combine with pipes for complex processing
  6. Use -F for awk field separator
  7. Create backups with sed -i.bak
  8. Learn regex for powerful patterns
  9. Use awk for columns, sed for lines
  10. Test on sample data first

Resources

Back to Cheatsheets