abbazsIntroduction
This tutorial covers five essential Linux text processing tools: grep, sed, awk, cut, and tr.
1. grep (Global Regular Expression Print)
grep is used for searching text using patterns.
Example 1: Basic text search
$ echo -e "This is an error\nThis is a warning\nThis is info" | grep "error"
This is an error
Example 2: Case-insensitive search
$ echo -e "Warning: System overload\nwarning: Low memory" | grep -i "warning"
Warning: System overload
warning: Low memory
Example 3: Display line numbers
$ echo -e "Line 1\nLine 2\nCritical error\nLine 4" | grep -n "Critical"
3:Critical error
Example 4: Recursive search in directories
Assume we have a directory structure:
project/
├── file1.txt (contains "TODO: Fix this")
└── subdir/
└── file2.txt (contains "TODO: Implement feature")
$ grep -r "TODO" project/
project/file1.txt:TODO: Fix this
project/subdir/file2.txt:TODO: Implement feature
Example 5: Invert match
$ echo -e "Success: Task 1\nError: Task 2\nSuccess: Task 3" | grep -v "Success"
Error: Task 2
Example 6: Use regular expressions
$ echo -e "Phone: 123-456-7890\nInvalid: 123-45-6789" | grep -E "[0-9]{3}-[0-9]{3}-[0-9]{4}"
Phone: 123-456-7890
Example 7: Print only matched parts (-o option)
$ echo "The IP address is 192.168.1.1 and 10.0.0.1" | grep -oE "\b([0-9]{1,3}\.){3}[0-9]{1,3}\b"
192.168.1.1
10.0.0.1
2. sed (Stream Editor)
sed is used for text transformation.
Example 1: Basic substitution
$ echo "The color is red" | sed 's/red/blue/'
The color is blue
Example 2: Global substitution
$ echo "one two one two" | sed 's/one/three/g'
three two three two
Example 3: In-place editing
Assume we have a file named "colors.txt" with content "The color is red"
$ sed -i 's/red/green/' colors.txt
$ cat colors.txt
The color is green
Example 4: Delete lines
$ echo -e "Keep this\nDelete me\nKeep this too" | sed '/Delete/d'
Keep this
Keep this too
Example 5: Print specific lines
$ seq 10 | sed -n '3,7p'
3
4
5
6
7
Example 6: Multiple operations
$ echo -e "foo bar\nbaz foo" | sed -e 's/foo/qux/g' -e '/baz/d'
qux bar
3. awk (Aho, Weinberger, and Kernighan)
awk is a powerful text-processing tool for working with structured data.
Example 1: Print specific fields
$ echo -e "John Doe 25\nJane Smith 30" | awk '{print $1, $3}'
John 25
Jane 30
Example 2: Use custom field separator
$ echo "name:John:age:30" | awk -F':' '{print $2}'
John
Example 3: Sum values in a column
$ echo -e "10\n20\n30" | awk '{sum += $1} END {print sum}'
60
Example 4: Conditional printing
$ echo -e "John 25\nJane 35\nMike 40" | awk '$2 > 30 {print $0}'
Jane 35
Mike 40
Example 5: Calculate average
$ echo -e "10\n20\n30\n40\n50" | awk '{sum += $1; count++} END {print sum/count}'
30
Example 6: Use regular expressions
$ echo -e "apple\nbanana\ncherry" | awk '/^b/ {print $0}'
banana
4. cut
cut is used for extracting sections from each line of files.
Example 1: Extract specific characters
$ echo "Hello, World!" | cut -c 1-5
Hello
Example 2: Extract specific fields with delimiter
$ echo "John,Doe,30,New York" | cut -d',' -f 2,4
Doe,New York
Example 3: Extract range of fields
$ echo "field1:field2:field3:field4:field5" | cut -d':' -f 1-3
field1:field2:field3
Example 4: Extract to end of line
$ echo "Name: John Doe, Age: 30, City: New York" | cut -d':' -f 2-
John Doe, Age: 30, City: New York
Example 5: Exclude specific fields
$ echo "1,2,3,4,5" | cut -d',' --complement -f 2,4
1,3,5
5. tr (Translate)
tr is used for translating or deleting characters.
Example 1: Convert lowercase to uppercase
$ echo "hello world" | tr 'a-z' 'A-Z'
HELLO WORLD
Example 2: Delete specific characters
$ echo "hello 123 world" | tr -d '0-9'
hello world
Example 3: Squeeze repeating characters
$ echo "hello world" | tr -s ' '
hello world
Example 4: Translate characters
$ echo "hello world" | tr 'hw' 'HW'
Hello World
Example 5: Delete all except specified characters
$ echo "hello 123 world" | tr -cd '0-9\n'
123
Conclusion
These five tools - grep, sed, awk, cut, and tr - are powerful utilities for text processing in Linux. The examples provided demonstrate their basic usage and some advanced features. Practice with these examples and explore their man pages for more advanced usage using man command.
