Filters
Filters are programs that take plain text (stored in a file or produced by another program) as standard input, transform it into a meaningful format, and then return it as standard output. Linux has a multitude of filters.
Cat
The primary purpose of the cat command is to display the contents of one or more text files on the terminal. This command allows quick viewing of file contents.
root@hackerbox:~$ cat /etc/ssh/sshd_config
# Port 22
# AddressFamily any
# ListenAddress 0.0.0.0
# ListenAddress ::
PermitRootLogin no
PasswordAuthentication yes
PermitEmptyPasswords no
ChallengeResponseAuthentication no
UsePAM yesIn the example above, the configuration file for the SSH service located at /etc/ssh/sshd_config is displayed on the terminal using the cat command.
Head
The head command is used to display the first few lines of a specified file. By default, the head command shows the first 10 lines, but this number can be changed using the -n parameter.
This command is handy when you want to quickly review the beginning of a file without displaying its entire content.
In the example above, the first 3 lines of the log file located at /var/log/apache2/access.log for the Apache2 Web Server service are displayed on the terminal.
Tail
The tail command is used to display the last few lines of a specified file. By default, the tail command shows the last 10 lines, but this number can be changed using the -n parameter.
This command is extremely useful for monitoring the most recently added content to continuously growing files, such as log files.
In the example above, the last three lines of the auth.log file are displayed. The auth.log file logs events related to user authentication on a Linux system, including user logins and logouts, sudo command usage, SSH sessions, and other authentication-related events.
Sort
The sort command sorts the contents of a given file alphabetically.
In the example above, the contents of "names.txt" are sorted alphabetically.
Uniq
The uniq command filters out consecutive duplicate lines from a file and shows the unique lines.
It is often used in conjunction with the sort command because, when used alone, it only detects consecutive duplicate lines. To address duplicates throughout the file, it's recommended to first sort the data.
In the example above, although "Alice" appears twice, the uniq command does not remove non-consecutive duplicates.
Therefore, to eliminate non-consecutive duplicates, first sort the file and then apply the uniq command.
Grep
The grep command searches files for specific text strings, filters lines, and displays matching results.grep is a powerful tool commonly used to search log files, configuration files, or any text file.
This command will display all records in the Apache 2 web server access logs located at /var/log/apache2/access.log that contain the IP address 192.168.1.1.
Wc
The wc (word count) command quickly determines how large a file is or how much data it contains.
In the example above, the wc command returns the number of lines, words, and characters respectively in the/etc/passwd file, which contains the list of registered users on a Linux system.
46
Number of lines
67
Number of words
2544
Number of characters
/etc/passwd
File path
The wc command has various parameters:
-l: Displays only the number of lines.
-w: Displays only the number of words.
-c: Displays only the number of bytes.
-m: Displays only the number of characters (useful for multi-byte character sets).
For example, to see the total number of log entries recorded by the Apache 2 web server to date, you can use the -l parameter with the following command:
The output indicates that there are a total of 54,230 log entries.
Sed
The sed(stream editor) command is a tool capable of performing various text edits such as processing, modifying, adding, deleting, or replacing texts between files.
The sed command is commonly used to filter and transform texts.
In the example above, the name Alice in names.txt is replaced with George using the sed command. However, note that thesedcommand only prints the change to the screen and does not save it to the file.
Awk
The awk command is designed for text and data processing tasks, and it is especially effective when working with column-based data. It reads files line by line, splits each line into fields (columns), and processes them based on specified conditions.awk offers numerous functions and control structures for complex text processing.
In this example, the file names.txt contains three name-surname pairs: John Doe, Emily Clark, and Alex Turner. The command awk '{print $1}' names.txt processes the content of this file using the awk program.awk reads the text files line by line, splitting each line into fields separated by spaces or tabs. In this case, the expression{print $1} instructs awk to print only the first field (the first name) of each line.
Using these useful Linux filters, you can process, search, and transform text files quickly and efficiently according to your specific need.
Last updated