Intro

Hi fellow hackers, although we haven’t done much than just covering commands, it’s a necessary step before jumping into hacking. I used to try “hacking” with Youtube tutorials initially, but when you get to the point that something doesn’t work and can’t resolve it immediately, it gets frustrating. So to become real hackers and no script kiddies, we need to know how to troubleshoot and use certain tools to make our lives easier in the long run. And rely less on forums, tips and tricks from others.

Today we’ll cover four commands: grep, sed, cut and awk. These are commands we’ll use pretty often when dealing with text files and make them look a certain way before passing them into other tools or commands. Or for making large logs human-readable and searching in them.

Grep

Let us start with the most popular one, grep. Grep stands for Global Regular Expression Print.

Grep searches for patterns in text files, more specifically for regular expressions and outputs or prints any line containing a match to the standard output.

Above we searched for the word “kali” in the “passwd” file. Grep by default is case sensitive, so it only found two results and printed the line containing them. Using the “-i” option, it found three results, containing “Kali” with a capital letter.

Another useful example will be for searching in all files in the current directory. Herefore we’ll use an asterisk (*) after the search pattern.

We can also search recursively, i.e. read all files under each directory for a string “kali” with the “-r” option.

And the last example is for printing the exact search string, using the “-x” option.

Sed

Sed stands for stream editor, and it can perform functions like searching, find and replace, insertion or deletion in text files. Most commonly used to find and replace.

By default, the sed command replaces the first occurrence of the pattern in each line, and it won’t replace the second, third…occurrence in the line, as seen below.

By adding the “g” option standing for global, we replace all occurrences. In case you only want to replace the first, second, or third occurrence of the pattern, use 1, 2 or 3 instead.

Sed is a powerful command, so there’s a lot more we can do as it is the case with grep. We’ll explore them more later on in action.

Cut

The cut command is used to cut a section of text from a line and output it to the standard output. Cut can be used to cut parts of a line by byte position, character and field. 

The most commonly-used switches include “-f” for the field number we are cutting and “-d” for the field delimiter.

The cat command’s output is passed to cut, which prints the specified field using “,“ as the delimiter.

Awk

“Awk” is a scripting language designed for advanced text processing on the command line, mostly used as a reporting and analysis tool. We can say that “awk” is the advanced version of the “cut” command.

By default, the field delimiter is a space character. Unlike with “cut”, we can define more than a single character as a field separator. Let’s do a few examples.