Day 7 OSCP | Text Searching and Manipulation

Reading Time: ( Word Count: )

December 26, 2020
Nextdoorsec-course

Intro

Hi fellow hackers, although we haven’t done much than just covering commands, it’s a necessary step before jumping into hacking. I used to try “hacking” with Youtube tutorials initially, but when you get to the point that something doesn’t work and can’t resolve it immediately, it gets frustrating. So to become real hackers and no script kiddies, we need to know how to troubleshoot and use certain tools to make our lives easier in the long run. And rely less on forums, tips and tricks from others.

Today we’ll cover four commands: grep, sed, cut and awk. These are commands we’ll use pretty often when dealing with text files and make them look a certain way before passing them into other tools or commands. Or for making large logs human-readable and searching in them.

 

Grep

Let us start with the most popular one, grep. Grep stands for Global Regular Expression Print.

Grep searches for patterns in text files, more specifically for regular expressions and outputs or prints any line containing a match to the standard output.

  ┌──(kali㉿kali)-[~]      └─$ grep kali /etc/passwd kali:x:1000:1000:Kali,,,:/home/kali:/usr/bin/zsh
  ┌──(kali㉿kali)-[~]      └─$ grep Kali /etc/passwd kali:x:1000:1000:Kali,,,:/home/kali:/usr/bin/zsh
  ┌──(kali㉿kali)-[~]      └─$ grep -i kali /etc/passwd kali:x:1000:1000:Kali,,,:/home/kali:/usr/bin/zsh

 

Above we searched for the word “kali” in the “passwd” file. Grep by default is case sensitive, so it only found two results and printed the line containing them. Using the “-i” option, it found three results, containing “Kali” with a capital letter.

Another useful example will be for searching in all files in the current directory. Herefore we’ll use an asterisk (*) after the search pattern.

We can also search recursively, i.e. read all files under each directory for a string “kali” with the “-r” option.

And the last example is for printing the exact search string, using the “-x” option.

  ┌──(kali㉿kali)-[~]      └─$ grep -i kali * example1.txt:kali is my favorite operating system example2.txt:Most widely used system for pentesting is Kali example3.txt:I love kali
  ┌──(kali㉿kali)-[~]      └─$ grep -r kali etc/ etc/theHarvester/names_small.txt:kalinovsky-k etc/theHarvester/names_small.txt:kalithies etc/theHarvester/names_small.txt:sayasukalirik etc/theHarvester/names_small.txt:chakali etc/theHarvester/names_small.txt:kalintv6 ...
  ┌──(kali㉿kali)-[~]      └─$ grep -r -x kali etc/ etc/theHarvester/dns-big.txt:kali etc/hostname:kali

 

 

Sed

Sed stands for stream editor, and it can perform functions like searching, find and replace, insertion or deletion in text files. Most commonly used to find and replace.

By default, the sed command replaces the first occurrence of the pattern in each line, and it won’t replace the second, third…occurrence in the line, as seen below.

By adding the “g” option standing for global, we replace all occurrences. In case you only want to replace the first, second, or third occurrence of the pattern, use 1, 2 or 3 instead.

Sed is a powerful command, so there’s a lot more we can do as it is the case with grep. We’ll explore them more later on in action.

  ┌──(kali㉿kali)-[~]      └─$ cat kali.txt ubuntu is a very powerful OS for pentesters, ubuntu is life, ubuntu is all.
  ┌──(kali㉿kali)-[~]      └─$ cat kali.txt | sed 's/ubuntu/kali/' kali is a very powerful OS for pentesters, ubuntu is life, ubuntu is all.
  ┌──(kali㉿kali)-[~]      └─$ cat kali.txt | sed 's/ubuntu/kali/g' kali is a very powerful OS for pentesters, kali is life, kali is all.

 

 

Cut

The cut command is used to cut a section of text from a line and output it to the standard output. Cut can be used to cut parts of a line by byte position, character and field. 

The most commonly-used switches include “-f” for the field number we are cutting and “-d” for the field delimiter.

  ┌──(kali㉿kali)-[~]      └─$ cat kali2.txt I've heard hackers have hobbies, such as lock picking, HackNet, OverTheWire, etc.
  ┌──(kali㉿kali)-[~]      └─$ cat kali2.txt | cut -d "," -f1 I've heard hackers have hobbies
  ┌──(kali㉿kali)-[~]      └─$ cat kali2.txt | cut -d "," -f3 HackNet

 

The cat command’s output is passed to cut, which prints the specified field using “, as the delimiter.

Awk

Awk” is a scripting language designed for advanced text processing on the command line, mostly used as a reporting and analysis tool. We can say that “awk” is the advanced version of the “cut” command.

By default, the field delimiter is a space character. Unlike with “cut”, we can define more than a single character as a field separator. Let’s do a few examples.

  ┌──(kali㉿kali)-[~]      └─$ date | awk '{print $3,$2,$4}' Dec 26 2020
  ┌──(kali㉿kali)-[~]      └─$ echo "How::are::you?" | -F "::" '{print $1,$2,$3}' How are you?
  ┌──(kali㉿kali)-[~]      └─$ echo "How=:=:are=:=:you?" | -F "=:=:" '{print $1,$2,$3}' How are you?
Aydan Arabadzha

Aydan Arabadzha

Author

Aydan, a cybersecurity ace and AI visionary, thrives on the frontlines of offensive security. His passion birthed NextdoorSec, a groundbreaking cybersecurity firm. A relentless pioneer, Aydan is persistently pushing boundaries, shaping the future of the digital world one byte at a time.

Other interesting articles

Automated vs Manual Penetration Testing

Automated vs Manual Penetration Testing

Pentesting is largely divided into two methodologies: Automated vs Manual Penetration Testing. Both have ...
8 Steps in Penetration Testing You Should Know

8 Steps in Penetration Testing You Should Know

Mastering the art of penetration testing has become a critical ability for security experts to combat cyber ...
Spear Phishing vs Whaling: What is the Difference

Spear Phishing vs Whaling: What is the Difference

Spear phishing is a particularly devious type of phishing assault in which the individual targeted plays a ...
How Often Should Penetration Testing Be Done

How Often Should Penetration Testing Be Done

Penetration testing is a crucial technique that involves simulating a cyberattack on networks, computer systems, ...
0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *