Command Line Fun

April 26, 2021 ยท View on GitHub

Table of Content

Command Line Fun

Introductory Look to Popular Linux Command Line Program

Environment Variables

  • When opening a terminal window, new bash process which has it's own Bash Environment Variables are initialised.
  • Environment Variables: Form of Global storage inherited by any application which will run during terminal session

Comman Environment Variables:

  1. PATH : kali@kali:~$ echo $PATH
  2. USER : kali@kali:~$ echo $USER
  3. HOME : kali@kali:~$ echo $HOME
  4. PWD : kali@kali:~$ echo $PWD

Define Environment Variables using export cmd

  • Let's say you don't want to type IP address repeatedly then you can export the $ip variable & can use it across terminal sessions
kali@kali:~$ export $ip=192.168.43.70
kali@kali:~$ ping -c 2 $ip
  • Variables exported without export cmd are only accessible in that current bash session
kali@kali:~$ lol="Rasode Mein Kon Tha?"
kali@kali:~$ echo $lol    # We will get value of $lol
kali@kali:~$ bash 
kali@kali:~$ echo $lol    # No Value
kali@kali:~$ exit   
kali@kali:~$ echo $lol    # Again can access value of $lol

  • NOTE: Variables defined using export cmd can be accessed using different terminal window

  • Viewing Default Kali Linux Environment Variables: env

Tab Completion

  • Used in auto text completion.

Bash History Tricks

  • We can see previous executed cmds using history cmd

  • For executing last executed terminal cmd, use !! double exclamation mark.

  • Bash terminal history is saved in .bash_history file in the user home directory: tail -n 3 .bash_history

  • Two Common Env Variables:

  1. $HISTSIZE : Controls no. Of cmds stored in memory for current terminal session.
  2. $HISTFILESIZE : Controls How many cmds are kept in history file.
  • These 2 env variables can be edited according to are needs.

Useful Keyboard Keys

  1. Up ArrowKey: Use to select/scroll previous cmds in Upwards direction.
  2. Down ArrowKey: Use to select/scroll cmds in Downwards direction.
  3. CTRL + R: Reverse I search facility, can search previously executed cmds using Cmd Search Feature.

Piping and Redirection

  • Every program executed from terminal has 3 streams connected to it that serves the communication channel to external env.
  1. Standard Input (STDIN): Data fed into the program.
  2. Standard Output (STDOUT): Output from the program. (defaults to terminal)
  3. Standard Error (STDERR): Error message. (defaults to terminal)

Redirecting to a new file

  • Redirecting output to a non existing file will leads to file creation: kali@kali:~$ echo "Corona Go! Go Corona" > EminemRap.txt Typical file reading: kali@kali:~$ cat EminemRap.txt

  • If file already exist, then that file will be overwritted: kali@kali:~$ echo "Selfie meinai leli aaj" > EminemRap.txt

  • Note : Be very careful while file redirection, there is no undo function.

Redirecting to a existing file

  • Use >> sign to append data to a existing file: kali@kali:~$ echo "Chura ke dil mera goliya chali" >> EminemRap.txt

Redirecting from a file

  • Use < sign to redirect file data to any utility: kali@kali:~$ wc -m < EminemRap.txt
  • Above cmd will count total words a file.

Redirecting STDERR

  • We can redirect standard error of particular command using 2>: kali@kali:~$ ls . /test 2>error.txt
  • Example content of error.txt file: ls: cannot excess './test': No such file or directory

Piping

  • Redirecting output of one cmd as a input for another cmd: kali@kali:~$ cat error.txt | wc -m
  • Above cmd will perforn word count of error.txt file.

kali@kali:~$ cat error.txt | wc -m > count.txt kali@kali:~$ cat count.txt

Text Searching and Manipulation

grep

  • Searches text file for given regex & outputs any line containing a match to stdout (stdout == usually terminal)
  • Example: kali@kali:~$ ls -la /usr/bin | grep zip
  • Use man cmd to learn more about grep

sed

  • Powerful stream editor + very complex. It performs text editing in stream of text.
  • Example: kali@kali:~$ echo "Go Katrina! Go" | sed 's/Katrina/Corona/'
  • Example Output: Go Corona! Go

cut

  • Used to cut a stdout of one cmd to pieces on the basic of given delimiter, then it also be used to select a particular part of string.
  • Commonly used flags: -d for Delimiter & -f for Field Selection.
  • Example: kali@kali:~$ echo rahul, saksham, rohit | cut -d "," -f 2
  • Example Output: saksham
  • Extracting list of users from etc/passed file: kali@kali:~$ cut -d ":" -f 1 /etc/passwd

awk

  • Programming language designed for text processing.
  • Typically used for data extraction and reporting tools.
  • Only going to scratch the surface.
  • Splitting stdout of one cmd using delimeter, Extracting 1st & 3rd part & then finally printing it: kali@kali:~$ echo "hello::there::friends" | awk -F "::" '{print \$1, \$3}'
  • Example Output: hello, friends

Editing files from the Command Line

nano

  • Opening file using nano: nano anyfilename.txt
  • Nano commands menu is located at the bottom.
  • Important Memorable cmds: - CTRL + O : Write changes to a file. - CTRL + K : To cut current line. - CTRL + U : To uncut a line & paste it at cursor location. - CTRL + W: To search within a line. - CTRL + X : Use to exit.

vi

  • Opening file using vi: vi anyfilename.txt
  • Press i key to Escape Command Mode & start typing in text file.
  • Press esc key in order to return back to Command Mode.
  • Press dd to remove the current line.
  • Press yy to copy the current line.
  • Press p to paste the clipboard context.
  • Press x to delete the current character under the cursor.
  • Press :w to write current file to disk & remain open in vi
  • Press :q! to quit without writing a file.
  • Press :wq! to quickly save the file & quit vi

Comparing Files

comm

  • Compares two text file, displaying the line that are unique to each one as well those lines that are common.
  • Example File-A & File-B:
kali@kali:~$ cat File-A.txt
192.168.1.1
192.168.1.2
192.168.1.3
192.168.1.4
192.168.1.5
kali@kali:~$ cat File-B.txt
192.168.1.1
192.168.1.3
192.168.1.4
192.168.1.5
192.168.1.6
  • File Comparison example using comm:
kali@kali:~$ comm File-A.txt File-B.txt
                               192.168.1.1
192.168.1.2
                               192.168.1.3
                               192.168.1.4
                               192.168.1.5
              192.168.1.6
  • 192.168.1.2 is unique to 1st file, thus it is displayed in 1st column.
  • 192.168.1.6 is unique to 2nd file, thus it is displayed in 2nd column.
  • Rest all common IPs/Lines are displayed in the 3rd column.
  • -1 , -2 and -3 flags could be given to comm in order to suppress 1st, 2nd or 3rd column, according to our need. For example:
kali@kali:~$ comm -12 File-A.txt File-B.txt
192.168.1.1
192.168.1.3
192.168.1.4
192.168.1.5

diff

  • Compares difference b/w files similar to comm, but it is more complex & support many output formats.
  • Two famous Formats:
  1. Context Format: kali@kali:~$ diff -c File-A.txt File-B.txt
  2. Unified Format: kali@kali:~$ diff -u File-A.txt File-B.txt

vimdiff

  • It open vim (extended version of vi), with multiple files, one in each window.
  • Difference b/w files is highlighted, making it easier to inspect them.
  • Useful shortcuts:
    • Press CTRL + W + Arrow Key for switching b/w windows
    • Press ] + C will jump the cursor to next change in the diff.
    • [ + C will jump it to the previous change.
    • Press D + O will get a change in other window and put it in current window.
    • Press D + P will put the change in other window from the current one.
    • Press :q! for quitting vimdiff (Same shortcuts as vi)

Managing Processes

  • Linux kernel manages multitasking through the use of process.

  • Kernel maintains info about each processes to help keep things organised.

  • Each process is assigned a no. called process id

  • Linux shell introduces the concept of jobs to ease our workflow during terminal session. For ex: kali@kali:~$ cat errors.txt | wc -m This whole cmd will be considered as one job.

  • jobs helps us to suspend or resume the execution of particular cmd.

Backgrounding Processes (bg)

  • Helps us to make the execution of cmd in background, leaving the shell free for further use.
  • & sign at the end of a cmd, will put the cmd in bg.

Ex: kali@kali:~$ ping -c 400 localhost > ping_result.txt &

  • Stopping the execution of cmd using CTRL + Z and then typing bg cmd will run the last cmd in background. Ex: kali@kali:~$ bg

Jobs Controls (jobs and fg)

  • jobs utility list the jobs that are running in current terminal session.
  • Running fg %JOBNumber returns a job in foreground. Example: fg %1
  • If only one job running in the bg, then typing fg without any additional flag will return that single job in the foreground.

Process Control (ps and kill)

  • ps : Process status, list all the running process in a Linux/Unix system, Swizz army knife for process management.

  • Listing all process with full format listing: kali@kali:~$ ps -ef where -e & -f stands for select all processes and full format listing respectively.

  • Finding process Id using command name: kali@kali:~$ ps -fC leafpad

  • Killing process using process id: kali@kali:~$ kill ProcessID

  • Verifying process termination using ps: kali@kali:~$ ps -fC leafpad

File and Command Monitoring

tail

  • Monitoring apache logs using tail: kali@kali:~$ sudo tail -f /var/log/apache2/access.log
  • Extracting last 2 lines from file: kali@kali:~$ sudo tail -n2 /etc/lsb-release where 2 is the desired no. of last lines.

watch

  • Used to run designated cmd at regular intervals, by default it runs for every 2 seconds which could be changed using -nX flag where X is the interval in seconds.
  • Example (This cmd will list logged in users output from the w cmd once very 5 seconds) : kali@kali:~$ watch -n 5 w

Downloading Files

wget

  • Utility to download file from the internet.
  • Example (Downloading file & saving it with different name) : kali@kali:~$ wget -O lol.txt https://example.com/text file.txt
  • Refer to its docs for more info: kali@kali:~$ wget --help | less

curl

  • Use to transfer data to and from a server using a host of protocol.
  • Can be used to Download and Upload files & build complex request.
  • Example (Download & save file with different name) : kali@kali:~$ curl -o lol.txt https://example.com/textfile.txt
  • Very versatile tool, lots of use case scenarios & ample docs are available online.

axel

  • It is Download accelerator that transfers a file from an FTP or HTTP server through multiple connections.

  • It has vast variety of options. Common option: kali@kali:~$ axel -a -n 20 -o lol.pdf https://example.com/lol.pdf

  • -a : For more concise progress indicator.

  • -n : For no. Of multiple connections to use.

  • -o : For different output name.

  • Extremely useful for downloading large file.

Customizing the Bash Environment

Bash History Customization

  • Whether or not to remove duplicate cmds: kali@kali:~$ export HISTCONTROL=ignoredups
  • Filtering out basic cmds : kali@kali:~$ export HISTIGNORE="&:ls:[bf]g:exit:history"
  • For showing TimeFormat in history output cmd: kali@kali:~$ export HISTTIMEFORMAT="%F %T" Other time format can be found in strftime man page: kali@kali:~$ man strfman

Alias

  • Defining large cmd to shorter name
  • Example: kali@kali:~$ alias lsa=ls -la
  • We can see defined alias using kali@kali:~$ alias cmd without giving any argument/flag.

Persistent Bash Customization

  • Behavior of interactive shell in bash is determined by system's /etc/bash.bashrc file.
  • Editing this bash.bashrc bash file, will leads to persistent bash customization.