All Tools Bookmark

Share

http://online-toolz.com/tools/linux-command-examples.php

Linux Command Examples

Examples of frequently used Linux shell commands for quick reference

Note: all commands were tested on Ubuntu Linux

Compression & Extraction

tar -cvf output.tar FOLDER

Tar compress folder [tar]

tar -xvf output.tar

Tar DEcompress folder [tar]

tar -jcvf output.bz2 --exclude=data FOLDER

Compress folder using bzip2 while excluding "data" subfolder

  • "-c" means compress

zip output.zip FOLDER/*

Zip folder [zip]

Sorting

sort textfile.txt | uniq

Sort and Remove duplicate lines from text file [sort, uniq]

  • "uniq -u" will print unique lines only

Word counting

wc -w textfile.txt

Count number of words in textfile.txt [wc]

  • "wc -l" will print number of lines in file
  • "wc -c" will print number of characters in file

Search in files (Grep)

grep "the" textfile.txt

Find and print occurrences of "the" in textfile.txt [grep]

grep -P "[0-9]+" yearly.stats.txt

Find and print all lines containing numbers in textfile.txt using perl Regular Expression "[0-9]+" [grep]

  • adding "-o" will print only the numbers not the full lines

Search for files

find . -name "*hadoop*"

Find and print all files names containing the word "hadoop" in current directory and all sub directories [find]

  • "." means current directory
  • Omitting first "*" will make the search in the current directory only

find . -type f -name 'CERTIFICATE*.pdf' -execdir cp {} /PATH/printing/{} \;

Find all files containing the word "CERTIFICATE" and copy them into the following path "/PATH/printing/"

  • "{}" refers to any file found

File head and tail

head textfile.txt tail textfile.txt

Show first (head) and last (tail) 10 lines in textfile.txt [head]

Set environment variables

export PATH=$PATH:/NEW_PATH

Append a new folder to "PATH" environment variable [export]

Restart Linux service

sudo service network-manager restart

Restart network-manager service [service]

Add user to group

sudo usermod -g vboxusers karim

Add user "karim" to "vboxusers" group [usermod]

Make files executable (Runnable)

sudo chmod +x file.sh

Make file.sh executable [chmod]

Change file ownership

sudo chown karim:karimgroup file.sh

Change ownsership of file.sh to user "karim" and group "karimgroup" [chown]

Access remote Linux machine using SSH

ssh karim@machine_host.com

Login to machine_host.com linux machine using "karim" username [ssh]

Show total memory used

free -m

[free]

Join two files horizontally using [paste]

paste -d "|" FILE1.txt FILE2.txt > FILE3.txt

Write lines consisting of the sequentially corresponding lines from each FILE

  • "-d" means delimiter

Join two files vertically using [cat]

cat FILE1.txt FILE2.txt > FILE3.txt

Compare two sorted files and show the differences using [comm]

comm -3 FILE1.txt FILE2.txt

  • "-3" means suppress common lines

Convert text file to lowercase using [tr]

cat FILE.txt | tr '[:upper:]' '[:lower:]

Get execution time of a Linux command using [time]

time GET http://online-toolz.com > /dev/null

  • "> /dev/null" is used to suppress GET command output

List contents of jar file

jar tf FILE.jar

Split large files [split]

split -l 100 INPUT_FILE.txt OUTPUT_FILE_PREFIX.txt

Split INPUT_FILE to multiple files 100 lines each

Remove first or last line from file [tail]

head -n NUM_OF_LINES_MINUS_ONE INPUT.TXT > OUTPUT.TXT
tail -n NUM_OF_LINES_MINUS_ONE INPUT.TXT > OUTPUT.TXT

First command will remove last line, Second command will remove first line

  • "NUM_OF_LINES_MINUS_ONE" should be the number of lines in the file minus one - use wc -l to get it

Show disk usage of first level sub-directories [du]

du -h -d 1 .

  • "-h" means human redable (ex: GB instead of long number of bytes)
  • "-d" is the depth of folders to show

TASKS: Sending HTTP Form post request using [curl]

curl -F "account_login=6pd9vkgk" -H "Content-Type: application/x-www-form-urlencoded" https://URL

  • "application/x-www-form-urlencoded" is the header for HTTP form
  • "account_login=6pd9vkgk" is the form payload

TASKS: Sniff your web traffic using tcpdump [tcpdump]

sudo tcpdump -i wlan0 -A port 80 -s 0

  • "wlan0" means your wifi card, change it to eth0 or other for your LAN
  • "80" is the HTTP port for web traffic

TASKS: PORT forwarding from port 8889 to port 8888 [nc]

mkfifo httpPIPE
nc -q -1 -l 8889 0<httpPIPE | nc -q -1 localhost 8888 1>httpPIPE

  • put httpPIPE in standard input "0" and put standard output "1" in httpPIPE
  • "-q" will prevent quiting after EOF, will wait forever

TASKS: Rename all files in current dir, add ".postfix" at the end [rename]

rename 's/(.*)/$1.postfix/' *

  • "$1" refers to the original filename

TASKS: Sending email from command line [sendmail]

echo "YOUR_MESSAGE" | mail -s "SUBJECT" TARGET_EMAIL@gmail.com

  • Make sure you have sendmail installed "sudo apt-get install sendmail"

TASKS: Download full website [wget]

wget --recursive --wait=2 -U "Mozilla" -p -H http://twitter.com

  • --wait will wait for 2 seconds between page downloads
  • -U for user agent
  • -p and -H are needed to download all pages linked to index page even if in different subdomain

TASKS: Run a specific file every 30 minutes [crontab]

crontab -e
0,30 * * * * /home/FILE_PATH
CTRL O
CTRL X

  • run "crontab -l " to make sure the cronjob is created correctly
  • 0,30 means minute 0 and minute 30

TASKS: Disable word wrapping in terminal [less]

cat index.html | less -S

  • -S did the job

TASKS: Count lines of code in a directory [less]

rm -rf temptotal.txt; find . -regex ".*.\(php\|html\|css\|js\)" -not -path "./libs/*" | xargs -I FILE sh -c "echo FILE; cat FILE | tr -d '\t' | tr -s '\n\n' '\n' | wc -l " | xargs -I COUNT sh -c "echo COUNT;echo COUNT >> temptotal.txt" ; echo "Total:" ; cat temptotal.txt | awk '{ sum+=$1} END {print sum}'; rm -rf temptotal.txt;

Count lines of code in files with php,html,css,js extentions excluding "libs" folder"

  • The code removes double end lines "\n\n" before counting

TASKS: Delete files that have more than 2MB of data [rm,xargs]

sudo find /YOUR/FOLDER_PATH -size +2M | xargs rm -rf

First find files with 2MB+ size and then pass each one to "rm" command to be deleted

  • xargs is used to pass each file to rm as a parameter
  • WARNING: make sure you are in the right folder since "-rf" will delete files without confirmation

TASKS: Show memory usage by process [ps]

ps -e -orss=,args=,command,user,pid | sort -b -k1n | awk '{ hr=($1/1024)/1024 ; printf("%13.2f GB ",hr) } { for ( x=2 ; x<=NF ; x++ ) { printf("%s ",$x) } print "" }'

TASKS: Print second line from CSV file [ps]

cat file.csv | awk -F "," '{ print $2 }'

  • For this example columns are assumed to be separated by ","

TASKS: Mail forwarding from root local mail to external email address using sendmail

echo "root: you@yourmail.com" >> /etc/aliases && newaliases

  • "newaliases" is used to rebuild alias DB

TASKS: Convert PDF file pages to JPG images [convert]

convert -quality 100 -density 122x122 -antialias FILE.pdf FILE_PREFIX-%d.jpg

  • "%d" will be replaced by page numbers

TASKS: Create video(Timelapse) from multiple images using [ffmpeg]

ffmpeg -r 15 -start_number 756 -i 'DSC00%3d.JPG' VIDEO.mp4

  • "-r 15" is the frame rate per second
  • "%3d" will be replaced by image number starting from "756" as indictaed by -start_number

TASKS: Count and sort duplicates in file [sort]

cat FILE.txt | sort | uniq -c | sort -nr

  • "-nr" means reverse numeric sorting

TASKS: Record screen session (Screencasting) Audio/Video [ffmpeg]

ffmpeg -video_size 1024x768 -framerate 25 -f x11grab -i :0.0 -f pulse -ac 2 -i default ~/output.mkv

TASKS: convert comma separated file into rows using [sed]

cat INPUT.txt | sed 's/,/\r\n/g' > OUTPUT.txt

sed will replace all "," with "\r\n" (endlines)

  • sed pattern is 's/SEARCH_FOR/REPLACE_WITH/g'
  • sed can also be used to clean text files

TASKS: Scrape web pages using [wget]

wget -q http://online-toolz.com -O - | tr -d '\r\n|\n' | grep -o "<title>[^<]*</title>" | sed -r "s|</?title>||g"

Get the title of online-toolz.com website

  • "-O -" is used to redirect output to stdout instead of file
  • "tr -d" will delete endlines to make searching and replacing easier
  • grep finds the "title" tag fragment and "sed" will remove the "title" tags keeping tag content


Keywords: Linux, Shell, Script, Commands, Examples, Bash