From Indie IT Wiki

In terminal examples anything shown in bold after a command is a comment and is not to be run, e.g:

sudo apt-get install fish Then scratch your ear

So in the above Then scratch your ear is not to be run in the terminal


BASH Convert File Formats


BASH curl

BASH Directories

BASH Disable

BASH Disk Tools

BASH File Manipulation



BASH Hardware Discovery

BASH History

BASH IP Tables


BASH Manipulate Files With Special Characters and Spaces

BASH Memory

BASH Modification


BASH Networking


BASH Shorewall

BASH Terminal Process

BASH Time and Date

BASH Website/Page Related

BASH Useful Tools

BASH Users

Show Top CPU Usage Processes

ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head


watch "ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head"

Show TCP Connections


lsof -i tcp:443

Test Server and Port using Telnet Then Quit

( echo "^]"; echo "quit" ) | telnet 25

Thanks -

List Running Processes and Their Ports

sudo lsof -i -P -n

Rename Files Add Suffix Recursively

This will rename all matroska files without an filename extension suffix to .mkv ...

find /path -type f -exec rename -v 's/$/\.mkv/i' {} \;

Copy files that need root access with SCP

From server to local machine:

ssh user@server "sudo cat /etc/dir/file" > /home/user/file

From local machine to server:

cat /home/user/file | ssh user@server "sudo tee -a /etc/dir/file"

Using netcat to test ports


Test the TCP port 443 (https) on the web site ...

nc -v -z 443
Connection to 443 port [tcp/https] succeeded!

-v verbose, -z scan

Test the UDP port 3478 (stun) on the server at ...

nc -v -z -u 3478
Connection to 3478 port [udp/*] succeeded!

-v verbose, -z scan, -u udp


Computer A

nc -l 4444

Computer B

nc 4444

Computer A


Computer B

See output

Find and remove blank lines in a file

This is a file called test.txt with a blank line to start and others in it ...




To see which lines are blank ...

perl -lne 'print $. if(/^$/)' test.txt

To remove the blank lines and save the result to a new file ...

grep . test.txt >no_blank_lines_test.txt

How to list all files where the first line is a blank line

find . -name '*.java' -exec sed -n '/^$/F;q' {} \;

How to remove trailing whitespaces in a file

sed -i 's/[[:space:]]*$//' inputfile.txt

How to remove unicode characters in a file

iconv -c -f utf-8 -t ascii
cat file.csv | iconv -c -f utf-8 -t ascii
cat file.csv | sed 's/"//g;s/ #$//;s/�//g;s/ ,/,/g;s/[[:space:]]*$//' | iconv -c -f utf-8 -t ascii | uniq > file__.csv

How to display a Specific Line in a Text File

sed -n '13p' file.txt

How to Run a Local Shell Script on a Remote SSH Server

ssh user@remotehost 'bash -s' <

Create a Folder for Each Movie

cd /path/to/your/movies/files/
find . -maxdepth 1 -type f -iname "*.mkv" -exec sh -c 'mkdir "${1%.*}" ; mv "${1%}" "${1%.*}" ' _ {} \;

Strip HTML From Text

| sed 's|<[^>]*>||g'

Remove up to last backslash from URL using sed

sed 's/^.*[/]//'


curl -I -s "" | grep 'location:' | sed 's/^.*[/]//'

Unzip Multiple Files

The trick is to use a single quote around the zip filenames...

unzip '*.zip'
unzip 'CSV*.zip'

Delete Blank Lines

sed '/^$/d'
cat file.txt | sed '/^$/d'

Add Blank Line

For adding a newline after a pattern...

sed '/pattern/{G;}'

Show Just Network IP Address

ip -4 addr show | grep -oP '(?<=inet\s)\d+(\.\d+){3}' | grep -v '127'

Sort Text File Containing IP Addresses

Using a dot as a delimiter, sort by the 3rd column in numerical order, then sort by the 4th column in numerical order :-)

sort -t . -k 3,3n -k 4,4n

Execute Bash Script Directly From a URL

For example the FritzFrog Detection script...

sudo curl -s | bash

On an AWS EC2 or AWS Lightsail, you can even run it without being logged in to a full SSH session...

ssh my-ec2-01 sudo curl -s | bash

Redirect Output and Error to a Black Hole

command >/dev/null 2>&1

Extract Substring From Variable


For example, the string '0copy' can be cut in two with...


...and tested with...

echo "${AUDIOTRACK}"
echo "${AUDIOCOPY}"

...which shows as...


Show Free Privileged Ports

This command line will give you all the free port numbers not used in the /etc/services file...

cat /etc/services | awk '{ print $2 }' | tr -d './[:alpha:]' | sort -n | uniq | awk '$1<1024' | \grep "\S" | awk '$1!=p+1{print p+1"-"$1-1}{p=$1}'

...and this shows you which port numbers are used...

cat /etc/services | awk '{ print $2 }' | tr -d './[:alpha:]' | sort -n | uniq | awk '$1<1024' | \grep "\S"

Using Netcat NC To Transfer Files Between Computers

On the host receiving the file...

nc -l port >file

On the client sending the file...

nc host port <file

ZIP Excluding Directories

zip -rv path/ -x path/to/exclude/**\*
zip -rv path/ -x path/to/exclude/**\* second/path/to/exclude/**\*

Check If Service Is Running And Restart If Not Running



case "$(pidof ${SERVICE} | wc -w)" in

0)  echo "Restarting ${SERVICE}:"
    service lighttpd restart
1)  echo "${SERVICE} already running"



Repeat String Echo x Times

echo $x{1..13}"Hello?"

Assign Default Variables

Turn on shell debugging output...

#!/bin/bash -x

Assign the default variable if not set...


Show the default variable if not set...

echo ${grandslam:=Maria Sharapova}

Mount USB Drive by UUID

This is safer than using the traditional block names (/dev/sdcX) because drives can be swapped or get lost by the system. Modern Linux distributions now use this method when booting and mount your main hard disk drive by its UUID instead of the old way. The UUID is generated randomly by the formatting software.

Create directory for mount point...

sudo mkdir /mnt/usb

Find out the Universally Unique Identifier...

sudo blkid

Mount the USB drive using that UUID to that mount point...

sudo mount -U "0b4df3c4-69e0-4087-8162-whatever-usb-drive-is" /mnt/usb

Kill Process Name If Running

if pgrep rsync; then pkill rsync; fi

Geolocation Of IP Address From Firewall Log

sudo apt-get install geoip-bin
egrep 'Jul 24.*UFW.*DPT=22' syslog | awk '{ print $12 }' | sed 's/SRC=//' | sort -n | uniq | xargs -L 1 geoiplookup --verbose


curl --silent | grep -o --color=never -E '[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}'

PING With Date and Time

ping -i1 -c 1000000 | while read line; do echo `date` - $line; done

PING Your Nameservers

alias pingns="fping `grep 'nameserver' /etc/resolv.conf | awk '{ print $2 }' | tr '\n' ' '`"

PING Multiple Hosts At Once

sudo apt-get -y install fping


ALIAS: Ping Your Router

alias pingr="ping -c3 `route -n | grep 'UG[ \t]' | head -n1 | awk '{print $2}'`"

HOWTO: Reboot On Connection Failure

echo '*/20 * * * * root /usr/bin/host > /dev/null 2>1 || (/usr/bin/logger "Rebooting due to connectivity issue"; /sbin/shutdown -r now)' > /etc/cron.d/reboot-on-connection-failure

HOWTO: Open XFCE Terminal Run Command With Working Prompt

xfce4-terminal -e 'bash -c "/usr/local/bin/ -t ls; bash"' -T "todo"

Thanks -

HOWTO: Work with Encrypted Disks on Command Line

LUKS cryptsetup LVM

Rescuing LVM+LUKS encrypted drive from LiveCD or LiveUSB

HOWTO: Find Your DHCP Server


sudo nmap --script broadcast-dhcp-discover -e <network-interface>

Sample Output:

Starting Nmap 7.01 ( ) at 2017-08-29 10:35 BST
Pre-scan script results:
| broadcast-dhcp-discover: 
|   Response 1 of 1: 
|     IP Offered:
|     DHCP Message Type: DHCPOFFER
|     Server Identifier:
|     IP Address Lease Time: 2m00s
|     Renewal Time Value: 1m00s
|     Rebinding Time Value: 1m45s
|     Subnet Mask:
|     Broadcast Address:
|     Domain Name:
|     Domain Name Server:
|     Name Server:
|_    Router:

Thanks -

HOWTO: Edit A File While Viewing It

less file.txt
(press the V key)

Thanks -

HOWTO: Find And Stop A Process By Name

This will find the Amazon S3 Tool s3cmd and stop it if it running... great for overnight backups that are still running the next morning and clogging up the office internet.

echo "$(pgrep s3cmd)"; kill "$(pgrep s3cmd)"


No Space Left On Device

sudo df -ih

Filesystem     Inodes IUsed IFree IUse% Mounted on
/dev/xvda1       1.0M  1.0M    0K  100% /

Find where it is eating all the space then prune a few days at a time. Below is an exmple of the Magento Session Cache not being cleared regularly and filling up the hard drive. In this command, we look for all files older than 120 days and delete them...

find /var/www/ -type f -mtime +120 -exec rm {} \;

ERROR: Unable To Shutdown System

echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger

Thanks -

HOWTO: Loop Through Directory Performing Action In Each

for d in */; do
    sleep 1s
    echo "$d"
    filebot -script fn:artwork.tmdb "$d"

HOWTO: Play Audio CD

sudo apt-get install nvlc
nvlc cdda://

HOWTO: Convert Animating GIF Into Separate Images

convert animating.gif frames.png

Thanks -

HOWTO: Timestamp File Based On Another File

touch --reference=/path/to/file /this/other/file

HOWTO: prevent 'stdin: is not a tty' errors when using SSH

Add the following line to the top of your ~/.bashrc file...

[ -z "$PS1" ] && return

Thanks -

HOWTO: Restore Tar Backup Over SSH

On the target filesystem, switch to root, change to the directory you want to extract to, then ssh in to the source server and cat to tar...

su - root
cd /
ssh root@ "cat /backup/wwwdata.tar.gz" | tar zxvf -

Thanks -

HOWTO: Extract Without First Directory

tar -xzvpf filename.tar.gz --strip-components=1 -C /path/to/extract/to/

e.g. the Magento tarball has the folder /magento/ in it, so we remove that with...

tar -xjvpf magento- --strip-components=1 -C /home/website/public_html/

HOWTO: Sort Linux Password File By User ID (UID)

sudo sort -t : -k 3 -g /etc/passwd

-t = separator -k = column

HOWTO: Get Memcached Stats By Using Netcat

echo stats | nc 11211

HOWTO: Kick Users Off The System

Sometimes, you forget to log out from the system. To remove your old login, connect via SSH, find out the PID, then kill it.

The actual system login (via a keyboard attached) is always ttyX.

who -u

plittlefield   tty2         2014-04-29 15:40  old         3333
root           pts/2        2014-06-02 10:45    .         8336

kill -9 3333

HOWTO: Force Filesystem Check On Reboot

sudo touch /forcefsck
sudo reboot

HOWTO: Close A Program Remotely (and Nicely!) In Terminal

Yes, you could use pkill or killall, but there is a nicer way.

Log in to your PC via SSH, then...

Install the software...

sudo apt-get install wmctrl

List running apps...

paully@mythbuntu-server3:~$ DISPLAY=:0.0 wmctrl -l
0x01600004 -1 mythbuntu-server3 xfce4-panel
0x01800003 -1 mythbuntu-server3 Desktop
0x01600020 -1 mythbuntu-server3 xfce4-panel
0x01e00004  0 mythbuntu-server3 Terminal - paully@mythbuntu-server3: ~

Close the app...

DISPLAY=:0.0 wmctrl -c "Terminal"

Show which programs you've been running the most in current session

hash | sort -n

How To Go To A Specific Line In LESS

Put +LINE_NUMBERg in front of the file you want to view...

less +50g /path/to/file

Securely forward TCP connections on local port 1234 to remote2 port 4321

ssh -L 1234:

Download Multiple Files / URLs Using Wget -i

First, store all the download files or URLs in a text file as:

cat > download-file-list.txt

Next, give the download-file-list.txt as argument to wget using -i option as shown below.

wget -i download-file-list.txt

USB Scanner / Scanning On The Command Line

Example used is a Canon CanoScan N670U/N676U/LiDE20 flatbed scanner...

lsusb -d 04a9:220d -v
scanimage -L
scanimage --help -d plustek:libusb:006:004
scanimage -d plustek:libusb:006:004 --resolution 75 --format=tiff >/tmp/output.tiff

Thanks -

Array Looping

ARRAY=( one two three four )
for i in "${ARRAY[@]}"
    echo Showing $i...

How To Burn A Directory On To A DVD

sudo apt-get install dvd+rw-tools mkisofs cdrecord
mkisofs -r -o /tmp/mydvd.iso /path/to/folder
growisofs -Z /dev/dvd=/tmp/mydvd.iso


Enhanced dd with progress bars

sudo apt-get install dcfldd

Monitor The Progress Of dd with Pipe Viewer

dd if=/dev/zero of=/dev/sdX bs=8M &
echo %1
ls -l /proc/XXXXX/fd
pv -d XXXXX:1


dd if=/dev/mmcblk0 | pv -s 4G -peta | gzip -1 > /media/sda1/sd_backup.img.gz

This displays a progress indicator like the following.

0:15:01 [1.57MB/s] [====================>               ] 51% ETA 0:14:06

Progress Of DD Command

Begin dd in 1 terminal window. This will wipe an external USB drive...

dd if=/dev/zero bs=1M of=/dev/sdb

In another terminal window, run this...

watch -n5 'sudo kill -USR1 `pgrep ^dd`'

Now switch back to the first window...

4701814784 bytes (4.7 GB) copied, 197.487 s, 23.8 MB/s
4595+0 records in
4595+0 records out
4818206720 bytes (4.8 GB) copied, 202.506 s, 23.8 MB/s
4706+0 records in
4706+0 records out
4934598656 bytes (4.9 GB) copied, 207.578 s, 23.8 MB/s
4818+0 records in
4818+0 records out
5052039168 bytes (5.1 GB) copied, 212.662 s, 23.8 MB/s
4930+0 records in
4930+0 records out
5169479680 bytes (5.2 GB) copied, 217.737 s, 23.7 MB/s
5040+0 records in
5040+0 records out
5284823040 bytes (5.3 GB) copied, 222.771 s, 23.7 MB/s
5151+0 records in
5151+0 records out
5401214976 bytes (5.4 GB) copied, 227.784 s, 23.7 MB/s

Steganography or How To Hide A File Inside Image

How to hide files within a picture file (especially png and jpeg files) and all you'll need is a program like WinRAR or RAR.

Firstly you'll need to find two things: the first is a picture that you'll use to hide your file in, the second is the actual file you want to hide.

Put both in the same folder, open a command prompt and navigate there.

Once there you'll need to put your file that you want to hide inside a zip file (rar or zip will work just fine, put a password or whatever if you want too). After this type the following into the command prompt...


copy /b display.png + secret_message.rar fake.png


cat display.png secret_message.rar > fake.png

...where display.png is the picture in which you want to hide the file, secret_message.rar is the file you want to hide and fake.png is the file name that you want the result to be created as.

After this is done open the newly created image file in your paint package and verify that it opens correctly. To see the hidden file all you need to do is open it in your favorite compression program and your done! One hidden file that can be uploaded to image sites and so forth.

You can also put a password on the rar hidden rar file in order to add extra protection.

Change User Password One Liner Batch Mode

echo "username:password" | chpasswd

Beautify A Shell Script

Upload File By FTP Command Line

ftp -u file.txt

Dictionary Attack On SSH

Oct 16 08:53:30 server2 sshd[23415]: Invalid user amcssa from
Oct 16 08:53:32 server2 sshd[23417]: Invalid user wupr from
Oct 16 08:53:34 server2 sshd[23419]: Invalid user glbt from
Oct 16 08:53:36 server2 sshd[23421]: Invalid user wusgg from
Oct 16 08:53:38 server2 sshd[23423]: Invalid user wundergr from
Oct 16 08:53:40 server2 sshd[23425]: Invalid user sos from
Oct 16 08:53:42 server2 sshd[23427]: Invalid user s0s from
Oct 16 08:53:44 server2 sshd[23429]: Invalid user sos1 from
Oct 16 08:53:46 server2 sshd[23431]: Invalid user overflow from
Oct 16 08:53:48 server2 sshd[23433]: Invalid user mariachi from
Oct 16 08:53:50 server2 sshd[23435]: Invalid user fsae from
Oct 16 08:53:52 server2 sshd[23437]: Invalid user wuma from
Oct 16 08:53:54 server2 sshd[23439]: Invalid user vsa from
Oct 16 08:53:56 server2 sshd[23441]: Invalid user pride from
Oct 16 08:53:57 server2 sshd[23443]: Invalid user yarn from
Oct 16 08:54:00 server2 sshd[23445]: Invalid user mixed from
Oct 16 08:54:02 server2 sshd[23447]: Invalid user mix3d from
Oct 16 08:54:03 server2 sshd[23449]: Invalid user washucrs from
Oct 16 08:54:05 server2 sshd[23451]: Invalid user kungfu from
Oct 16 08:54:07 server2 sshd[23453]: Invalid user lnyf from

How To Quickly Generate A Large File On The Command Line

To make a 1Mb file...

dd if=/dev/zero of=1mb_file count=1024 bs=1024

Foreach Loop In Script

for i in 1 2 3
  /usr/bin/nmap -sP -n "192.168.0.$i" |grep 'IP'

Extract A Single File From A Tarball Archive

cd /tmp/
tar -xjvpf /path/to/stage4.tar.bz2 etc/conf.d/modules

Create Cool SHA Code From Date (Forum Spam Bot Prevention)

date -u +%jXfce|sha256sum|sed 's/\W//g'

Format A USB Flash Drive With Linux Filesystem

Identify your drive:

cat /proc/partitions
dmesg |tail

Repartition drive:

fdisk /dev/sdb

Format drive:

mkfs.ext3 -L "BACKUP-32GB-B" -v /dev/sdb1

Output examples:

thinkpad ~ # dmesg |tail
sd 10:0:0:0: [sdb] Assuming drive cache: write through
sd 10:0:0:0: [sdb] No Caching mode page present
sd 10:0:0:0: [sdb] Assuming drive cache: write through
   sdb: sdb1
sd 10:0:0:0: [sdb] No Caching mode page present
sd 10:0:0:0: [sdb] Assuming drive cache: write through
sd 10:0:0:0: [sdb] Attached SCSI removable disk

thinkpad ~ # cat /proc/partitions 
major minor  #blocks  name
  8        0  312571224 sda
  8        1    1228800 sda1
  8        2   83103744 sda2
  8        3  114118656 sda3
  8        4  114109695 sda4
  8       16   31696896 sdb

thinkpad ~ # fdisk /dev/sdb
Command (m for help): p
Disk /dev/sdb: 32.5 GB, 32457621504 bytes
64 heads, 32 sectors/track, 30954 cylinders, total 63393792 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc3072e18

  Device Boot      Start         End      Blocks   Id  System

Command (m for help): n
Partition type:
  p   primary (0 primary, 0 extended, 4 free)
  e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-63393791, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-63393791, default 63393791): 
Using default value 63393791

Command (m for help): p

Disk /dev/sdb: 32.5 GB, 32457621504 bytes
64 heads, 32 sectors/track, 30954 cylinders, total 63393792 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xc3072e18

  Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    63393791    31695872   83  Linux

Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.

thinkpad ~ # mkfs.ext3 -L "BACKUP-32GB-B" -v /dev/sdb1
mke2fs 1.42 (29-Nov-2011)
fs_types for mke2fs.conf resolution: 'ext3'
Filesystem label=BACKUP-32GB-B
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
1982464 inodes, 7923968 blocks
396198 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
242 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000
Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

thinkpad ~ # umount /media/BACKUP-32GB-B/

thinkpad ~ # fsck.ext3 -v -C0 /dev/sdb1
e2fsck 1.42 (29-Nov-2011)
BACKUP-32GB-B: clean, 11/1982464 files, 168470/7923968 blocks

What Desktop Session Am I Using?


Schedule With AT Command

at 8pm
at midnight tonight
at 3pm tomorrow

To view scheduled 'at' jobs:


To delete scheduled 'at' jobs:

atrm 2

Where '2' is the job number found by using the 'atq' command

HOWTO: Use md5sum

md5sum -c