BASH

In a terminal example anything shown in bold after Terminal:~$ is a comment and is not to be run, e.g:

Terminal:~$ sudo apt-get install fish Then scratch your ear

So in the above Then scratch your ear is not to be run in the terminal

BASH AWK

BASH CRON

BASH Disable

BASH File Formats

BASH Find

BASH Fix

BASH Hardware Discovery

BASH History

BASH IP Tables

BASH Modification

BASH MP3

BASH Networking

BASH PDF

BASH Shorewall

BASH Terminal Process

BASH Time and Date

Convert Tabs To Spaces In a File
expand file.txt

Delete Characters On Just The First Line
This deletes all the quotation (") characters on just the first line of a CSV file...

sed '1s/"//g' /tmp/oldfile.csv > /tmp/newfile.csv

http://sed.sourceforge.net/sed1line.txt

HOWTO: Mass File Rename Deleting Part Of Name
e.g.

Scooby_Doo_Mystery_Incorporated_S02E25 Through The Curtain 720p x264.mp4 => Scooby_Doo_Mystery_Incorporated_S02E25.mp4

Do...

renamexm -v -s/" .*.mp4"/".mp4"/e *.mp4

HOWTO: Rename All Files Add File Extension
for f in *; do mv "$f" "$f.jpg"; done

HOWTO: Find Out Length Of Longest Line In A File
wc -L file

HOWTO: Find Line Exceeding N Characters
grep '.\{N\}' file

HOWTO: Trim Leading White Space
sed 's/^ *//g'

e.g.

lynx -nolist -dump http://wiki.indie-it.com/wiki/Television_Paully |sed -n '/What I Am Currently Watching/,/What I Have Been Watching/p' |sed 's/^ *//g' |egrep '^[0-9]'

Thanks to Stackoverflow.

HOWTO: Download Web Page And All Files Linked
$ wget -r -np -k http://syncapp.bittorrent.com/1.4.111/

HOWTO: Pattern Match Between 2 Lines
I Love Linux ***** BEGIN ***** BASH is awesome BASH is awesome ***** END ***** I Love Linux

sed -n '/BEGIN/,/END/p'

Thanks to Shell Hacks]

HOWTO: Show Web Page Headers
Terminal:~$ curl -I www.bbc.co.uk

Sample output...

HTTP/1.1 200 OK Server: Apache Content-Type: text/html Content-Language: en-GB Etag: "4c572592f520bedc9a1e2c0238accaa6" X-PAL-Host: pal041.back.live.cwwtf.local:80 Transfer-Encoding: chunked Date: Fri, 06 Mar 2015 11:30:51 GMT Connection: keep-alive Set-Cookie: BBC-UID=15d4ff99887f5eab78a60516b12f651317b26b71e7f4e4a61ad0970254e467200curl/7.35.0; expires=Tue, 05-Mar-19 11:30:51 GMT; path=/; domain=.bbc.co.uk X-Cache-Action: HIT X-Cache-Hits: 1149 X-Cache-Age: 64 Cache-Control: private, max-age=0, must-revalidate Vary: X-CDN

Thanks - http://www.cyberciti.biz/tips/howto-watch-http-headers.html

HOWTO: Restore Tar Backup Over SSH
On the target filesystem, switch to root, change to the directory you want to extract to, then ssh in to the source server and cat to tar...

su - root cd / ssh root@192.168.1.201 "cat /backup/wwwdata.tar.gz" | tar zxvf -

Thanks - http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/

HOWTO: Test Existence Of Directory
test -d /path/to/directory

You can use this to make sure a directory exists before copying or moving a file into it.

test -d /mnt/usb/backup && mv /home/website/backup-20140901.zip /mnt/usb/backup/

HOWTO: Dump HTTP Header Using WGET
wget --server-response --spider http://www.google.co.uk

HOWTO: Extract Without First Directory
tar -xzvpf filename.tar.gz --strip-components=1 -C /path/to/extract/to/

e.g. the Magento tarball has the folder /magento/ in it, so we remove that with...

tar -xjvpf magento-1.9.0.1.tar.bz2 --strip-components=1 -C /home/website/public_html/

HOWTO: Remove Blank Lines From A Text File
sed -i '/^$/d' file.txt

HOWTO: Sort Linux Password File By User ID (UID)
sudo sort -t : -k 3 -g /etc/passwd

-t = separator -k = column

HOWTO: Get Memcached Stats By Using Netcat
echo stats | nc 127.0.0.1 11211

HOWTO: Kick Users Off The System
Sometimes, you forget to log out from the system. To remove your old login, connect via SSH, find out the PID, then kill it.

The actual system login (via a keyboard attached) is always ttyX.

who -u plittlefield  tty2         2014-04-29 15:40  old         3333 root          pts/2        2014-06-02 10:45. 8336 kill -9 3333

HOWTO: Force Filesystem Check On Reboot
sudo touch /forcefsck sudo reboot

HOWTO: Close A Program Remotely (and Nicely!) In Terminal
Yes, you could use pkill or killall, but there is a nicer way.

Log in to your PC via SSH, then...

Install the software...

sudo apt-get install wmctrl

List running apps...

paully@mythbuntu-server3:~$ DISPLAY=:0.0 wmctrl -l 0x01600004 -1 mythbuntu-server3 xfce4-panel 0x01800003 -1 mythbuntu-server3 Desktop 0x01600020 -1 mythbuntu-server3 xfce4-panel 0x01e00004 0 mythbuntu-server3 Terminal - paully@mythbuntu-server3: ~

Close the app...

DISPLAY=:0.0 wmctrl -c "Terminal"

http://manpages.ubuntu.com/manpages/quantal/en/man1/wmctrl.1.html

HOWTO: Show The Resolution / DPI Of An Image
identify -verbose photo.jpg |grep 'Resolution' Resolution: 72x72

Show which programs you've been running the most in current session
hash | sort -n

How To Go To A Specific Line In LESS
Put +LINE_NUMBERg in front of the file you want to view...

less +50g /path/to/file

HOWTO: Display True Memory Usage
Most people use the free command to show RAM usage, but read it wrong. In the example below, it looks like the computer only has 67 MB available. However, Linux keeps data stored in RAM when reading disks and some more for programs just in case they need it, effectively being greedy for better performance.

What the Linux system is actually using or needs to use is shown on the 3rd line!

free -m total     used     free   shared     buffers     cached Mem:     4049      3982       67        0          16       3530 -/+ buffers/cache:  435     3614 Swap:    6142        53     6088

So, a slightly tweaked command would be...

free -m | awk 'NR==3 {print $4 " MB free."}' 3614 MB free.

(The above command for awk translates to "print 4th column of third line" :-)

Thanks to - http://thoughtsbyclayg.blogspot.co.uk/2008/09/display-free-memory-on-linux-ubuntu.html

Securely forward TCP connections on local port 1234 to remote2 port 4321
ssh -L 1234: http://remote2.example.com :4321 username@remote.example.com

Download An Entire Web Site
wget --quiet --recursive --no-clobber --page-requisites --html-extension --convert-links --domains domain.co.uk --no-parent http://www.domain.co.uk/

Download Multiple Files / URLs Using Wget -i
First, store all the download files or URLs in a text file as:

cat > download-file-list.txt URL1 URL2 URL3 URL4

Next, give the download-file-list.txt as argument to wget using -i option as shown below.

wget -i download-file-list.txt

USB Scanner / Scanning On The Command Line
Example used is a Canon CanoScan N670U/N676U/LiDE20 flatbed scanner...

lsusb lsusb -d 04a9:220d -v sane-find-scanner scanimage -L scanimage --help -d plustek:libusb:006:004 scanimage -d plustek:libusb:006:004 --resolution 75 --format=tiff >/tmp/output.tiff

Thanks - http://theopoon.rinnovative.com/2012/08/22/canon-canoscan-n670u-on-ubuntu-12-04-server/

Array Looping
ARRAY=( one two three four ) for i in "${ARRAY[@]}" do    echo Showing $i... echo done

How To Burn A Directory On To A DVD
sudo apt-get install dvd+rw-tools mkisofs cdrecord mkisofs -r -o /tmp/mydvd.iso /path/to/folder growisofs -Z /dev/dvd=/tmp/mydvd.iso

Monitor The Progress Of dd with Pipe Viewer
dd if=/dev/mmcblk0 | pv -s 4G -peta | gzip -1 > /media/sda1/sd_backup.img.gz

This displays a progress indicator like the following.

0:15:01 [1.57MB/s] [====================>              ] 51% ETA 0:14:06

Progress Of DD Command
Begin dd in 1 terminal window. This will wipe an external USB drive...

dd if=/dev/zero bs=1M of=/dev/sdb

In another terminal window, run this...

watch -n5 'sudo kill -USR1 `pgrep ^dd`'

Now switch back to the first window...

4701814784 bytes (4.7 GB) copied, 197.487 s, 23.8 MB/s 4595+0 records in 4595+0 records out 4818206720 bytes (4.8 GB) copied, 202.506 s, 23.8 MB/s 4706+0 records in 4706+0 records out 4934598656 bytes (4.9 GB) copied, 207.578 s, 23.8 MB/s 4818+0 records in 4818+0 records out 5052039168 bytes (5.1 GB) copied, 212.662 s, 23.8 MB/s 4930+0 records in 4930+0 records out 5169479680 bytes (5.2 GB) copied, 217.737 s, 23.7 MB/s 5040+0 records in 5040+0 records out 5284823040 bytes (5.3 GB) copied, 222.771 s, 23.7 MB/s 5151+0 records in 5151+0 records out 5401214976 bytes (5.4 GB) copied, 227.784 s, 23.7 MB/s

Hide File Inside Image
How to hide files within a picture file (especially png and jpeg files) and all you'll need is a program like WinRAR or RAR.

Firstly you'll need to find two things: the first is a picture that you'll use to hide your file in, the second is the actual file you want to hide.

Put both in the same folder, open a command prompt and navigate there.

Once there you'll need to put your file that you want to hide inside a zip file (rar or zip will work just fine, put a password or whatever if you want too). After this type the following into the command prompt...

Windows

copy /b display.png + secret_message.rar fake.png

Linux

cat display.png secret_message.rar > fake.png

...where display.png is the picture in which you want to hide the file, secret_message.rar is the file you want to hide and fake.png is the file name that you want the result to be created as.

After this is done open the newly created image file in your paint package and verify that it opens correctly. To see the hidden file all you need to do is open it in your favorite compression program and your done! One hidden file that can be uploaded to image sites and so forth.

You can also put a password on the rar hidden rar file in order to add extra protection.

Find And Copy Files With Directory Structure
find /path/to/search/ -type f -name '*.mov' -print -exec cp --parents "{}" /path/to/copy/to/ \;

Change User Password One Liner Batch Mode
echo "username:password" | chpasswd

Beautify A Shell Script
http://www.arachnoid.com/linux/beautify_bash/

Upload File By FTP Command Line
ftp -u ftp://username:password@www.domain.co.uk/remote/dir/file.txt file.txt

Speed Ping
alias ping='ping -c3 -n -i 0.2 -W1' alias pingg='ping www.google.co.uk' alias pingp='ping www.paully.co.uk'

Dictionary Attack On SSH
Oct 16 08:53:30 server2 sshd[23415]: Invalid user amcssa from 88.191.133.21 Oct 16 08:53:32 server2 sshd[23417]: Invalid user wupr from 88.191.133.21 Oct 16 08:53:34 server2 sshd[23419]: Invalid user glbt from 88.191.133.21 Oct 16 08:53:36 server2 sshd[23421]: Invalid user wusgg from 88.191.133.21 Oct 16 08:53:38 server2 sshd[23423]: Invalid user wundergr from 88.191.133.21 Oct 16 08:53:40 server2 sshd[23425]: Invalid user sos from 88.191.133.21 Oct 16 08:53:42 server2 sshd[23427]: Invalid user s0s from 88.191.133.21 Oct 16 08:53:44 server2 sshd[23429]: Invalid user sos1 from 88.191.133.21 Oct 16 08:53:46 server2 sshd[23431]: Invalid user overflow from 88.191.133.21 Oct 16 08:53:48 server2 sshd[23433]: Invalid user mariachi from 88.191.133.21 Oct 16 08:53:50 server2 sshd[23435]: Invalid user fsae from 88.191.133.21 Oct 16 08:53:52 server2 sshd[23437]: Invalid user wuma from 88.191.133.21 Oct 16 08:53:54 server2 sshd[23439]: Invalid user vsa from 88.191.133.21 Oct 16 08:53:56 server2 sshd[23441]: Invalid user pride from 88.191.133.21 Oct 16 08:53:57 server2 sshd[23443]: Invalid user yarn from 88.191.133.21 Oct 16 08:54:00 server2 sshd[23445]: Invalid user mixed from 88.191.133.21 Oct 16 08:54:02 server2 sshd[23447]: Invalid user mix3d from 88.191.133.21 Oct 16 08:54:03 server2 sshd[23449]: Invalid user washucrs from 88.191.133.21 Oct 16 08:54:05 server2 sshd[23451]: Invalid user kungfu from 88.191.133.21 Oct 16 08:54:07 server2 sshd[23453]: Invalid user lnyf from 88.191.133.21

How To Quickly Generate A Large File On The Command Line
To make a 1Mb file...

dd if=/dev/zero of=1mb_file count=1024 bs=1024

http://www.skorks.com/2010/03/how-to-quickly-generate-a-large-file-on-the-command-line-with-linux/

Show Size Of Directories
Sorted by time...

du --time -s */ |sort -k 2

Sorted by size..

du --time -s */ |sort -k 1 -h

Calculator
echo $[1+1]

or

scale=2; echo "11.45 + 7.95" | bc

Foreach Loop In Script
for i in 1 2 3 do  /usr/bin/nmap -sP -n "192.168.0.$i" |grep 'IP' done
 * 1) !/bin/bash

Extract A Single File From A Tarball Archive
cd /tmp/ tar -xjvpf /path/to/stage4.tar.bz2 etc/conf.d/modules

Create Cool SHA Code From Date (Forum Spam Bot Prevention)
date -u +%jXfce|sha256sum|sed 's/\W//g'

Remove Blank Lines From Text File
cat myfile.txt |sed '/^$/d'

Format A USB Flash Drive With Linux Filesystem
Identify your drive:

cat /proc/partitions dmesg |tail

Repartition drive:

fdisk /dev/sdb d n p 1 [enter] [enter] w

Format drive:

mkfs.ext3 -L "BACKUP-32GB-B" -v /dev/sdb1

Output examples:

thinkpad ~ # dmesg |tail sd 10:0:0:0: [sdb] Assuming drive cache: write through sd 10:0:0:0: [sdb] No Caching mode page present sd 10:0:0:0: [sdb] Assuming drive cache: write through sdb: sdb1 sd 10:0:0:0: [sdb] No Caching mode page present sd 10:0:0:0: [sdb] Assuming drive cache: write through sd 10:0:0:0: [sdb] Attached SCSI removable disk thinkpad ~ # cat /proc/partitions major minor #blocks  name 8       0  312571224 sda 8       1    1228800 sda1 8       2   83103744 sda2 8       3  114118656 sda3 8       4  114109695 sda4 8      16   31696896 sdb thinkpad ~ # fdisk /dev/sdb Command (m for help): p Disk /dev/sdb: 32.5 GB, 32457621504 bytes 64 heads, 32 sectors/track, 30954 cylinders, total 63393792 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc3072e18 Device Boot     Start         End      Blocks   Id  System Command (m for help): n Partition type: p  primary (0 primary, 0 extended, 4 free) e  extended Select (default p): p Partition number (1-4, default 1): 1 First sector (2048-63393791, default 2048): Using default value 2048 Last sector, +sectors or +size{K,M,G} (2048-63393791, default 63393791): Using default value 63393791 Command (m for help): p Disk /dev/sdb: 32.5 GB, 32457621504 bytes 64 heads, 32 sectors/track, 30954 cylinders, total 63393792 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xc3072e18 Device Boot     Start         End      Blocks   Id  System /dev/sdb1           2048    63393791    31695872   83  Linux Command (m for help): w The partition table has been altered! Calling ioctl to re-read partition table. Syncing disks. thinkpad ~ # mkfs.ext3 -L "BACKUP-32GB-B" -v /dev/sdb1 mke2fs 1.42 (29-Nov-2011) fs_types for mke2fs.conf resolution: 'ext3' Filesystem label=BACKUP-32GB-B OS type: Linux Block size=4096 (log=2) Fragment size=4096 (log=2) Stride=0 blocks, Stripe width=0 blocks 1982464 inodes, 7923968 blocks 396198 blocks (5.00%) reserved for the super user First data block=0 Maximum filesystem blocks=4294967296 242 block groups 32768 blocks per group, 32768 fragments per group 8192 inodes per group Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done thinkpad ~ # umount /media/BACKUP-32GB-B/ thinkpad ~ # fsck.ext3 -v -C0 /dev/sdb1 e2fsck 1.42 (29-Nov-2011) BACKUP-32GB-B: clean, 11/1982464 files, 168470/7923968 blocks

Shell Command Line Calculator
echo 'scale=25;57/43' | bc

http://www.basicallytech.com/blog/index.php?/archives/23-command-line-calculations-using-bc.html

What Desktop Session Am I Using?
echo $DESKTOP_SESSION

Schedule With AT Command
http://www.brunolinux.com/02-The_Terminal/The_at_Command.html

at 8pm

at midnight tonight

at 3pm tomorrow

To view scheduled 'at' jobs:

atq

To delete scheduled 'at' jobs:

atrm 2

Where '2' is the job number found by using the 'atq' command