BASH File Manipulation

CSV Files:

Delete Characters On Just The First Line

This deletes all the quotation (") characters on just the first line of a CSV file...

sed '1s/"//g' /tmp/oldfile.csv > /tmp/newfile.csv

http://sed.sourceforge.net/sed1line.txt

Text Files:

Number Beginning Of Lines

cat file.txt |nl

Convert Text To Lowercase

echo "FISH" | tr '[:upper:]' '[:lower:]'
cat file.txt | tr '[:upper:]' '[:lower:]'

View A Text File With NO Word Wrap

less -S file.txt

Remove Blank Lines

sed -i '/^$/d' file.txt
or
cat myfile.txt |sed '/^$/d'

Convert Tabs To Spaces In a File

expand file.txt

COPY FILE TO NEW DIRECTORY

IMG=~/Pictures/PhotosFromPhone/IMG_20150924_100801.jpg;
IMGDATE=$(exiftool "${IMG}" | grep -m1 'Create Date' |awk -F':' '{print $2"/"$3}' |tr -d ' ');
IMGDIR="~/Pictures/Photos/${IMGDATE}";
mkdir -p "$IMGDIR";
cp -a -v "${IMG}" "${IMGDIR}"

MOVE FILE TO NEW DIRECTORY

IMG=~/Pictures/PhotosFromPhone/IMG_20150924_100801.jpg;
IMGDATE=$(exiftool "${IMG}" | grep -m1 'Create Date' |awk -F':' '{print $2"/"$3}' |tr -d ' ');
IMGDIR="~/Pictures/Photos/${IMGDATE}";
mkdir -p "$IMGDIR";
mv -v "${IMG}" "${IMGDIR}"

COMPLETE WORKING SCRIPT

#!/bin/bash
PHOTOS=/home/paully/Pictures/PhotosFromPhone/*.jpg;
for P in ${PHOTOS}
do
IMG=${P};
IMGDATE=$(exiftool "${IMG}" | grep -m1 'Create Date' |awk -F':' '{print $2"/"$3}' |tr -d ' ');
IMGDIR="/home/paully/Pictures/Photos/${IMGDATE}";
mkdir -p "$IMGDIR";
cp -a -v "${IMG}" "${IMGDIR}"
done

Thanks to Stack Overflow

Trim Leading White Space

sed 's/^ *//g'

e.g.

lynx -nolist -dump http://wiki.indie-it.com/wiki/Television_Paully |sed -n '/What I Am Currently Watching/,/What I Have Been Watching/p' |sed 's/^ *//g' |egrep '^[0-9]'

Thanks to Stack Overflow.

Print Lines Between Two Patterns Using SED and AWK

To print (display) only text between two lines that contain patterns ‘BEGIN’ and ‘END’ in the following text file called info.txt.

I Love Linux
***** BEGIN *****
BASH is awesome
BASH is awesome
***** END *****
I Love Linux

Using SED

Allows the specifying of the starting pattern and the ending pattern, to print the lines between strings with these patterns.

sed -n '/StartPattern/,/EndPattern/p' FileName

Option Description:

• -n, –quiet, –silent - Suppress automatic printing of pattern space
• p - Print the current pattern space

Example:

sed -n '/BEGIN/,/END/p' info.txt

Using AWK

AWK works in a similar way to the sed command specifying the starting pattern and the ending pattern.

awk '/StartPattern/,/EndPattern/' FileName

Example:

awk '/BEGIN/,/END/' info.txt

Thanks to Shell Hacks

Image Files:

Create Directory From Photo Creation Date

sudo aptitude install libimage-exiftool-perl

YEAR/MONTH/DAY

IMG=~/Pictures/PhotosFromPhone/IMG_20150924_100801.jpg;
IMGDATE=$(exiftool "${IMG}" | grep -m1 'Create Date' |awk -F' ' '{print $4}' | tr ':' '/');
echo "${IMGDATE}";
mkdir -p "~/Pictures/Photos/${IMGDATE}"

YEAR/MONTH

IMG=~/Pictures/PhotosFromPhone/IMG_20150924_100801.jpg;
IMGDATE=$(exiftool "${IMG}" | grep -m1 'Create Date' |awk -F':' '{print $2"/"$3}' |tr -d ' ');
echo "${IMGDATE}";
mkdir -p "~/Pictures/Photos/${IMGDATE}"

Tweaks...

Show The Resolution / DPI Of An Image

identify -verbose photo.jpg |grep 'Resolution'

Resolution: 72x72

Filename Manipulation:

Add A Zero To The Beginning Of A Filename

$ renamexm -t -v -s/^/0/r [0-9] *.mp3
rename 1 Subdivisions.mp3   => 01 Subdivisions.mp3  : tested
rename 2 The Analog Kid.mp3 => 02 The Analog Kid.mp3 : tested
rename 3 Chemistry.mp3      => 03 Chemistry.mp3     : tested
rename 4 Digital Man.mp3    => 04 Digital Man.mp3   : tested
rename 5 The Weapon.mp3     => 05 The Weapon.mp3    : tested
rename 6 New World Man.mp3  => 06 New World Man.mp3 : tested
rename 7 Losing It.mp3      => 07 Losing It.mp3     : tested
rename 8 Countdown.mp3      => 08 Countdown.mp3     : tested

Mass File Rename Incrementing Number in File Name

export j=0
for i in *.mkv ; do let j+=1 ; mv -v $i "Willow the Wisp S01E$j.mkv" ; done

Mass File Rename Deleting Part Of Name

e.g.

Scooby_Doo_Mystery_Incorporated_S02E25 Through The Curtain 720p x264.mp4 => Scooby_Doo_Mystery_Incorporated_S02E25.mp4

Do...

renamexm -v -s/" .*.mp4"/".mp4"/e *.mp4

Rename All Files Add File Extension

for f in *; do mv "$f" "$f.jpg"; done

Finding Information Within A File

Length Of Longest Line In A File

wc -L file

Line Exceeding N Characters

grep '.\{N\}' file

To print (display) only text between two lines that contain patterns ‘BEGIN’ and ‘END’ in the following text file called info.txt.

I Love Linux

• • • • • BEGIN *****

BASH is awesome BASH is awesome

• • • • • END *****

I Love Linux