BASH Website

HOWTO: Dump HTTP Header Using WGET

wget --server-response --spider http://www.google.co.uk

HOWTO: Download Web Page And All Files Linked

$ wget -r -np -k http://syncapp.bittorrent.com/1.4.111/

Download An Entire Web Site

wget --quiet --recursive --no-clobber --page-requisites --html-extension --convert-links --domains domain.co.uk --no-parent http://www.domain.co.uk/

HOWTO: Show Web Page Headers

Terminal:~$

curl -I www.bbc.co.uk

Sample output...

HTTP/1.1 200 OK
Server: Apache
Content-Type: text/html
Content-Language: en-GB
Etag: "4c572592f520bedc9a1e2c0238accaa6"
X-PAL-Host: pal041.back.live.cwwtf.local:80
Transfer-Encoding: chunked
Date: Fri, 06 Mar 2015 11:30:51 GMT
Connection: keep-alive
Set-Cookie: BBC-UID=15d4ff99887f5eab78a60516b12f651317b26b71e7f4e4a61ad0970254e467200curl/7.35.0; expires=Tue, 05-Mar-19 11:30:51 GMT; path=/; domain=.bbc.co.uk
X-Cache-Action: HIT
X-Cache-Hits: 1149
X-Cache-Age: 64
Cache-Control: private, max-age=0, must-revalidate
Vary: X-CDN

Thanks to CyberCiti.