Blogosphere, Linux, productivity, Technology and Software, tricks

cURL – Swiss Army Knife for Web Sites

Lately, I have been using cURL a lot to diagnose & get more information about web pages download times, header information, automation of such tasks etc. Therefore I thought some of the very useful but not so obvious usage of cURL.

first of all, if you want to download a copy of a page to a local file, you can use

curl http://www.example.com -o local_copy.htm

If you don’t want to neccessarily download the file but only want to see response headers, you can do

curl -I http://www.example.com

If the web site is returning a redirect, 30x HTTP response code, you can ask curl to follow the redirect for example

curl -IL http://www.example.com

You can specify multiple URLs to curl as part of pattern for instance:

curl -I http://{www,play,sawaal}.ibibo.com

Output:

[1/3]: http://www.ibibo.com –> <stdout>
–_curl_–http://www.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:05 GMT
Server: Apache
Vary: Accept-Encoding
ServerID: Myibibo-34
Content-Type: text/html

[2/3]: http://play.ibibo.com –> <stdout>
–_curl_–http://play.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:08 GMT
Server: ibibo-WS
Cache-Control: no-transform
Vary: User-Agent,Accept,Accept-Encoding
ServerID: 25
Content-Type: text/html

[3/3]: http://sawaal.ibibo.com –> <stdout>
–_curl_–http://sawaal.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:08 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Set-Cookie: ASP.NET_SessionId=tu5c1y555oywv4fvxj1vj245; path=/; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 112910

You can also specify advanced regular expressions when using the URLs for instance:

ftp://ftp.numericals.com/file[1-100].txt  ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)  ftp://ftp.letters.com/file[a-z].txt

Nested sequences are not supported, but you can use several ones next to each other:

http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

You can specify any amount of URLs on the command line. They will be fetched in a sequential manner in the specified order.

You can specify a step counter for the ranges to get every Nth number or letter:

http://www.numericals.com/file[1-100:10].txt  http://www.letters.com/file[a-z:2].txt

More later in another post.

Advertisements
Geeks, Tech, Startups, Linux, Ruby on Rails, RubyOnRails

Free PHP 5 / Rails Hosting for Facebook app developers

Facebook developers get some more good news. If you are worried about having a reliable hosting space for your Facebook application, you can get it Free. Joyent is giving free PHP5 hosting along with MySQL, Rails 1.8.6 ,Python free for 1 year, if you are a Facebook application developer.

Free Accelator

Other companies offering free hosting for Facebook developers include Jumpline.com & Freeunmeteredhost.com.

Close Read more >>   Options >>  

Visit Answers.com

Linux, Technology and Software

KNOPPIX – Linux Live CD

I recently tried Knoppix.What

is Knoppix:

KNOPPIX is a bootable Live system on CD or DVD, consisting of a representative collection of GNU/Linux software, automatic hardware detection, and support for many graphics cards, sound cards, SCSI and USB devices and other peripherals.

It just simply works great. I have tried several other linux distros Slackware, Mandrake, Meentoo etc but all failed to detect one or another hardware device on my PC. Sometimes network card, mouse, USB etc etc but Knoppix detected everything and was able to connect to my Router without any configuration at all. It has all the latest stuff such as Skype, Gaim, FireFox etc that I use.

Knoppix is highly recommended who is looking for an easy to setup Linux distribution.