cURL – Swiss Army Knife for Web Sites

Lately, I have been using cURL a lot to diagnose & get more information about web pages download times, header information, automation of such tasks etc. Therefore I thought some of the very useful but not so obvious usage of cURL.

first of all, if you want to download a copy of a page to a local file, you can use

curl http://www.example.com -o local_copy.htm

If you don’t want to neccessarily download the file but only want to see response headers, you can do

curl -I http://www.example.com

If the web site is returning a redirect, 30x HTTP response code, you can ask curl to follow the redirect for example

curl -IL http://www.example.com

You can specify multiple URLs to curl as part of pattern for instance:

curl -I http://{www,play,sawaal}.ibibo.com

Output:

[1/3]: http://www.ibibo.com –> <stdout>
–_curl_–http://www.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:05 GMT
Server: Apache
Vary: Accept-Encoding
ServerID: Myibibo-34
Content-Type: text/html

[2/3]: http://play.ibibo.com –> <stdout>
–_curl_–http://play.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:08 GMT
Server: ibibo-WS
Cache-Control: no-transform
Vary: User-Agent,Accept,Accept-Encoding
ServerID: 25
Content-Type: text/html

[3/3]: http://sawaal.ibibo.com –> <stdout>
–_curl_–http://sawaal.ibibo.com
HTTP/1.1 200 OK
Date: Thu, 30 Jun 2011 18:19:08 GMT
Server: Microsoft-IIS/6.0
X-Powered-By: ASP.NET
X-AspNet-Version: 2.0.50727
Set-Cookie: ASP.NET_SessionId=tu5c1y555oywv4fvxj1vj245; path=/; HttpOnly
Cache-Control: private
Content-Type: text/html; charset=utf-8
Content-Length: 112910

You can also specify advanced regular expressions when using the URLs for instance:

ftp://ftp.numericals.com/file[1-100].txt  ftp://ftp.numericals.com/file[001-100].txt (with leading zeros)  ftp://ftp.letters.com/file[a-z].txt

Nested sequences are not supported, but you can use several ones next to each other:

http://any.org/archive[1996-1999]/vol[1-4]/part{a,b,c}.html

You can specify any amount of URLs on the command line. They will be fetched in a sequential manner in the specified order.

You can specify a step counter for the ranges to get every Nth number or letter:

http://www.numericals.com/file[1-100:10].txt  http://www.letters.com/file[a-z:2].txt

More later in another post.

Dr. Naresh Trehan & Medanta Medicity Management, Please Fix your hospital’s parking staff

Update (Jul 4th, 11) : After about 10 days, since Dr. Nalini Jain, Director, Medanta Hospital Contacted me, I am yet to hear regarding any response or update regarding any inquiry or action, whatsoever. Still Waiting for any response from Medanta 😦

Update (Jun 24th, 11) : Dr. Nalini Jain, Director, Medanta Hospital Contacted me & asked me further details about the experience, She consented that the behaviour of Parking Manager Mr. Rakesh Ranjan was extremely disrespectful and improper. The Medanta hospital doesn’t condone or approved of such rude behaviour to the visitors to the hospital, who anyway may be going through some distress upon arrival to the hospital. She assured me to look into this matter & take necessary action. I am glad to have talked to her and look forward for some real action.

Respected Dr. Trehan & Medanta Management Team,

First of all, I would like to congratulate & thank you from deep of my heart for having setup, such an esteemed medical institution, The Medanta medicity in Gurgaon, which has been a boon not only to Gurgaon / NCR residents but to patients from all across India and abroad.

I myself, have gone there a number of times and found the doctors, administrative staff, nurses, housekeeping, radiology, sample collection, cafeteria and all others staff extremely courteous, polite & helpful. My experience has been more than fulfilling & I thank you for the same.

On several occasions, I parked my car on the surface parking and there has never been any issues and the staff was very helpful. Today, I went there for some blood tests and to meet my doctor again & happen to decide to park in the under-ground parking.

That is when, a nightmare, which I wouldn’t even expect even from the worst of worst hospitals in India started. On basement 1, there were a number of vacant parking slots available but those were being cordoned by ropes and a few cars were being allowed, reasons best known to the parking attendants. I inquired, if I can park here as I was getting late for appointment, but was very rudely told that this floor is reserved for staff only and I need to go further down. When all over the campus, it is mentioned that the underground parking is Rs 20/-, It was beyond my understanding why the first floor was reserved for staff.

Upon asking, why some cars are being allowed, I was told that they are staff or they are pass holders. Never the less, I went one floor below again and wanted to park in the early free slots available but then again, I was told that I need to go further down and these are reserved. Again note, there were more than ample parking spaces available but they were cordoned by ropes. I asked to talk to supervisor but wasn’t allowed to do so.

Finally I parked my car & then I went up and then I wanted to meet the Parking Manager Mr. Rakesh Ranjan. I was hoping that he would at-least listen to my grievance but to my extreme surprise, he started giving me excuses that these spaces that are cordoned by ropes are meant for doctors. Upon my asking, where is it not mentioned that they are reserved for doctors, he said that the boards are damaged and we would get new boards. He was extremely rude in his language.

He even showed me a Mercedes and said this is Mr. Trehan’s Mercedes and how can you park next to it. OMG !!

I asked his manager number & he told me that he is not in India, albeit he gave me the name Mr. Rajiv Kaushik and some email address, which proved to be invalid.

Respected Sir: After this experience, I am having serious thoughts of ever visiting Medanta again as the same doctors are available in nearby hospitals also. In any case, I am not going to park my car in Medanta compound.

I am extremely surprised that such a reputed hospital has such a poorly managed underground parking and if I may say so, some goons running the parking lot. (please note, I am only talking about underground parking. my experience with surface parking has been satisfactory & without any issues so far). Hope you would look into this matter.

Regards

PS

Price Discrimination in India

Likethatonly has a great article on Price discrimination in India. He articulates very well some of the abnomalies in pricing for instance:

Convenience charges – It is quite true that vendors save cost with customers doing the booking online instead of doing physical transaction, In India, most of the time, as of now, you end up paying higher if you do the booking online. This is the case for train bookings, movie tickets etc.

Higher pricing for driving lessons for women – as the perceived value to women drivers is higher and as, very right mentioned in the article, they would likely be coming from rich family – Many driving schools have higher pricing for driving lessons for women.

Other examples such as “Peak rate mandis” & different prices for foreigners  for tourist attractions are also very interesting.

The last one “language” is also very interesting that shows price discrimination based upon the language.