CURL

[Rajinder Yadav]

Thanks Myles,

no kidding those rfc docs are dry reading and I am not writing a http
server so way too much info =)

I came across the man page for cURL but was actually looking for a
tutorial on how to use all the various switches from the command line
for http analysis. I manage to figure out that ones I needed to
pull-down web pages using http get. Still the power/complexity or curl
is something I would love to tame one day.

I was actually looking to write a data scraper / bot in Ruby, there
were a few gems that interface with curl and I wanted to use that
since curl is very fast and stable, but then I managed to find a ruby
gem that made writing bots a cakewalk.

Right now I got a twitter bot sucking down posts, and it should be
easy to write code to tweet from my account using ruby. The next phase
will be to start scraping my followers list and then other people
followers list and then really start having some fun. I've got some
cool ideas for a twitter bot, you know the ones that listen to post to
reply based on certain keyword, that will be my first pet project for
xmas break.

-- 
Kind Regards,
Rajinder Yadav

http://DevMentor.org

Do Good! - Share Freely, Enrich and Empower people to Transform their lives.


On Mon, Dec 14, 2009 at 8:49 AM, Myles Braithwaite
<me-qIX3qoPyADtH8hdXm2+x1laTQe2KTcn/@public.gmane.org> wrote:
> curl man page[1] is actually really good. But if you don't know basic
> HTTP protocol it is really complicated. Take a look at the HTTP
> Protocol RFC 2616[2] which provides all the information you will ever
> need.
>
> A good place to learn about cookie's is the Wikipedia page[3].
>
> Also if you don't like the command line check out hurl[4] and it's source[5].
>
> I provide a basic guide to interfacing curl with Apache CouchDB in my
> slides from Ontario Linux Fest[6].
>
> Sorry I couldn't provide anymore information. The HTTP protocol is
> really complex and learning it will require reading a lot of RFCs.
>
> ... [1]: http://curl.haxx.se/docs/manpage.html
> ... [2]: http://j.mp/rfc2616
> ... [3]: http://en.wikipedia.org/wiki/HTTP_cookie#Setting_a_cookie
> ... [4]: http://hurl.it/
> ... [5]: http://github.com/defunkt/hurl
> ... [6]: http://www.slideshare.net/mylesb/apache-couchdb-talk-at-ontario-gnu-linux-fest
>
> On Sun, Dec 13, 2009 at 8:13 PM, Rajinder Yadav <devguy.ca-Re5JQEeQqe8AvxtiuMwx3w at public.gmane.org> wrote:
>> Does anyone know of a good curl tutorial? I want to learn the basic of this
>> tool as well as basic http concepts.
>>
>>
>> --
>> Kind Regards,
>> Rajinder Yadav
>>
>> http://DevMentor.org
>>
>> Do Good! - Share Freely
>> --

> --
> Myles Braithwaite
> me-qIX3qoPyADtH8hdXm2+x1laTQe2KTcn/@public.gmane.org
> http://mylesbraithwaite.com/
--
The Toronto Linux Users Group.      Meetings: http://gtalug.org/
TLUG requests: Linux topics, No HTML, wrap text below 80 columns
How to UNSUBSCRIBE: http://gtalug.org/wiki/Mailing_lists