?

Log in

fanf

Simple shell scripting for Twitter

« previous entry | next entry »
22nd Aug 2010 | 23:56

I have a few scripts which manage my URL log, including posting a copy of the feed to my Twitter and del.icio.us accounts. Until recently the scripts have just used wget's HTTP Basic Auth support to authenticate to my accounts. This has to change because Twitter is switching to oauth. This switch has already been delayed but is now due to occur by the end of August.

Most oauth implementations are not designed for old school languages like the Unix shell, so I procrastinated because I didn't fancy dealing with the dependency hell of mainstream scripting languages. But I perked up when I noticed Jef Poskanzer mentioning his stand-alone oauth implementation on his twitter feed. I have liked Jef's approach to writing simple code since I worked on thttpd in support of Demon's homepages service.

Posting to Twitter with basic auth didn't require anything beyond a Twitter account. You could just POST to https://twitter.com/statuses/update.json - which is the essence of the security problem that the Twitter crew want to solve.

To use oauth you must register an application. Go to https://dev.twitter.com/ -> Get started -> Your apps -> Register a new app. Fill in the form. Tell it the app is a client app and give it read+write permission.

When you have done that you will be presented with your application's settings page. The interesting parts are the "consumer key" and the "consumer secret" which are the half of your app's oauth credentials which authenticate the application to Twitter. The other half of the credentials prove that your app may do something to a particular person's Twitter account. In order to obtain the second half oauth normally requires a fairly complicated dance. Happily, for simple cases where you want to script your own account, Twitter provides a shortcut.

On your application's settings page, click the "My Access Token" link. This gives you a page containing your "access token" and "access token secret" which together with your "consumer key" and "consumer secret" allow your app to post to your Twitter account.

Now you need some software. Fetch Jef Poskanzer's oauth_sign and http_post packages. You can use oauth_sign with wget or curl, but http_post is more convenient since both it and oauth_sign encode the query parameters for you, whereas wget and curl do not. Typing make should be enough to build oauth_sign; for http_post you probably want make SSL_DEFS="-DUSE_SSL" SSL_LIBS="-lssl -lcrypto".

Now you have everything you need to write a simple shell twitter client. Something like:

    #!/bin/sh

    consumer_key="COPY-FROM-APP-SETTINGS-PAGE"
    consumer_secret="COPY-FROM-APP-SETTINGS-PAGE"
    access_token="COPY-FROM-MY-ACCESS-TOKEN-PAGE"
    access_secret="COPY-FROM-MY-ACCESS-TOKEN-PAGE"
    url="https://api.twitter.com/1.1/statuses/update.json"

    http_post -h Authorization "$(oauth_sign \
	$consumer_key $consumer_secret \
	$access_token $access_secret \
	POST "$url" status="$*")" \
	     "$url" status="$*"

That should be enough to get you going.

For completeness (and since I worked out how to do it I'm going to inflict it on you) here's how to use Jef's tools to do the full three party oauth procedure.

  • First obtain a request oauth token and secret. For this transaction your oauth token and secret are empty. The HTTP request is a POST with an empty body.
    http_post -h Authorization "$(oauth_sign \
        $consumer_key $consumer_secret "" "" \
        POST https://api.twitter.com/oauth/request_token)" \
             https://api.twitter.com/oauth/request_token
    
  • The response will contain oauth_token and oauth_token_secret parameters which are your request token and secret. You must then get your victim to visit the URL https://api.twitter.com/oauth/authorize?oauth_token=$request_token. They will be asked to give your app permission to diddle with their account. (They may have to log in first.)
  • When they have done this they will be redirected to your app's callback URL, with some extra parameters. The documentation says these will be a copy of the request oauth_token and an oauth_verifier but in my testing the verifier was missing.
  • If your app is a client app (which has no callback URL or the URL is "oob") then the user will be presented with a PIN which is the request verifier.
  • You can now obtain the access token and secret as follows.
    http_post -h Authorization "$(oauth_sign \
        $consumer_key $consumer_secret \
        $request_token $request_secret \
        POST https://api.twitter.com/oauth/access_token oauth_verifier="$verifier")" \
             https://api.twitter.com/oauth/access_token oauth_verifier="$verifier"
    
  • The response will contain oauth_token and oauth_token_secret parameters which are the access token and secret that you must use when your app wants to do something with your victim's account.

I hope this might be of use to someone else...

| Leave a comment | Share

Comments {19}

from: anonymous
date: 22nd Aug 2010 23:14 (UTC)

Hm. Technically impressive, I'm sure, but is auto-posting a log of your links really a good use of Twitter?

It certainly doesn't make for interesting reading... if I were Twitter I'd ban all auto-posting, but as it is I'll just have to not follow anyone who excessively auto-posts.

S.

Reply | Thread

Tony Finch

from: fanf
date: 22nd Aug 2010 23:26 (UTC)

I like Twitter because it is a good way to find interesting stuff to read via links posted by other people, so I think it's worth contributing my interesting links in a similar way. I have enough followers and retweets to make me think this is a worthwhile use of the medium.

Reply | Parent | Thread

Ross

from: crazyscot
date: 23rd Aug 2010 07:42 (UTC)

Interesting links are a much better use of Twitter than a lot of the random noise on it (though personally I read fanf's by RSS).

Reply | Parent | Thread

pndc

from: pndc
date: 23rd Aug 2010 08:15 (UTC)

They're random *interesting* links. Which is a good use of Twitter in my book, because I get a constant dribble of useful bits of CS research that I wasn't aware of. Given I'm a degree-less oik blagging a career in this industry, it's quite handy.

Unless of course a "good use of Twitter" is self-absorbed witterings about irrelevant trivia, like the rest of Twitter.

Reply | Parent | Thread

from: anonymous
date: 23rd Aug 2010 08:53 (UTC)

Nah, for self-absorbed wittering thinking of Live Journal.

A good use of Twitter is one hundred and forty characters of pure comedy gold.

S.

Reply | Parent | Thread

Tony Finch

from: fanf
date: 23rd Aug 2010 10:18 (UTC)

If you want @serafinowicz you know where to find him :-)

Reply | Parent | Thread

Murk

from: murkee
date: 4th Sep 2010 07:42 (UTC)

I'm getting

usage: /Users/Mark/geektool/http_post/http_post [-c cookie] [-t timeout] [-r referer] [-u user-agent] [-a username:password] [-h header value] [-v] url
./twitteroauth: line 13: https://api.twitter.com/1/statuses/update.json: No such file or directory


(The path is as I haven't updated the path search)


I've tried tweaking a few things (including adding a \ between these lines to link them:

POST "$url" status="$*")"
"$url" status="$*"

... but no joy.

I have updated all the variables at the top of the file.

Reply | Thread

Murk

from: murkee
date: 4th Sep 2010 07:44 (UTC)

(What I'm trying to do is grab the twitter feed for geektool, so I can then embed it in my desktop)

Reply | Parent | Thread

feed for geektool

from: anonymous
date: 8th Sep 2010 10:21 (UTC)

Anyone else having trouble doing this since Twitter switched to OAuth?

Reply | Parent | Thread

Tony Finch

from: fanf
date: 6th Sep 2010 19:24 (UTC)

Yes, you have identified a missing backslash. Thanks. I have fixed it.

I'm not sure why it isn't working for you even after that fix. Make sure you don't have any typos in the variable names or any extra spaces in the assignments.

Note that you don't need authentication to fetch a user's feed unless they have hidden their tweets. E.g. http_get http://api.twitter.com/1/statuses/user_timeline.json?screen_name=geektool

Reply | Parent | Thread

Murk

from: murkee
date: 6th Sep 2010 19:26 (UTC)

can't see twitter from one of my common locations - so want script on machine I can see to get my full feed for me (including those folks who I can see as logged in user) :)

Reply | Parent | Thread

Murk

from: murkee
date: 6th Sep 2010 19:27 (UTC)

....as well as the embedding in desktop (same script, two applications)

Reply | Parent | Thread

Murk

from: murkee
date: 8th Sep 2010 06:00 (UTC)

http_get not found ....

.... also, there's nothing I can see in http://api.twitter.com/1/statuses/user_timeline.json?screen_name=geektool that specifies which user feed to grab...?

It's all very confusing.

Reply | Parent | Thread

Tony Finch

from: fanf
date: 8th Sep 2010 19:03 (UTC)

http_get is another of Jef's little tools.

That URL is the feed of the twitter user geektool. Perhaps I misunderstood your comment about geektool earlier...

Reply | Parent | Thread

Thanks For The Info

from: anonymous
date: 7th Sep 2010 23:49 (UTC)

Very informative and accurate. I'm really busy right now and don't have the time to sort this out. Thankfully, you made it easy.

Reply | Thread

pingback_bot

следить отовсюду

from: pingback_bot
date: 22nd Sep 2010 07:08 (UTC)

User dsjkvf referenced to your post from следить отовсюду saying: [...] немного [...]

Reply | Thread

Dosn't work with special char

from: anonymous
date: 4th Oct 2010 14:55 (UTC)

Hello,

First i would like to say thank you for this post..

After some test, it seems that the script does not work with special characters.

For example :

tweet "Some text like this à ^$"

And the json associated :

{"request":"/1/statuses/update.json","error":"Incorrect signature"}

Could you help me please ?

Reply | Thread

Tony Finch

Re: Dosn't work with special char

from: fanf
date: 22nd Oct 2010 15:52 (UTC)

There is a bug in http_post which the following patch fixes:

--- http_post.c~	2010-06-20 03:20:25.000000000 +0100
+++ http_post.c	2010-10-22 16:49:09.000000000 +0100
@@ -709,7 +709,7 @@
 	    }
 	else
 	    {
-	    (void) sprintf( to, "%c%02x", '%', *from );
+	    (void) sprintf( to, "%c%02x", '%', *from & 0xff );
 	    to += 3;
 	    tolen += 3;
 	    }

Reply | Parent | Thread

Jef Poskanzer

Re: Dosn't work with special char

from: jef_poskanzer
date: 18th Jan 2011 18:22 (UTC)

FYI I got around to storing out a new version of http_post with this fix. Curiously I have three other programs that include the same routine and they all had the same fix already. I'm usually better about keeping code in sync.

Reply | Parent | Thread