Books, Cats, Tech

"Everything that moves serves to interest and amuse a cat." — F.A. Paradis Moncrif

Vicki Brown

My home on the WWW
Est: 1994

Email: vlb@cfcl.com
Home

More About Me

Lifestream

  • Background
  • How I Built My Twitter Reader
  • Install and Configure TTYtter
  • Get the Tweets
  • TWikification of Output
  • Wrap It Up (Packaging)
  • The Code

Building a Twitter Reader

Using TTYtter, Perl, and TWiki

DISCLAIMER - Before you read this...

If you want to implement what I have here, you will need a computer capable of running Perl and the Unix cron command.

Capable systems include Linux, BSD, Mac OS X, and Windows (if properly configured, probably with cygwin installed). For any other system, your mileage will vary widely. I have no idea if Windows has something approximating cron.

I use Mac OS X, which is based on BSD Unix. I cannot help you with any other system!

Background

One of the features of Twitter is that it runs 24/7/365. (Un)fortunately, I don't. So, I miss things. I didn't want to miss things, so I looked for a solution.

Being a programmer myself, I wanted a solution I could control and tweak if necessary. However, I didn't want to write something from scratch if I didn't have to!

Twitter has a popular, published API, so I figured someone would have written what I wanted. Someone did.

The Front End

I initially found pyTwerp (written in Python). I'm not a Python programmer, but pyTwerp looked like it does pretty much everything I needed, out-of-the-box. And it did... up till September, 2010 when Twitter went to OAuth only and pyTwerp stopped working.

TTYtter

I looked around a little more and this time I found TTYtter. TTYtter is written in Perl (a plus, given that I know Perl (and I don't know Python)). More important, TTYtter understands OAuth.

TWiki

I could easily read the TTYtter output with any text editor, or as plain .txt files in my web browser. But I wanted something that looked a little nicer. Enter TWiki.

TWiki is a structured enterprise wiki with a lot of programmable features. By using TWiki, I don't need to convert the output to html in order to read it on the web and I get the following useful features:

  • Blank lines are retained as "paragraph" breaks.
  • Anything that begins with http:// is rendered as a clickable link.
  • *word* is converted to bold; _word_ is converted to italic.
  • I can "script" the results, using TWiki to hide everything I'm not currently reading, or only show me this month's output.
  • I can easily adjust the look and feel with CSS.

How I Built My Twitter Reader

Steps

  • Install and Configure TTYtter
  • Get the Tweets
  • Filter the Tweets
  • Format For TWiki
  • Wrap it up (Packaging)

Install and Configure TTYtter

Start by downloading TTYtter from Floodgap Software.

You will also need Perl 5.005 or better (5.6 and up are recommended) and cURL. Nothing else is necessary

Installation and COnfiguration

To install TTYtter, download it to the location you want to run it from (such as your home directory or bin/), make it executable with chmod +x, and then start it from your shell prompt with something like ./ttytter. If your Perl is not in /usr/bin, change the first line.

When you start TTYtter without a keyfile, which will be the case the first time you run it, it will automatically start an assistant to help you create the keyfile. You only need to create the keyfile once for each account. It will never expire.

Full instructions with examples can be found at the TTYtter home page. .

Get the Tweets

First I used
   ttytter  -runcommand="/r" 

but I didn't like the output format. So I asked for assistance and created an "extension" that provides the output format I wanted:

$handle = sub {
    my $ref = shift; # this is a hashref to the tweet structure
    my $sn = &descape($ref->{'user'}->{'screen_name'});
    my $string = ${sn} . '  ' .
            &descape($ref->{'text'}) . 
            ' -- ' .  $ref->{'user'}->{'protected'} . 
            ' ' .  "[[Twitter:${sn}/statuses/$ref->{'id'}][view]] " .
            "DATE=" . &descape($ref->{'created_at'}) . 
            "\n_EOTWEET_\n";
    print $streamout $string;
    return 1; # one logical tweet accepted
};
Now I can run
ttytter -exts=/home/vlb/ttytter/vlbext.pl -runcommand="/r" 

Filter the Tweets

Next, I wrote a small shell/Perl script I named gottytter. This does some housekeeping tasks and runs ttytter. It also time-stamps the output every 30 minutes. (Full gottytter code can be seen at the end of this article.)

Prettification With CSS

I made a few tweaks to the look of the output, increasing the font size slightly, highlighting italic (<em>) in green, and setting color classes so that different senders are highlighted in different colors. (The latter idea came from using Adium's "Minimal_mod" message style with the "Sender Color" variant.)

CSS Code

Wrap It Up (Packaging)

Next I wrote a TWiki application that lets me search through output pages and read the latest tweets.

Then I set gottytter to run every 5 minutes, sending its results to text files in my TWiki directory.

    0,10,20,30,40,50 * * * *  $HOME/bin/gottytter
And here we are: TwitterReader.

The Code

CSS

  • twitterstyles.css

  • Show CSS Code in place
    #patternOuter {
       margin-left:0;
    }
    #patternLeftBar {
        display:none;
    }
    /* for the content left margin use a bit smaller margin */
    #patternMainContents {
        padding-left:2em; /*S6*/
    }
    .preElement
    {
     white-space: pre; /* Camino */
     white-space: -moz-pre-wrap;
     white-space: pre-wrap;  /* Firefox 3 */
     width:800px;
     font-size: large;
     line-height: 120%;
    }
    .preElement em {
      color: #106b71;
    }
    .tweet {
      font-size:larger;
      line-height:110%;
    }
    .tweet em {
      color: #666;
    }
    .tweet strong em {
      color: #000000;
    }
    .tweet h3 {
      font-size:large;
    b.p0 {color: #000000;}
    b.p1 {color: #990033;}
    b.p2 {color: #3333FF;}
    b.p3 {color: #009966;}
    b.c0 { color: #000000; }
    b.c1 { color: #000033; }
    b.c2 { color: #000066; }
    b.c3 { color: #000099; }
    
    

gottytter

  • gottytter.pl
    Remember to change vlb or Vicki if you copy the code for yourself.

Twitter Reader

  • View raw TWiki code

  • Show Code for Twitter Reader in place  

    Remember to change vlb to your own Twitter handle!

    <noautolink>
    <!---
       * Set TODAY = %SERVERTIME{"$year-$month-$day"}%
       * Set WHEN =  %URLPARAM{"when" default="%TODAY%"}%
       * Set PAGETITLE = %TOPIC% - %WHEN%
    -->
    ---+!!  Twitter Reader (@vlb)
    
    [[TwitterReaderWhat][What is this?]]
    
    <table cellspacing=20><tr><td valign="top" width="60%">
    <form action="%SCRIPTURL{"view"}%/Vicki/Twitter/TwitterReader" >
      <input type="text" name="when"  size="20" value="%WHEN%" id="datecell" \
      class="twikiEditFormTextField" /><input type="image" name="calendar"  \
      src="%PUBURL%/TWiki/JSCalendarContrib/img.gif" align=MIDDLE alt="Calendar" \
      onclick="return showCalendar('datecell', '%Y-%b-%d')" /> 
    <br />
      <input type="submit" value="Show Me">
    </form>
    
    *Twitter Logs for [[http://twitter.com/vlb][@vlb]] from [[TwitterLog%WHEN%][%WHEN%]]*
    
    %TWISTY{showlink="Contents" start="hide"}%
    %TOC%
    %ENDTWISTY%
    
    </td><td valign="top">
    %CALENDAR{daynames="Mo|Tu|We|Th|Fr|Sa|Su" showweekdayheaders="1" width="200" gmtoffset="-8"}%
    </td></tr></table>
    <!--
    <!input type="button" value="Embiggen" onclick='Embiggen={"runNow":true}; var s=document.createElement("script"); s.type="text/javascript"; s.src="http://ghill.customer.netspace.net.au/embiggen/embiggen.js"; document.getElementsByTagName("head")[0].appendChild(s);'><!/input>
    -->
    
    #AnchorTop
    
    <table width="900">
    <tr><td>
    <div class="tweet">
    
    %INCLUDE{"TwitterLog%WHEN%"}%
    
    </div>
    
    </td></tr>
    </table>
    
    [[#AnchorTop][Top]]
    
    </noautolink>
    
    <!-- JSCALENDAR -->
    %INCLUDE{"%TWIKIWEB%/JSCalendarContribInline"}%
    
    <link rel="stylesheet" href="%PUBURL%/%WEB%/WebHome/twitterstyles.css" type="text/css" media="all" />
    
    <!--
       * #Set COVER = print
    -->