Observing & Querying Tweets (Part 2)

In my last post I walked through grabbing tweets from Twitter using their API. We then took those tweets and built the necessary XML “resource” for pushing that data into Saffron Sierra. In this post I’m going to cover the process of actually getting the data into (observing) Sierra.

To start, I should probably cover a minor modification I made to the code from the last post as I worked through this post. Last time, I created one XML resource for Sierra that contained all of the tweets from my timeline. After some testing, and thinking, I decided to put each tweet into it’s own separate XML resource. I think this is cleaner, and it’s the “supported” path for Sierra anyway.

All of the code blocks mentioned here are part of a single Groovy file named “Twitter.groovy”. I’ve added this to our Sierra sample code which lives in a publicly accessible Subversion repository. Feel free to go grab it, or you can look at it here.

At the end of my last post we had an XML document that was ready to post to Sierra. After my modifications mentioned above we now have a set of XML documents ready to post. The first thing we need to do is generate a properly formatted URL for our POST call. Sierra uses an access key and secret code combination to authenticate API calls. The following is the groovy code for building the URL.

1
2
3
4

// build the Sierra URL with the proper signature
def resourcePostUrl = "http://${sierraHostname}:8080/ws/spaces/default/resources?access_key=${sierraAccessKey}"
def urlToEncode = "${resourcePostUrl}${sierraSecretCode}"
resourcePostUrl = "${resourcePostUrl}&signature=${Hex.encodeHex(MessageDigest.getInstance("SHA-1").digest(urlToEncode.getBytes("UTF-8")))}"

Once we have the URL we need to iterate over our set of tweet XML documents and POST them to Sierra. The following code handles that.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

// setup the HTTPBuilder call
def sierraHttp = new HTTPBuilder(resourcePostUrl)
 
tweetDocs.each {tweetDoc ->
    // debug
    println tweetDoc.toString()
 
    // Base-64 encode the twitter doc
    def tweetDocBase64 = tweetDoc.toString().bytes.encodeBase64()
 
    // Build the xml need to post to Sierra
    def sierraResourceDoc = xml.bind {
        resource {
            action('add')
            type('Transient')
            template('WS XML Raw')
            data(tweetDocBase64)
            flush('true')
        }
    }
 
    // POST the resource to sierra
    sierraHttp.post(body: sierraResourceDoc.toString(), requestContentType: groovyx.net.http.ContentType.XML) {resp ->
        // debug
        println resp.statusLine
    }
}

Let’s walk through this code in a bit more detail. On line 2 we’re creating an HTTPBuilder instance using our URL that we’ll use to post the resources. Then, on line 9, we generate a base64 encoded string for our twitter XML document. Next, on line 12, we generate another simple XML document that wraps our base64 encode twitter XML document, and adds a few other elements which represent instructions for Sierra. You can read more about the requirements for that XML document here. Finally, we POST all of this to our Sierra instance. That’s it. Done. Twitter data in Sierra ready for us to query.

The querying part will come in my next post. Stay tuned!

Tags: development, Groovy, REST, SaffronSierra, twitter

This entry was posted on Monday, December 28th, 2009 at 9:49 am and is filed under SaffronSierra. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.