A few weeks ago I was in the “post-holiday doldrums” and needed a lift out of my rut. I was missing Lisp (hadn’t written any Lisp code in two years); was curious about the details of the EC2 API protocol; and wanted a better environment for managing our Sierra instances that fit my style of working (I’m an old system-software guy that uses Emacs and Bash a lot). So I took a couple of days and wrote an EC2 API for Common Lisp, and then built a Sierra management interface on top of that. As it turned out, there were several open-source Common Lisp packages that helped: Edi Weitz’s drakma, a tidy web client; ironclad, an easy-to-use cryptography package; s-xml, a simple xml parser; and s-base64 for simple base64 encoding/decoding. Using these packages I had a functioning EC2 interface in half a day, and by the third day the package had everything I needed to build my Sierra layer on top. The decisive advantage this approach has over my previous python command-line interface is that my Sierra environment is running all the time (Lisp within an Emacs session); and thus I am able to interact with Sierra via the Lisp REPL., make changes without having to restart my Sierra environment, etc. Very cool.
Archive for January, 2010
Amazon EC2 Interface for Common Lisp
Friday, January 29th, 2010 by David Young
Tags: amazon, ec2, lisp
Posted in SaffronSierra | 3 Comments »
EBS-backed Sierra Instances
Monday, January 18th, 2010 by David Young
We’ve begun experimenting with the creation of Sierra AMIs using Amazon’s new EBS-backed AMI machinery, which allows one to store an AMI’s root device on an EBS volume rather than in bundled format on S3.
Right now, there seems to be little useful Amazon documentation on EBS-backed AMIs; the information necessary to attempt this experiment was gathered from several blog posts. Sierra is based on custom, internally built CentOS 5.3 AMIs; since none of the posts I found were CentOS-specific, some twiddling and guesswork was involved in getting a basic EBS Sierra instance started. However, this particular blog was pretty good: http://www.elastician.com/2009/12/creating-ebs-backed-ami-from-s3-backed.html.
Tags: amazon, ec2
Posted in SaffronSierra | No Comments »
Observing & Querying Tweets (Part 3)
Wednesday, January 13th, 2010 by Jared Peterson
In my previous posts I’ve discussed how to fetch data from Twitter and massage it into the form that is needed when observing data in Sierra. I also talked about how to “POST” that data to Sierra so that it’s available to query. In this post I’m going to discuss the process of actually querying that data once it’s in Sierra.
Before you read on you may want to browse over the Sierra documentation. These docs list all of the REST calls that are available on Sierra. In my example below I’ll be using the “connections” call.
I should also note that I’ve made some additions to the source code that I’ve used across all of these posts. That code is available along with the rest of the Sierra sample code. In particular I’ve made the code aware of command-line options that specify whether you are “observing” (-o) or “querying” (-q). You can also define all of the needed usernames and passwords via the command-line as well.
Tags: development, Groovy, json, REST, SaffronSierra, twitter
Posted in SaffronSierra | 1 Comment »
Ingesting Data into Sierra
Monday, January 11th, 2010 by Chet Patel
After signing up for a Sierra account, a few simple steps can be taken to push your data into Sierra and then analyze your data.
First, you’ll need to grab our sample code. You can find the instructions for downloading our sample code here.
Once you have downloaded the sample code, look for the Ingest.java class (in the src/java directory). This is a command line tool that can be run from your JAVA IDE or as a compiled java application.
Tags: development, REST, SaffronSierra
Posted in SaffronSierra | No Comments »
Even a Congressman knew SQL wouldn’t connect the dots of the Christmas Terror
Friday, January 1st, 2010 by Manny Aparicio
It is not easy to change things. It is hard enough to build a new thing and then even harder to evangelize a new market. I am encouraged by the growing understanding and need for a new analytic store – beyond databases, but as I watch the aftermath of the Christmas Day bombing attempt, a radical and more rapid shift is now a matter of survival. The intelligence community has been evolutionary rather than revolutionary in its approach to technology since 9/11. The data stores and rule-based processing are “so twentieth century”, using vintage 1980 technology. The use of analytic visualization has grown but is still based on the same old data stores and manually-intensive connecting of “dots”, one by one. We are still addressing the war on terrorism with evolutionary change rather than adopting revolutionary new technologies to address a radical new enemy.
Tags: analogies, national security, object orientation
Posted in Natural Intelligence | No Comments »
