A Bit More About the New Joyent Cloud API

As part of SmartDataCenter 6.5, we've introduced something I'm pretty excited about sharing, which is an open signing protocol for HTTP. We're using it in what we call CloudAPI, which is the REST API that powers our customer portal, and is exposed to you to create your own applications. In this post I'll go over what the signing protocol is and why you should care, how SmartDataCenter is using it, and some ideas we have in mind for the future.

If you're not an RFC nerd, HTTP defines a "pluggable" framework for authentication so that any number of authentication mechanisms can be used, and goes on to define Basic Authentication. Basic Authentication is nothing more than sending your username and password across the wire, so it's not ideal from a security perspective. Following Basic Authentication, HTTP also goes on to define Digest Authentication, which involves sending a hashed form of your password across the wire, but it's hard to use and doesn't see much use these days.

However, for the best security properties, you really want to be leveraging public key cryptography to authenticate. HTTPS (well, really SSL) has a mode of allowing this called "client-auth"; basically your client has a certificate and private key, and in addition to validating the server's certificate, the server asks you (the client) to present your certificate. Sounds great, and does do what we want from a security perspective. However, it's plagued by poor usability in browsers, is more or less a programming nightmare (especially with "custom" CAs), and can incur significant server-side cost, since it's usually a load balancer (i.e., HTTPS termination point) that is handling the cryptography validation. All of this means that we want the benefits of public key crypto without the burden of client-auth SSL.

To solve this problem with SmartDataCenter, we're introducing the HTTP Signature authentication scheme, which is already released as open source. The HTTP Signature scheme allows for digital signatures to be leveraged as your credentials, and does not imply any particular key management scheme. It should be suitable for just about any REST API out there, so we're certainly hoping if you're an API vendor you take a look at it, but rather than jump into the technical details of the spec itself, I thought it would be interesting to show you some CLI candy, so lets walk through some examples.

Some Signing Candy

First, assuming you're a node.js user (and if you're not, you should be!), use npm to install the smartdc toolkit:

$ npm install smartdc -g

The smartdc module installs the CloudAPI SDK for node.js, which includes a fullcommand line interface (CLI), and we're going to show off some of its features here.

To "configure" your CLI, just run the setup command:

$ sdc-setup https://us-west-1.api.joyentcloud.comUsername (login): (james) jamesPassword:The following keys exist in SmartDataCenter: [1] id_rsaWould you like to use an existing key? (yes) yesIf you set these environment variables, your life will be easier:export SDC_CLI_URL=https://us-west-1.api.joyentcloud.comexport SDC_CLI_ACCOUNT=jamesexport SDC_CLI_KEY_ID=id_rsaYou'll additionally want to set SDC_CLI_IDENTITY to the full path location of the sshkey on your system (try export SDC_CLI_IDENTITY=/Users/james/.ssh/id_rsa).

In my case, sdc-setup found an existing ssh key in SmartDataCenter, and I simplyselected that; if I didn't already have one "on record" sdc-setupwould have asked me to upload a new one. At any rate, sdc-setup gave me someenvironment variables to set, so I just put them in my ~/.bashrc file(sorry Windows users, smartdc's CLI isn't there yet).

Note that there's no more use of your password, and assuming you did set up thoseenvironment variables then you can simply run:

$ sdc-listmachines[{"id": "30e8d834-7210-44f7-be58-7179396bfe20","name": "foo","type": "smartmachine","state": "running","dataset": "sdc:sdc:smartos:1.3.18","ips": [""],"memory": 128,"disk": 5120,"metadata": {},"created": "2011-09-15T04:12:53+00:00","updated": "2011-09-15T04:14:35+00:00"}]

The CLI just authenticated to SmartDataCenter using my SSH key. That meansfor interacting with SmartDataCenter you don't need to maintain any "extra"keys; in my case I can then go ahead and do this:

$ ssh -i ~/.ssh/id_rsa admin@ login: Thu Sep 15 04:17:52 2011 from[admin@foo ~]$ exit

While not technically part of the CLI, check this out:

$ sdc-createkey ~/.ssh/id_dsa.pub{"name": "id_dsa.pub","key": "ssh-dss AAAAB3Nza... james@Howl.local","created": "2011-09-15T04:19:10+00:00","updated": "2011-09-15T04:19:10+00:00"}$ ssh -i ~/.ssh/id_dsa admin@ login: Thu Sep 15 04:18:05 2011 from[admin@foo ~]$ exit

Oh yeah, SmartDataCenter has "live" key rotation of your SSH keys in runningSmartMachines. Finally for the "candy" section, here's "one more thing":

$ ssh-addIdentity added: /Users/james/.ssh/id_rsa (/Users/james/.ssh/id_rsa)Identity added: /Users/james/.ssh/id_dsa (/Users/james/.ssh/id_dsa)$ ssh admin@ login: Thu Sep 15 04:19:13 2011 from[admin@foo ~]$ export SDC_CLI_URL=https://us-west-1.api.joyentcloud.com[admin@foo ~]$ export SDC_CLI_ACCOUNT=mcavage[admin@foo ~]$ export SDC_CLI_KEY_ID=id_rsa[admin@foo ~]$ sdc-listmachines[{"id": "30e8d834-7210-44f7-be58-7179396bfe20","name": "foo","type": "smartmachine","state": "running","dataset": "sdc:sdc:smartos:1.3.18","ips": [""],"memory": 128,"disk": 5120,"metadata": {},"created": "2011-09-15T04:12:53+00:00","updated": "2011-09-15T04:14:35+00:00"}]

Wow! So what we just did is use the OpenSSH ssh-agent program to stash our keyson your local host and the smartdc CLI is slick enough to interact withthat (if it's available) instead of looking for a private key on the filesystem.Using that mechanism, you can actually SSH from box to box, without worryingabout copying your keys around. Now you've actually got single sign on (SSO)happening between your hosts and SSO to the CloudAPI.

Some Technical Details

HTTP Signature is actually really straightforward, and if you've ever lookedat OAuth, or WRAP, et al, this should be both familiar and much simpler. It'sas simple as pointing out that you just have to sign the HTTP Date header, whichaccomplishes a few things:

  • We can securely authenticate you using your SSH private key.
  • Guarantees there at least some entropy in the string to be signed.
  • Allows us to prevent replay attacks on a coarse level by expiring requestsafter 5m.

Post written by joyentjames