Using Cloudant with Node.js

By Max Thayer

Node.js is an open-source platform for writing JavaScript on the server, and has become my weapon of choice for new web apps. It's fast, asynchronous, and has a tremendous community that is only getting bigger. Because Cloudant indexes are in JavaScript, along with all client-side code, writing JavaScript on the server means my head never needs to switch gears. But perhaps most importantly, there are a ton of tools that make developing on CouchDB / Cloudant with Node.js effortless.

To get started with Node.js, download the binary for your operating system here. I highly recommend it :D

Packages: NPM

npm, or Node Package Manager, is a CouchApp that hosts packages for the Node.js community. It comes with Node.js, and you use it like this:

npm install [package]

Because npmjs.org is a CouchApp, you can replicate it and host your own registry using Cloudant. Specifically, replicate from https://registry.npmjs.org/ to your database, and bam, you have your own private registry. Then, you can push custom or private libraries to your registry and download them just like you would from npm.

Frameworks: Express and Express-Cloudant

Express is a web framework around Node.js that simplifies things like middleware and URL routing. It even comes with a starting template! To install, run npm install -g express; to scaffold a project, run express wherever you want to get started. The getting started guide serves as a fantastic introduction to Node.js itself, too.

Express-Cloudant is an extended Express template for working with Cloudant. Like Express, it stays out of your way, but comes with features for making life easy:

  • Built-in reverse proxy lets you query your Cloudant data from the client.
  • Custom API in routes/api.js using nano, exposing your database in a more controlled fashion.
  • Uses Grunt to manage static assets and design documents.
  • Manages design documents in the ddocs folder as JavaScript rather than raw JSON.

The project NoSQL-Listener is built using Express-Cloudant as a base.

Libraries: Nano and Cradle

Nano and Cradle are libraries for interacting with Cloudant and CouchDB. Here's an example of each:

Nano

Nano tries to be as out-of-your-way as possible, so it's very lightweight to use:

// require nano, point it at our instance's root
var nano = require('nano')('https://garbados.cloudant.com');
// create a database
nano.db.create('example');
// create an alias for working with that database
var example = nano.db.use('example');
// fetch the primary index
example.list(function(err, body){
  if (err) {
    // something went wrong!
    throw new Error(err);
  } else {
    // print all the documents in our database
    console.log(body);
  }
});

Nano has begun to support Cloudant-specific features like search, which makes it my library of choice for working with Cloudant from Node.js.

Cradle

Cradle is a more full-bodied library than Nano, with features like caching, and convenience methods to get and update documents. This usage example comes from its readme:

var cradle = require('cradle');
var db = new(cradle.Connection)().database('starwars');

db.get('vader', function (err, doc) {
    doc.name; // 'Darth Vader'
    assert.equal(doc.force, 'dark');
});

db.save('skywalker', {
    force: 'light',
    name: 'Luke Skywalker'
}, function (err, res) {
    if (err) {
        // Handle error
    } else {
        // Handle success
    }
});

CouchApps: node.couchapp.js

node.couchapp.js is a utility for writing CouchApps, like the Python CouchApp or Erlang Erica utility. I love it because you can use it to write design docs as either a series of separate files across multiple folders, or as a single JavaScript file like this:

var couchapp = require('couchapp')
  , path = require('path');

ddoc = {
    _id: '_design/app'
  , views: {}
  , lists: {}
  , shows: {} 
};

module.exports = ddoc;

ddoc.views.byType = {
  map: function(doc) {
    emit(doc.type, null);
  },
  reduce: '_count'
};

ddoc.lists.people = function(head, req) {
  start({
    headers: {"Content-type": "text/html"}
  });
  send("<ul id='people'>\n");
  while(row = getRow()) {
    send("\t<li class='person name'>" + row.key + "</li>\n");
  }
  send("</ul>\n")
};

ddoc.shows.person = function(doc, req) {
  return {
    headers: {"Content-type": "text/html"},
    body: "<h1 id='person' class='name'>" + doc.name + "</h1>\n"
  };
};

ddoc.validate_doc_update = function (newDoc, oldDoc, userCtx) {
  function require(field, message) {
    message = message || "Document must have a " + field;
    if (!newDoc[field]) throw({forbidden : message});
  };

  if (newDoc.type == "person") {
    require("name");
  }
};

couchapp.loadAttachments(ddoc, path.join(__dirname, '_attachments'));

Nifty, eh?

Check out these CouchApps built with node.couchapp.js as examples:

  • Egg Chair: like Pinterest and Flickr, but without the terms and conditions.
  • Chaise Blog: A CouchApp blog, using two databases and filtered replication to share only what you want.

Workflow: Yeoman, Grunt, Bower

Yeoman, Grunt, and Bower are the backbone of a modern JavaScript workflow:

  • Yeoman scaffolds new projects
  • Grunt automates their build, test, and deploy processes
  • Bower retrieves client-side JavaScript libraries

All three of these tools have vibrant communities around them, generating plugins that do an increasing amount of your work for you. If you are using Make in your JavaScript projects, stop, and learn Grunt. Future-you will be thankful. Thanks to these tools, my workflow typically looks like this:

mkdir [app] && cd $_
yo [template]
// answer the template's questions
grunt

With only a few commands, my project is built, tested, and deployed. Here are some generators I use frequently at Cloudant:

And Grunt plugins:

  • grunt-couchapp automates pushing CouchApps, using node.couchapp.js, along with creating and deleting databases, using nano
  • grunt-couch: like grunt-couchapp, but with an interface more like the classic Python CouchApp utility.

Gotchas: Callbacks

When I was starting out with JavaScript, asynchronous programming threw me for a loop. Just like the JavaScript executing in your browser, Node.js is single-threaded and event-driven, which makes it wicked fast, but can feel strange coming from synchronous languages like Python.

For example, if you make an HTTP request, Node.js doesn't wait for it to finish. It creates an event to fire once you get a response, then continues executing your program while your request adventures about the interwebs. So, rather than getting a return value with the HTTP response, you give your request a function, called a callback, with instructions on how to handle the eventual response.

In Nano, for example, this code...

// get all our docs
db.list(function(err, body){
  if (err) {
    // something went wrong!
    throw new Error(err);
  } else {
    // print all the documents in our database
    console.log(body);
  }
});
// print hello world :D
console.log('hello, world!');

... will yield this:

> 'hello, world!'
> {"total_rows": 1, "offset": 0, "rows": [...]}

For more on asynchronous programming in Node.js, check out Control Flow in Node from Tim Caswell at How To Node.


As always, if you have any trouble, check our docs, post your question to StackOverflow, ping us on IRC, or if you'd like to discuss the matter in private, email us at support@cloudant.com.

Happy coding!

Create an account and try Cloudant yourself

Sign Up for Updates!

Recent Posts