Starting with the Cloudant Accelerator Program: SpaceCraft, CouchDB & Redis

By Mike Broberg

The Cloudant Accelerator Program just launched, and we spoke with SpaceCraft, who came on board through the program. The Accelerator Program is designed to make it more affordable for startups to leapfrog multi-tenant Cloudant and launch their businesses on dedicated Cloudant infrastructure.

David Wood, CTO, SpaceCraft

SpaceCraft logo

SpaceCraft makes a CMS that’s designed for small businesses to easily publish content and ensure their websites look and work great across different browsers, TV platforms, and mobile devices. There’s no coding required, and their service comes with built-in SEO and unlimited cloud hosting. It’s the same mentality we have at Cloudant: simplifying the experience for our customers so they can #buildMore.

We are of like minds, SpaceCraft and Cloudant. SpaceCraft is down in Texas in Austin (rhymes with “Boston”). We’re both fans of CouchDB (relax). And they know our friends at Rackspace (we both already had servers there, who knew?). So we sat down with David Wood, CTO at SpaceCraft, to learn what else we had in common.


Cloudant: Why did you choose CouchDB?

David: We set out to build SpaceCraft two years ago, with the goal of developing a content management platform for small businesses to publish websites that render across a wide variety of form factors and devices of varying capabilities. We also wanted to ensure that in the future, content could be utilized for other channels, such as email campaigns, digital signage and even offline services. To enable this level of content reuse, we needed a strict separation of structured content and its presentation.

I've spent the better part of ten years doing CMS implementations and custom development, and the majority of systems I've worked on have used relational databases. We wanted our system to store structured data, and allow the user to control the structure. This could have been built using a relational database, but going the NoSQL route was the path of least resistance.

I used CouchDB previously and it mapped well to how we wanted to store data. If you think of a website, pages are essentially documents, so JSON documents are a natural fit. Our system stores content in smaller chunks, that we call blocks, and a page is a collection of blocks. From a storage standpoint, we're just dealing with collections of JSON documents. We knew all of our queries in advance and didn't have a need for ad-hoc queries, so CouchDB met our needs.

Another concern of ours that CouchDB helped solve was the isolation of customer data. We didn't want to store data for all our client sites together in one database, partitioned by a site ID. Instead, we wanted to isolate our client data into separate databases, with the ability to create them quickly as new sites are created on our platform, and CouchDB made this very easy. This approach allows us to remove databases easily when trial periods are over or customers delete their accounts. The ability to move individual databases between physical servers also provided for a crude, but effective, way to scale our data tier. Prior to moving to Cloudant, we were using replication, but only to ensure availability of customer data in the event of a hardware outage.

Cloudant: What was your architecture like?

David: SpaceCraft is a hosted solution and at a very high level our platform is comprised of five distinct application tiers, each of which can be scaled independently of each other.

SpaceCraft architecture diagram

Our Cloudant cluster is the core piece of the infrastructure upon which all other services rely. When it comes to fine-tuning our cluster's performance, we have a solid understanding of how each application tier impacts the read and write paths. Cloudant gives us the control to tune our cluster to ensure the performance we need. For example, we have sites on our platform that have experienced high spikes in traffic for several hours before returning to normal levels. To be able to scale up reads during that time is important. The MVCC [multiversion concurrency control] approach of CouchDB doesn’t require locking, so writes to the database during that time don’t block reads, giving us more flexibility to scale.

Cloudant: What was it like managing CouchDB yourself?

David: We have a small engineering team, with only three of us developing the product. CouchDB, as a running process, has been no maintenance for us. It’s actually the longest-running machine in our infrastructure and we’ve never had to restart the CouchDB process. We’ve migrated the machine once, when we rebalanced our capacity with Rackspace, but other than that, we’ve never had to touch it.

We’re resource constrained, and as developers, we’d rather work on the application than the IT maintenance side of the business. CouchDB’s reliability allowed us to do just that, and reduced the need for a team member dedicated solely to the care and feeding of our database.

Cloudant: Given your success with CouchDB, what led you to Cloudant?

David: SpaceCraft has been growing at a decent clip. We launched our product a year ago, and we have just shy of 1,000 customers. Leading up to Cloudant, we were growing at a rate where we were starting to exceed the capacity of our CouchDB servers. We ran into problems with virtualization and disk performance, that we were able to mitigate with SSD block storage on Rackspace. Right now we have 4,500 databases, and excluding images, are storing over 50 GB of data.

We really like the durability of CouchDB, and we wanted that same kind of ease when it comes to growth. We’re at the stage in our business where we’re exploring partnership deals that have the potential to rapidly add thousands of sites to our platform. As our customer base grows, our backend infrastructure will also need to scale to accommodate our increased needs. We’re very happy with CouchDB, and know it is rock-solid, but wanted to be in a situation where we knew it could grow to handle the volume we expect to be seeing. We also wanted to alleviate the need to have to manage the database -- at all.

Previously, we scaled CouchDB through sharding, bringing on different master-master peers. Cloudant’s Dynamo clustering handles scale much better. We also have a significant number of requests from our clients to provide full-text search capabilities. We evaluated providing this with Elasticsearch, but it’s nice to work with a database provider who already has that capability baked in with Lucene, and has it fully tested and supported so we can just start using it.

That’s what led us to Cloudant and the Accelerator Program. Very few vendors understand the situation many Web startups are in, but Cloudant definitely does. The program has been the perfect fit because we couldn’t have gotten the performance we needed on Cloudant’s multi-tenant offering. The Accelerator Program allows us to use the infrastructure we need to support our rapid growth, while managing our costs.

Cloudant: You mentioned traffic spikes for some of your customers. What’s that like, in terms of your backend architecture?

David: I can't give exact numbers, but we have had spikes in traffic that in a 24 hour period have resulted in millions of additional page views. In one case, a client site saw an increase in traffic in a 24-hour period that exceeded the previous 6 months.

The servers responsible for serving publishing sites can handle requests for any of the sites on our platform, so we can scale this tier by adding more servers. Published sites are generated using a compilation process that takes documents out of CouchDB and transforms them into HTML. The HTML is then cached in Redis to reduce the reads to our Cloudant cluster. The compilation process is fast, so for the most part we lazy-compile the pages when they are first requested. The only requests we serve directly out of Cloudant are attachments. Sites can be extended with custom JavaScript and CSS, and we store these source files as attachments.

Cloudant: So what’s next for your growth?

David: Based on the partnership deals we’re signing, we’re supporting larger volumes of traffic, and making adjustments to our platform to support this growth. We’re adding redundancy, both to our existing infrastructure and through additional data centers. At the core of all our planning is the need for a scalable database, which is why we chose to migrate to Cloudant sooner than later.

In addition to supporting increased traffic growth, we’re continuously adding new features and extending the functionality of the current features. We have a rapid release cycle, and on average, it takes less than two weeks for a new feature to be designed, developed, tested, and deployed. This year we’re looking to increase the size of our team to allow us to work on more in parallel.

One of the biggest motivations for moving to a service like Cloudant is that we can focus our developers on developing our product. We don’t have to spend time scaling database infrastructure. If a machine crashes, we don’t have to go out and fix it. We don’t have to constantly monitor database servers. Cloudant lets us get back to building our product.

Cloudant: Thanks!


We had a lot of fun talking to David. Thanks again to him and the team at SpaceCraft. They’ve had some great input on our product roadmap, and we loved hearing about their approach to caching in front of CouchDB and Cloudant.

If you’re a startup and want to learn more about the Accelerator Program that got SpaceCraft going on dedicated Cloudant service, check out https://cloudant.com/accelerator/ for more info.

-- Mike Broberg, marketing communications manager, Cloudant

Sign Up for Updates!

Recent Posts