Engineering an "Elastic Application"

It is great to run production applications in the cloud where one can easily provision additional servers when the need arises, and these servers can be released when they are no longer needed. Applications can be engineered to automatically trigger these changes in response to some defined situations (utilization, transaction rate, time of the day etc.,). Rules engines can be further refined to take cost into consideration when making these adjustments.

The economies of operation in the cloud are rapidly erased if the application requires a large number of servers (sized to meet peak or near peak demand, a.k.a excess capacity) to be operational at all times. It is therefore cost prohibitive to operate a production application if it does not make full use of the “On Demand” nature of resources in the cloud. Greg Arnette describes this in his recent blog post describing the new thinking that is required in order to build an application for cloud deployment, something he calls “gaming the cloud with elastic applications”. Greg writes that:

“A simple fact proven through real-world experience: “Net new” software stacks, designed with cloud operating principles firmly rooted in the core design, are a “must have” to take advantage of cloud computing economics and reliability.”

The term “Elastic Application” seems to best describe this new design paradigm.

Stateless systems are best suited to this “Elastic Application” paradigm, and the traditional web farm serving up static content is a very well understood example of this kind of a system. In response to increased demand, additional web servers can be provisioned and network load balancers can be used to route traffic evenly to all of the web servers. When demand decreases, some of the servers can be released.

Stateful systems are harder to model in this paradigm, and the traditional database is often the least elastic part of the application stack. An application is only as elastic as its least elastic component, and unfortunately, that dubious distinction most often goes to the data tier. As a direct consequence, the traditional relational database that is highly stateful is much harder to engineer in this paradigm. Relational Databases have therefore been more commonly associated with “scale-up” than “scale-out”. (See this article for a comparison of the two techniques for achieving scale).

Adrian Otto makes the case in a recent blog post that data driven applications must be architected in such a way that the data repository (traditional relational database or some other schema-free eventually consistent repository) is also elastic. In this article, Adrian writes,

“If you have an application that you want to deploy into a cloud, and you want it to be very elastic, you should think about the subject of how you arrange your data. If you use a centralized data design, you will probably have scalability bottlenecks when you add lots of servers. You should aim to decentralize the data in a way that you can easily add more servers to horizontally scale the environment, and not stumble on the limits of the database server.”

At ParElastic, we believe that the application developer should not have to worry about these kinds of things. The thought process described by Adrian above is strictly a result of the fact that the solutions available today are not well suited for cloud elastic applications, and addressing this huge shortcoming is exactly what ParElastic is all about.

ParElastic is middleware that provides elastic database scalability using multiple off-the-shelf relational database servers.

With ParElastic,

• you transparently scale out the data tier across multiple database servers while appearing as a single server to the application,
• applications can dynamically add database servers as storage needs grow without moving existing data or downtime,
• applications can add and remove query processing capacity on the fly to match variations in workload.

ParElastic makes the data tier in your application elastic. With ParElastic in the data tier, you can engineer a completely elastic application that can operate cost effectively in the cloud. It exposes to the application, a standards based interface and supports querying in standard SQL, further driving down application development and integration costs.

ParElastic’s Horizontal Elastic Scale-OutTM technology makes it possible for data driven applications to exploit the true OnDemand nature of the cloud, and makes it possible for you to bring the “Elastic Application” design paradigm to your data driven applications.