Archive for the ‘Web Services’ Tag

SaaS Design Checklist

gears2I’ve been asked several times recently about the design considerations for an application that is to be delivered via the Software-as-a-Service (SaaS) model.  In other words, beyond the core functionality of the application itself, what other features or system aspects need to be addressed for the business offering of that application as a commercial service?

The following is a list of such features or aspects.  I make no claim as to the all-inclusiveness of this list, but it’s a good start.  Certain system aspects that apply broadly whether the application is SaaS or just critical-internal have been omitted (e.g., disaster recovery, health monitoring, etc.).  As for items listed, they may or may not apply to every situation, but they should at least be considered serious before deciding.

Security

  • Subscriber-Level Authentication & Authorization:  A “subscriber” is the entity with whom the business relationship exists for use of the SaaS application and comprises one or more users.  Each request must be authenticated to know the subscriber to which the user belongs thereby enabling license checks and usage metering.  Subscriber authorization comes into play if the application has separately licensable modules to which their users may or may not have access.
  • User-Level Authentication & Authorization:  As in all applications, each request must be authenticated to know the originating user and authorized to access specific capabilities by their role.  This authorization may be further constrained by subscriber-level authorization constraints.
  • Parametric Throttling:  A request may contain parameters that if unchecked could harm the system intentionally or otherwise.  For example, consider a request argument that dictates the number of records to return.  The application may protect itself from crippling values like 1,000,000,000 by simply throttling it to some configurable maximum like 500.  Throttling rules may need to be subscriber-specific.
  • Frequency Throttling:  Also important generally but particularly for APIs is the notion of throttling request rates (i.e., maximum hits per second from the same IP address) to prevent anything from abusively heavy subscribers to denial of service attacks.  This is often achieved within the network infrastructure as opposed to the application itself, but this is an opportunity for making the point that a successful SaaS deployment is about more than just the software engineering.

Service Level Agreements

  • Availability Monitoring:  SaaS contracts often carry SLAs that specify minimum application uptime.  When this is the case, a means for self-monitoring availability must be established whether to tout your success, to be the first to know about issues, or simply to address disputes.  Be specific about the very many ways to define uptime.
  • Performance Monitoring:  SLAs may also specify performance thresholds and require similar monitoring for similar reasons.  Individual performance data points should include the subscriber ID and request type to enable rollups in these dimensions since a) different subscribers may demand different SLAs, and b) different request types may have inherently different performance characteristics that can be called out separately with different thresholds.
  • Performance Exclusions:  Depending on the nature of the application or specific requests, there may be portions of execution time that should be excluded from performance calculations.  For example, the implementation of a request may call out to external services or execute a subscriber-specific workflow (i.e., things beyond the SaaS provider’s control).  Such activities may have been excluded from the performance SLAs and thus must be captured enabling the appropriate adjustments.
  • Compliance Auditing:  Collecting all supporting data is necessary, but not sufficient.  Reporting on this data for the purpose of auditing specific SLAs must be established and should be exercised internally to avoid surprises.

Subscription Servicing

  • Request Metering:  Requests incident on the application should be counted by subscriber ID and request type.  This enables usage monitoring by subscriber which may be required to support billing depending on the business relationship.  It also enables internal sensing of more heavily used features; information that can be useful in several ways (e.g., tuning, marketing, deprecation).
  • Subscriber-Level Reporting:  Separate from whatever reporting the application itself provides, there should be a means to generate summary information about a subscriber’s SaaS interaction whether periodically or on-demand.  This information may include usage levels, SLA compliance, license status, strange account activity if detectable, etc.  Minimally, the SaaS provider should be able to retrieve such information, but may also consider making it available to subscribers perhaps as an admin role capability.

External Services

  • Performance Monitoring:  Many applications integrate with externally provided services to perform portions of their request functionality (e.g., information retrieval, payment processing, etc.).  As the consumer of these services, the SaaS application should monitor their performance.  Whether or not the time spent waiting for these services to execute is included in formal SLAs, they will absolute impact user experience.  Downward trends may lead you to shop around for equivalent alternatives.
  • Availability Monitoring:  For all the same reasons, apparent availability of any external services should be tracked.  Apparent availability is the percentage of calls to a service that the service responded in a functionally meaningful way.

Resource Sharing

  • Multi-Tenancy:  A single infrastructure or slice thereof serving multiple subscribers is central to the SaaS economic model.  The most crucial aspect of this is the notion of a multi-tenant database schema.  The problems associated with isolating subscriber data logically rather than physically are easily offset by the maintenance benefits of dramatically reducing the number of production database instances.
  • Partitioning:  Economically, a single multi-tenant database may be ideal.  At scale, however, it may become necessary to have multiple databases each supporting a subset of subscribers.  This may be done to support different SLAs, to service very different usage patterns, to reduce the impact scope of an outage, or simply to handle high scale loads.
  • Selective Purging:  Even the best SaaS applications will lose subscribers.  Purging their data from a multi-tenant database is usually straightforward, but not so when it comes to backup media of multi-tenant databases.  If you’re entering into a contract that originates from the subscriber’s legal department, read the termination clause carefully and be sure it’s feasible.
  • Subscriber Portability:  If subscribers are partitioned across multiple databases, the need to move a subscriber from one instance to another will eventually arise as usage patterns change (the SaaS analog to rebalancing your 401k).  The biggest hurdle to this is avoiding ID collisions across databases.  The catch-22 is that this is rarely considered in release 1.0 and the downstream fix usually requires prohibitively invasive surgery.
  • Cross Partition Monitoring:  Partitioning subscribers across multiple databases or even whole infrastructure slices obviously adds to operational complexity.  As the number of partitions grows, consider some form of central monitoring hub to assist the Operations support staff.  This can start out simple and evolve over time as the ROI increases, but good sensors within the application can greatly facilitate this when the time comes.

Flexibility & Extensibility

  • UI Customization:  User interface flexibility can range from take-it-as-is to full white labeling per subscriber.  It can be as trivial as showing the subscriber’s logo or hacking up every form and navigation to comply with some internal process guidelines.  Ultimately the market will decide what level of customization capabilities are worth the engineering for a given application.
  • Data Model Customization:  Similarly, subscribers may have additional data fields, whole data objects, or even multimedia content that they wish to store along with the application’s data model.  Again, this type of flexibility has many prices and the value in supporting it needs to be assessed case by case.
  • Behavioral Customization:  A more complex type of flexibility is that of business behavior (e.g., configurable workflows, proprietary decisioning rules, calculation policies, etc.).  Unless tightly and explicitly bounded, this type of flexibility in a multi-tenant SaaS deployment can be an insidious slippery slope.  Tread carefully.
  • Platform API:  Many applications perform services that can be exposed via an API (e.g., web services).  Doing so can enable subscribers to incorporate the application more deeply using, for example, a Service Oriented Architecture (SOA) while increasing subscriber stickiness for the SaaS provider.  It also opens up the potential for multiple UIs, which may be a path to extreme UI customizations.  However, while exposing such APIs may appear straightforward, it is definitely not to be undertaken lightly.  More on this in another post. 

SOA in Good Eternal Company

graveyard-689x407There’s a place where good acronyms go to die.  I call it the GAG (Good Acronym Graveyard).  It’s a dark foreboding place where over-hyped acronyms lie interred separated from their perfectly valid and useful living legacies.

Terminal Terminology

The first GAG funeral that I witnessed in my career personally was Artificial Intelligence.  In the 80s and early 90s, AI was hyped to the point where our brains would surely atrophy as expert systems, neural networks, fuzzy sets, and other goodies would put homo-sapiens out of business.  AI would be the computing industry’s super hero of the era.  But just as most of our super heroes eventually disappoint as they fail to live up to impossible expectations, AI came crashing down.  So many companies and investors were burnt in the process that the term itself became a pariah.  A proposal or business plan could promise to cure cancer, but would be rejected out of hand if it included the term “AI”.

In reality, the AI funeral was for the term itself.  The living legacy of AI is all around us.  We have automated decisioning and diagnostic systems that use many expert systems concepts.  Rule based systems are widely used to codify business policies, determine insurance quotes, and manage the complexities of telecommunications billing.  Neural networks among other techniques are used in pattern analyses such as facial recognition and linguistics.  Just about every complex search technique in use today owes its roots to a university AI lab.  More generally, heuristic algorithms are now pervasive in everything from music recommendations to counter terrorism.

The principles and techniques of AI have been staggeringly successful, but the over-hyped term and its unreasonable expectations rest in peace in the GAG.  This was no time for sorrow, however.  With this burial went the wasteful distraction of trying to satisfy the insatiable.  Released from this burden, practitioners were free to focus and produce the awesome results that have transformed large tracts of the computing landscape.

So Soon SOA

Service Oriented Architecture or SOA has now entered the GAG.  Following a similar pattern as AI, there is nothing wrong with its principles.  In fact, SOA is exactly the transformative movement required by complex enterprises that require breakthrough advances in agility while avoiding the infeasible cost and limitations of wholesale legacy replacement.  Over the past several years, however, the term SOA has been over-hyped as a silver bullet, a specific technology, or a turnkey solution depending on the agenda of the “hyper”.  To these expectations, SOA must and has failed.

In a 01-05-2009 post entitled “SOA is Dead; Long Live Services“, Anne Thomas Manes writes the following insightful obituary:

SOA met its demise on January 1, 2009, when it was wiped out by the catastrophic impact of the economic recession.  SOA is survived by its offspring: mashups, BPM, SaaS, Cloud Computing, and all other architectural approaches that depend on “services”.

SOA is a strategy and an architecture (people tend to forget that’s what the “A” stands for).  It is a path to which enterprises must commit and in which they must invest in order to realize the returns.  When a project is framed as full blown “SOA”, compelling returns on investment are exceedingly difficult to devise and sell.  However, Software as a Service (SaaS) has gained acceptance as an agile, cost effective alternative to wide-scale software installation and maintenance.  Cloud computing is rapidly ascending to acceptance as a nimble alternative to sizing data centers to handle peak-plus demands.  Mashups are everywhere from grass-roots developers to the enterprise back office.  As these mindset changes continue to cure, the principles of SOA will flourish – even better without the baggage of the term itself.

Requiem

And so we gather together on this cold day in January of 2009 to lay to rest the body of SOA, but not its spirit.  We do not mourn this passing as untimely or empty.  Rather we rejoice in the opportunity to move past empty promises and impossible expectations.

Perhaps now that the GAG is sporting yet another tombstone, we can attend to the real business of enterprise transformation through service orientation.  Perhaps we can even throw in a little AI for good measure… D’OH!!!

Taxonomy for Web Service Sources

Various taxonomies for web services are possible.  A focus on technology might produce classifications such as transport (e.g., HTTP, JMS), representation (e.g., SOAP, REST), response handling (e.g., blocking, asynchronous via polling, asynchronous via callback).  A focus on purpose might look more like data source vs. computational vs. legacy API exposure.

In the Web 2.0+ era, web services are proliferating wildly.  The mashup community is providing huge demand satisfied in part by sites such as ProgrammableWeb where over 1,000 web services can be found, and increasingly online services are opening their platforms via APIs.  Enterprise level SOA (Service Oriented Architecture) initiatives, while slowed by a slowing economy, are also beginning to consume external services as well as exposing their own services for internal use.

In recognition of this proliferation, I believe a new taxonomy is required that addresses the source of services from the perspective of the user, orthogonal to technology or purpose.  By “user” in this context, I am referring to the human developer of any application that consumes web services.

Source Taxonomy

The figure illustrates a draft web service taxonomy where services are classified by the nature of their sources or providers as seen by potential users.

ws-source-taxonomy

Classification:  Ownership

Ownership is the distinction between services that are sourced within the user’s organization (e.g., company, business unit) versus those sourced by parties unaffiliated with their organization.

Internal:  “Services sourced within the user’s organization implying some potential for control over their implementation.”  Examples include web service APIs to internal legacy systems as part of a SOA project.

External:  “Services sourced independent of the user’s organization.”  Examples include information services (e.g., news, market quotes, credit report) and APIs to platforms like Twitter or SalesForce.

Classification:  Provision

Provisioning refers to the relationship between the entity supporting the web service endpoint from the user’s perspective (i.e., the provider) and the entity that supplies the functional implementation of the web service (i.e., the source).  When the provider is the source, the service is said to be original.  Conversely, if the provider is some form of third party intermediary between user and source, the service is said to be syndicated.

External / Original:  “External services that are called directly by a user’s application.”

External / Syndicated:  “External services for which users call a third party provider or syndicator which would then call the original source on their behalf.”  Presumably in this type of structure, the syndicator would add some value for acting as intermediary.  For example, a syndicator could serve as a common front for many original service sources thereby presenting an additional interface abstraction and a common point of billing and support.

Internal / Original vs. Syndicated:  Based on the foregoing definitions, the notion of a syndicated internal service seems oxymoronic.  The taxonomic intent is to enable larger enterprises to make the distinction, for example, between point-to-point calling of a legacy API (i.e., original source) versus the use of an intermediate hub, messaging service, or some other abstraction layer (i.e., a form of syndication).

Classification:  Differentiation

This classification addresses the potential for multiple sources to provide functionally equivalent services and how the user perceives the relative value of those sources.  A service is referred to as a commodity if it is possible for multiple sources to provide functionally equivalent implementations of that service, whether or not multiple such sources actually exist.  In contrast, a service is referred to as branded if there can be only one source of that service.

External / Original / Commodity:  “Services provided by an original source that can also be offered by other sources.”  The functional equivalence of these services can enable a user to select a source based on non-functional factors such as price, performance, and reliability.  Examples might include data services such as weather or financial market data.  In this scenario, the commodity sourcing decision must be performed by the user.  Despite functional equivalence, each source may present differing interfaces to which the user must code.

External / Syndicated / Commodity:  “Services provided by a syndicator fronting for potentially multiple functionally equivalent sources.”  The key value of commodity syndication lies in the fact that functional equivalence does not imply interface equivalence.  For a given commodity service, a commodity syndicator has the opportunity to normalize interfaces across functionally equivalent sources, thus providing the user with a single stable interface per service.  This scenario would support the commodity sourcing decision being made either by the user or transparently by the syndicator.

External / Original / Branded:  “Services provided by an original source that can only be available from that source.”  The most common types of these services are APIs to specific applications or platforms.  For example, consider writing a mashup for your back office that uses the SalesForce API.  It cannot decide lightly to call a different CRM application since it is highly unlikely that its API will be functionally equivalent at the service level, not to mention the fact that your company’s data lives at SalesForce.

External / Syndicated / Branded:  “Services that can only be available from a single source, but are accessed through a syndicator.”  This class is included for taxonomic completeness although it is unclear what significant value the syndicator would provide in this case.  There may be some value in a single gateway to multiple branded services for billing, support, or auditing purposes, but this alone hardly seems compelling relative to the overhead.

Classification:  Session

This classification recognizes that certain logical operations may require multiple web service calls.  While this may seem like a technical distinction, its relevance to this taxonomy is in the context of commodity source selection.

… / Commodity / Stateless:  “Completely independent web service calls enabling commodity sourcing decisions on a per call basis if desired.”  This is the finest granularity of web service commoditization.  An example of this might be a request for a stock quote for a known ticker symbol.  A single call does the job and there is any number of functionally equivalent service sources for this information.

… / Commodity / Statefull:  “A logically related group of web service calls that all must be made to the same source, thus necessitating a single commodity sourcing decision for the group.”  An example might be obtaining a credit report on a company.  A first call requests a “list of similars” based on the company name.  The returned list includes a set of possible matches with additional data for disambiguation and source specific IDs.  After selecting the desired company from the list, the second call requests the actual report based on the ID.  The user may not care which source is used, but having made the sourcing decision for the first call, the rest of this conversation must return to the same source since it carries source specific information.

Summary

The last 5 years have seen the rapid proliferation of available web services and a growing appetite of Web 2.0+ developers anxious to consume them.  Thus far, the focus has been on what mashups and service oriented applications can do and how to achieve them functionally.  Going forward, we will see increased attention to qualities of service, stability, and source redundancy analogous to that of cloud computing.  Lack of maturity in these areas is among the factors holding back enterprises from full scale consumption of external web services in their business applications.  Concepts such as syndication and commoditization can play a key role in breaking through this barrier.

Performance != Scalability

Performance and scalability are often conflated, sometimes equated, but only somewhat related.  In web services platforms, both characteristics are business critical.  But while each exerts influence over the other, they are highly distinct and have profoundly different business impacts.

Performance & Scalability Revealed

Using a web services platform as context, performance and scalability can be defined as follows:

particlePerformance is a measure of speed from a caller’s perspective; the time it takes for the platform to accept a request, process it, and return a response.  We measure response time at the platform’s front door (i.e., excluding Internet latencies, etc.).  From a platform perspective, performance is also a measure of throughput; the number of operations that can be completed per unit time.  This is a function both of response time and concurrency.

Scalability is an observation about the trend in performance under increasing demands.  For example scalability can be characterized as the rate of response time degradation relative to increasing platform traffic.  Alternately, scalability can be viewed as the increase in throughput attained via an increase in system resources (e.g., if I double my server capacity, will I actually get twice the throughput?).

Note that this definition of scalability is irrespective of absolute performance.  For example, a slow operation may be considered scalable if a 10X increase in load results in only a 1.2X increase in response time.  Conversely, a lightning fast operation may require a resource that can only be accessed serially, thereby causing a concurrency bottleneck and thus may not be scalable.

Clearly performance and scalability are related, but are not equivalent.  In practice however, the faster an operation’s response time, the less system resources it consumes.  This means that it reduces load on other operations and is less affected by the load of others.  Both effects can positively impact throughput and thus scalability; doing more per unit system resource.

Economics

In our practical universe, scalability and performance are highly related through economics.  Per the previous point, the faster an operation’s response time, the less system resources it consumes.  Generally speaking, this translates into higher throughput by a given set of physical resources.  Thus, the higher the performance, the lower the rate of capital expenditures required to handle a given rate of traffic growth.  Therefore, scalability and performance both contribute to economic scalability; doing more per unit cost.

Put another way, both higher performance and higher scalability reduce the cost of scaling.

So Who Cares?

Performance as response time is business critical because it directly impacts user experience, without which there is no business.  Our users and our business partners care.

Performance as throughput is business critical because it directly impacts system expenditures and the ongoing total cost of ownership.  I just made the CFO happy.

Scalability is business critical for avoiding infrastructure meltdowns in the face of market success and is one of technology’s key contributions to profitability.  Clearly we all care about this one.

Finally, an early understanding of scalability characteristics is critical to aligning engineering investments with company growth.  Unlike performance tuning, scalability limitations are rarely fixed quickly since they are often a function of architecture rather than implementation.  Many resource constrained startups choose functionality over scalability for good and obvious reasons.  However, the sooner an engineering team can target some serious focus on scalability, the fewer techies will be seeing 3am from the office.