Introduction

CAP stands for Consistency, Availability and Partition tolerance. The CAP theorem first appeared in autumn 1998 and published as the CAP principle in 1999 also named Brewer’s theorem after computer scientist Eric Brewer states that it is impossible for a distributed data store to simultaneously provide more than two out of the following three guarantees:

  • Consistency: Every read receives the most recent write or an error
  • Availability: Every request receives a (non-error) response, without the guarantee that it contains the most recent write
  • Partition tolerance: The system continues to operate despite an arbitrary number of messages being dropped (or delayed) by the network between nodes

Fallacies of Distributed Systems

The distributed systems are heavily dependent on various moving parts connected over a network and heavily rely on partitioning to achieve the scalability, and be informed that network failure is inevitable http://www.mypassionfor.net/2020/03/11/8-fallacies-of-distributed-computing/

 Image Credit – Denise Yu Twitter: https://lnkd.in/eJYnpC

Can a system have all CAP?

The only exception which can practically demonstrate all CAP would be a single node system, which can’t be regarded as a distributed system. Moreover, it can’t scale to the level of what a distributed system is capable of and designed with a goal in mind. 

Note: Single node can utilize only Scale-Up or Vertical-Scaling strategy. Any system which is using Scale-Out or Horizontal-Scaling has to abide by the CAP theorem.

Explanation

No distributed system is safe from network failures, thus network partitioning has to be tolerated. In the presence of a partition, one is then left with two options: consistency or availability. When choosing consistency over availability, the system will return an error or a time out if particular information cannot be guaranteed to be up to date due to network partitioning.

When choosing availability over consistency, the system will always process the query and try to return the most recent available version of the information, even if it cannot guarantee it is up to date due to network partitioning (which is better in many cases for user experience). When a network connection is established, consistency will be achieved again across nodes/partitions i.e. eventually consistent.

Database systems designed with traditional ACID guarantees in mind such as RDBMS choose consistency over availability, whereas systems designed around the BASE philosophy, common in the NoSQL databases, for example, choose availability over consistency.

CAP in Practice

Let’s say you are designing a distributed system that has a cluster of 4 data nodes. Replication factor is 2 i.e. any data written in the cluster must be written on 2 nodes; so when one goes down – second can serve the data.

https://www.linuxtopia.org/

Now try to apply CAP theorem on this requirement. Remember, in a distributed system, two things may happen anytime i.e. node failure (hard disk crash, EC2/VM failure) or network failure (the connection between two nodes go down) [Fallacy of distributed computing].

CP [Consistency/Partition Tolerance] Systems

Distributed systems ensure data consistency at the time of reading the data, by a voting mechanism, where all nodes who have a copy of data mutually agree that they have “same copy” of requested data. Now let’s assume that requested data is present in two nodes A-1 and A-2. The client tries to read the data; and our CP system is partition tolerant as well, so an expected network failure occurred and A-1 is detected as down. Now the system cannot determine that A-2’s data copy is the latest or not; it may be stale as well. So the system decides to send an error to the client.

Here system chose to prefer data consistency over data availability. Similarly, at the time of writing the data if the replication factor is 2, then the system may reject write requests until it finds two healthy nodes to write data fully in consistent manner. Hence, now the system is considered not “Available” i.e CAP’s “A” is sacrificed.

CP system is harder and expensive to implement.

AP [Availability/Partition Tolerance] Systems

In today’s customer-obsessed mindset, a system is expected to up-and-running 24×7 hence, you have to put proper thinking in place and make informed decisions while choosing trade-offs. As with CP, the system is known not to be “Available” in the event of a failure.

A resilient system design would rather consider instead of sending ERROR (in case A-1 is down); it sends the data received from A-2. Chances are that the data client will get may not be the latest data (hard to decide). Hence, now the system is considered “Available” i.e CAP’s “C” is sacrificed.

CA [Consistency/Availability] Systems, Really?

In a distributed environment, we cannot avoid “P” of CAP. So we have to choose between CP or AP systems. If we desire to have a consistent and available system, then we must forget about partition tolerance and it’s possible only in non-distributed systems such as an RDBMS system.

No alt text provided for this image

Closing Note

CAP is frequently misunderstood as if one has to choose to abandon one of the three guarantees at all times. In fact, the choice is really between consistency and availability only when a network partition or failure happens; at all other times, no trade-off has to be made.

CAP is to choose your tradeoff in the event of a failure (network/node). In the absence of network failure – that is, when the distributed system is running normally – both availability and consistency can be satisfied.

In today’s world, we can achieve all 3 in a distributed system (if not fully, then partially at least). E.g. Tunable consistency in Cassandra

Abstract

Those days are gone when an enterprise was happily serving its customers by just a handful of on-prem servers, delivering content within seconds to minutes, thinking hours of downtime for maintenance and deployment is fine and data consumption is in gigabytes.

Today’s distributed systems are meant to serve an ever-expanding number of customers “Anytime, Anywhere, on Any Device“, customers expecting a response in milliseconds and 100% uptime. Users expect millisecond response times with 100% uptime, and data is measured in Petabytes.

Today’s demands are simply not met by yesterday’s software architectures.

To meet such enlarged expectations, our distributed systems must be Responsive, Resilient, Elastic and Message Driven, and this is called a Reactive System. Systems built as Reactive Systems are more flexible, loosely-coupled and scalable. They are significantly more tolerant of failure and when failure does occur they meet it with elegance rather than a disaster.

Four Reactive Principles

Responsive: The system responds in a timely manner if at all possible. Responsiveness is the cornerstone of usability and ensuring early problem detection and resolution. Responsive systems focus on providing rapid and consistent response times and delivering consistent quality of service. This consistent behavior, in turn, simplifies error handling, builds end-user confidence, encourages and promotes continued interaction with the software application.

Resilient: The system stays responsive in the face of failure. Any system that is not resilient will be unresponsive after a failure. Microservice Architecture lays the foundation for building a resilient system. Resilience is achieved by isolation, replication, and containment, Failures are contained within each component, isolating components from each other and thereby ensuring that parts of the system can fail and self-heal without compromising the system as a whole.

Elastic: The system stays responsive under varying workloads and increased demands. Reactive Systems can react to changes in the input rate by increasing or decreasing the resources allocated and support the business continuity. This implies designs that have no contention points or central bottlenecks, resulting in the ability to shard or replicate components and distribute inputs among them.

Message Driven: The system relies on asynchronous message-passing to ensures loose coupling, isolation, location transparency, and provides the means to delegate errors as messages. Asynchronous messaging supports nonblocking communication which allows recipients to only consume resources while active, leading to less overhead on the system due to non-blocking I/O.

We all are living in the era of “Distributed Computing”, regardless of how simple it may appear from outside, it’s way over complicated under the hood. System Design approach must be not to fall in the trap of “8-Fallacies-Of-Distributed-Computing”. When I design and architect or even review, my favorite ones are to solve/look for the challenges related to #1, 2, 3, 4, 7, and 8 (however all are equally important). Which one grabs your focus when you design and architect? please comment, and I welcome your questions and/or suggestions. Image Credit – Denise Yu Twitter: https://lnkd.in/eJYnpCw

Are You Cloud-Native?

December 11th, 2019 | Posted by Vidya Vrat in Architecture | Azure - (0 Comments)

Introduction

Cloud-native is an approach to build applications using microservices, containers, Kubernetes, DevOps, CI/CD, and cloud (public, private, hybrid). With this great combination of great architecture, platform, culture, dev practice, and cloud-computing model your applications will be built for scale, continuous change, fault-tolerant, and manageable. Let’s explore and understand each of the key players in the cloud-native approach.

No alt text provided for this image

Microservices

 Microservice is a small autonomoustightly scopedloosely coupledstrongly encapsulatedindependently deployable, and independently scalable application component. Microservice has a key role to play in distributed system architecture. 

No alt text provided for this image

Microservices architecture offers a lot of benefits, but there is a cost involved. You may want to learn more about its advantages and disadvantages by following the link https://www.linkedin.com/pulse/microservice-architecture-vidya-vrat-agarwal/

Containers

A container is a standard unit of software that packages up code and all its dependencies, so the application runs quickly and reliably from one computing environment to another. A Docker container image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries, and settings. Containers provide an abstraction at the app layer that packages code and dependencies together. 

No alt text provided for this image

Kubernetes (K8s)

Kubernetes was born from Google’s over 15-year experience running production workloads. It is designed to grow from tens, thousands, or even millions of containers. Kubernetes is a container runtime agnostic. Kubernetes is an open-source container orchestration tool designed to automate deploying, scaling, and operating containerized applications. “Distributed Management and Orchestration “of containers is an integral part of making your application cloud-native, and Kubernetes does it all.

DevOps

DevOps is NOT a “Packaged Solution”. I.e. You cannot buy and install it.

No alt text provided for this image

DevOps is the union of people, process, and products to enable continuous delivery of value to our end users.” – Donovan Brown, MSFT. DevOps is a “culture” (Agile is a mindset), where development, test, and operations work together in a collaborative manner to automate the delivery of quality software. DevOps culture develops a “production-first mindset”. I.e. applying DevOps ensures that your code is always ready to be deployed to production 

No alt text provided for this image

DevOps breaks the wall of confusion between teams and fosters better communication and collaboration throughout the application development lifecycle and results in better visibility and small frequent deployments together.

You can learn more about DevOps here https://www.linkedin.com/pulse/devops-what-why-how-vidya-vrat-agarwal/

CI/CD

Continuous Integration (CI) and Continuous Delivery (CD) enables and empowers any software delivery team in an organization to continuously deliver value to their end-users.

With CI/CD practices in place, development teams are more likely to commit code changes more frequently, which leads to better collaboration and software quality.

Continuous Integration (CI)

CI is the process through which the building and validation of developer code are automated to ensure quality check-ins. You can build a CI/CD pipeline in the DevOps tool of your choice, and for cloud-native it will not be complete unless you have an important step “push the image to container registry”. Here, is a visual of how CI phases look like in Azure DevOps (previously known as VSTS).

No alt text provided for this image

If you are interested in building your own CI/CD pipeline using Azure DevOps then you can read more by following the link https://www.linkedin.com/pulse/building-cicd-pipeline-from-scratch-azure-deployment-using-agarwal/?trackingId=2ZXCXz7mpm8mnjAQB8BF7g%3D%3D

Continuous Delivery (CD)

CD picks up the package built by CI, deploys those into multiple environments like Dev, QA, staging, etc. then runs various tests such as integration tests, performance tests, load test, etc. and if all works fine then on a button click it can be finally deployed into production.

No alt text provided for this image

Summary

While each of the above serves tremendous value in its own entity, but if all the above combined together then it will contribute towards a true cloud-native approach. Imagine using benefits of cloud, microservices, containers, Kubernetes, DevOps, CI/CD combined together, which unlocks the formula to the success of any organization that want to reap the multifold benefits to the health of software, delivering value to the customers, support demand at scale, and lowering the development and operational cost of the application.

If you are not cloud-native yet, then you may be interested in knowing that by 2025, over 80% of enterprise apps will become cloud-based (source: forbes.com) or be in the process of transferring themselves over to cloud-native apps. Are you?

I spoke on “Microservices as a Product” at #productschool Bellevue WA on Oct 30th, 2019 Wednesday 5:45 pm – 7:45 pm. This was a #FREE session  https://lnkd.in/gAdS2fT 

No alternative text description for this image

Please find the content here at my #github https://lnkd.in/gwTDFfi and Recording at my #youtubechannel https://lnkd.in/gjFxnH3 or here below

On Sep 14, 2019, Saturday, I spoke at #SeattleCodeCamp (this was my 4th year in a row). Both of my sessions were unique in nature, simple to follow, and heavy on core concepts, and practical implementation. 
1- Writing code with a product mindset. – Pigott Room #306 8:30-9:30am
2- Real-world examples of MicroService Architecture and Implementation. – Pigott Room #205 11-12pm 
This was a FREE Tech Community event available to be registered at https://lnkd.in/eM-tzVs 

You can find the decks here at https://lnkd.in/gXCn8Rs

Other Articles you may find helpful: https://lnkd.in/gGzSWbV https://lnkd.in/gBAZc7t

No alternative text description for this image
No alternative text description for this image

An evening on Thu Aug 8th, 2019 well spent with people who appreciate the knowledge and aspire for learning. Thanks again City University of Seattle Clark Jason Ngo and Sam Chung for having me to talk about “Microservices Architecture”

No alt text provided for this image

No alt text provided for this image

No Rehearsal, No Re-Take. Past weekend, I recorded a podcast on “Microservices Architecture” with host Mario Gerard. The podcast is live now, it has 3 parts with greater details and insights #microservices #softwarearchitecture #mvpbuzz

We talk about:- TPM Podcast with Vidya Vrat Agarwal on Microservices.

  1. What is a microservice? TPM Podcast with Vidya Vrat Agarwal on Microservices.
  2. Advantages of microservices
  3. Monolithic applications vs microservices?
  4. Why is there is renewed uptick in organizations using Microservices?
  5. Pain points of microservices
  6. How do you break down monolithic applications into microservices
  7. Microservices Advanced Topics

Part -I – https://www.mariogerard.com/tpm-podcast-vidya-agarwal-on-microservices-part-i/
Part -II – https://www.mariogerard.com/tpm-podcast-vidya-agarwal-on-microservices-part-ii/
Part -III – https://www.mariogerard.com/tpm-podcast-vidya-agarwal-on-microservices-part-iii/

Microservice Architecture

May 13th, 2019 | Posted by Vidya Vrat in .NET | Architecture - (0 Comments)

Abstract

The microservices architecture style is an evolution of the Monolith SOA (Service Oriented Architecture) architecture style. The difference between the SOA and microservice approach is how these are being developed and operationalized. With every new technology addition, our responsibility also increases to be abreast of pros-and-cons the new member has, and the pain points it is designed to solve.

Monolith

Think of any MVC pattern-based API codebase, where all your controllers and POJOs (Plain Old Java Objects) or POCOs (Plain Old C# Objects) were developed, build and deployed as a single unit, and for almost all the times a single data store was used for the enterprise. I.e. One database is housing all the tables for various responsibilities, for example, Customer, Payment, Order, Inventory, Shipping, Billing, etc. as shown in the logical architecture diagram below. I.e. all the various responsibilities are together.

Monolith architecture

  Monolith Pros:

• Less Crosscutting Concerns: Being monolith and having all the responsibilities together, single or few implementations can cover all the major cross-cutting concerns such as security, logging.

• Less Operational Overhead: Having one large monolith application means there’s only one application you need to set up logging, monitoring, testing for. It’s also generally less complex to deploy, scale, secure and operationalize.

• Performance: There can also be performance advantages since shared-memory access is faster than inter-process communication (IPC).

Monolith Cons:

• Tightly Coupled: Monolithic app services tend to get tightly coupled and entangled as the application evolves, making it difficult to isolate services for purposes such as independent scaling or code maintainability.

• Harder to Understand: Due to many dependencies, monolithic architecture easily become harder to understand.

• Deploy all or none: When a new change needs to be pushed, all the services need to be deployed. I.e. if something changed in OrderController and you want to proceed with deployment, all other controllers and code will be deployed unwantedly.

• Scale all or none: Scale-up or Scale-out, it’s for entire functionality.

Microservice

Gartner defines a “Microservice as a small autonomoustightly scopedloosely coupledstrongly encapsulatedindependently deployable, and independently scalable application component. “ Microservice has a key role to play in distributed system architecture, and it has brought a fresh perspective.

Unlike monolith, a microservice strictly follows the Single Responsibility Principle (SRP) due to being tightly scoped around the business capability/domain for example Payment. Think of an MVC pattern-based API codebase where a controller and POJOs (Plain Old Java Objects) or POCOs (Plain Old C# Objects) were developed, build and deployed for just one single responsibility i.e. business capability. This microservice architecture will then lead to having many such projects and each business capability having its own database.

No alt text provided for this image

Microservice Pros:

  • Easier Deployments: Monolith was deploying all or none. Microservice is a small service, the dev team has complete control on what needs to be deployed and leave another feature’s code untouched. I.e. if changes made in Payment service, only payment can be deployed. This was not possible with a monolith.
  • Scalability: Being a small tightly scoped service design, it gives freedom and flexibility to scale whichever service you want. For example, if needed only Payment can be scaled up/down/out.
  • Easier maintainability: Each microservice service is autonomous and so being small service it’s much easier to maintain than a monolith.
  • Problem isolation: Each service has its own codebase and deployed in its own space. Hence, any problem with self or other services remains completely isolated.
  • Single Responsibility: Single Responsibility Principle is at the core of microservice design. This drives all other goodness microservice architecture offers.
  • Separation of Concern: Microservices architecture leads towards building tightly scoped services with distinct features having zero overlaps with other functions. 
  • Deep domain knowledge: Microservice encourages the product mindset and leads towards establishing deep domain knowledge with “you build it, you run it” mantra.

Microservice Cons:

  • Cultural Changes: It requires a cultural shift and organizational alignment to adapt microservice architecture, and the organization will require a mature agile and DevOps culture. With a microservices-based application, the service team needs to be enabled to manage the entire lifecycle of a service. Read my article on STOSA (Single Team Owned Service Architecture)
  • More Expensive: Microservice architecture leads to growing costs, as there are multiple services. Services will need to communicate with each other, resulting in a lot of remote calls. These remote calls result in higher costs associated with network latency and processing than with traditional monolith architecture.
  • Complexity: Microservices architecture by nature has increased complexity over a monolith architecture. The complexity of a microservices-based application is directly correlated with the number of services involved, and a number of databases.
  • Less Productivity: Microservice architecture is complex. Increased complexity impacts productivity and it starts falling. The Microservice development team usually requires more time to collaborate, test, deploy and operationalize the service which certainly requires more time.

Summary

Microservice and monolith architecture, both have their pros and cons. Microservice architecture is not a silver bullet, but teams that aspire to be digitally transformed and writing services with a product mindset, microservice architecture is certainly worth consideration. 

Single Team Owned Service Architecture (STOSA) is a guiding principle for large organizations that have many development teams that build and own microservices to cater to one or more enterprise-wide application(s).

A Microservice must be owned and maintained by a single development team within your organization.

To be a true STOSA based team/organization you must meet the following criteria:

  • Have application(s) which is/are developed using a Service (monolith/microservice) architecture.
  • Have multiple development teams that are responsible for developing and supporting the application. 
  • Each service should be assigned to only one development team.
  • A development team may own more than one service.
  • Teams are responsible for all aspects of managing the service, from service architecture and design, through development, testing, deployment, monitoring, and incident resolution.

Typically, a STOSA-based organization follow the above-mentioned criteria and each service team is of a reasonable size (typically three to seven developers also known as a two-pizza team). If a team is too small, it cannot manage a service effectively. If it’s too large, it becomes cumbersome to manage the team.

Non-STOSA-based organization

Above shown image represents a non-STOSA organization, and you’ll notice a couple things. First, Service “Shipping” does not have an owner. Yet, services such as Order and Billing are owned and maintained by more than one team (Team1, Team2, and Team3). There is no clear ownership. If you need something done in Order and/or Billing service, it’s not clear which team is responsible. If one of those services has a problem, who responds? Who do you contact for Shipping service? This lack of clear ownership and responsibility makes managing a complex application even more complicated.

STOSA based organization

The ovals represent development teams that own the enclosed services for example Team1 owns Payment, and team4 owns Shipping and Address. All the 6 services are managed by four teams. You’ll notice that each service is managed by a single team, but several teams may manage more than one service for example, Team3 and Team4. Main point is that every service has an owner, and no service has more than one owner. Clear ownership for every aspect of the application exists. For any part of the application, you can clearly determine who is responsible and who to contact for concerns, clarifications, questions, issues, or even changes.

Advantages of a STOSA Application and Organization

Clear and definitive ownership of a service is the key for the success of any application or the organization. As applications grow, they grow in complexity. A STOSA-based application can grow larger and be managed by a larger development team than a non-STOSA based application. As such, it can scale much larger while still maintaining well, documented, supportable interfaces within the established SLAs, and it’s all due to well defined ownership. 

A STOSA-based organization can handle larger and more complex applications than a non-STOSA organization. This is because STOSA shares the complexity of a system across multiple development teams effectively while maintaining clear ownership and lines of responsibility.