Cloud-aware Application Design and Architecture - V 3

Introduction and Purpose

The purpose of this document is to introduce the concept of a cloud-aware or cloud-native application and reconcile that with the underlying technologies associated with the CloudOne environment.

Design Patterns of Microservices and equivalent UI

• The CloudOne environment is intended for hosting applications consisting of small application components, known as Microservices
• A Microservice is an application service called or triggered via a common or standard protocol, such as HTTP REST calls or consumption of worked queued in a system like Kafka, with a narrow scope of functionality designed to do one thing – the intent in dividing an application into smaller Microservices is to keep all development and deployable units small in scope, therefore enabling a more rapid development and test cycle with a smaller risk associated with more frequent deployments of these narrow-scope units of functionality
• The scope of a microservice is narrow enough to fit within a single small business team
• A web-based user interface (UI) layer can be designed and written with the same approach taken as with microservices, emphasizing the same approaches for modularity and isolation
• A Microservice is intended to be self-contained as a deployment unit, meaning that ideally it can be deployed independent of other application units and containing its own set of unit tests that can be run via automation to ensure successful deployment and expected functionality

Implications of container technology

• Containers are ephemeral – they are created when needed and can be shut down as needed or may simply be restarted during the normal course of runtime execution. Data within these containers will be lost when a container is shut down or restarted. Therefore no data should be stored locally within a container except for data held temporarily for caching purposes for the life span of a single transaction.
• Containers are independent units of execution – they run independently of other code and infrastructure except where they explicitly call other application or infrastructure services to retrieve and/or persist data, to call dependent or required services, etc. This means that any number of containers executing a given application components should be running independently from each other. Any of these components may pick up work (e.g. via web service calls, picking up the work from an asynchronous message queue or by polling a backend data service, etc.) and should be able to safely execute its application logic.
• Containers have work distributed to them via a load balancing system – interaction with a container is session-less, which means session state is not assumed to be preserved and session stickiness is not enforced by the distribution mechanism for the work. Therefore, requests issued to containers and transactions executed by the application components within containers must not be session-oriented (unless session context is preserved by a backend service and retrieved on every transaction call, which adds overhead).
• Containers are self-contained units of execution – they rely on automation to be provisioned into existence and cannot rely on manual preparation steps after coming into existence in order to be ready to run the application components they contain. This means that all steps to bring the deployed application component into a ready state for execution must be contained within the definition of the container and the deployment logic triggered by deployment pipelines.
• Containers are small – they generally fit into relatively small memory footprints and consume limited amounts of CPU cycles in order to allow many of them to run concurrently. When more is needed to keep up with workload, container management systems scale out the number of containers horizontally to simply run more containers to keep up with load.
• Containers are independently managed entities – they should have interfaces that expose the health of their application components running within (e.g. application listening on the exposed HTTP request port, a /health endpoint exposing the health of internal components, etc.) to allow the container management system as well as other monitoring systems to track their health and react as needed.

Application Design Guidelines

• A common codebase for a single application component is maintained and used across all deployment environments
• Configuration details are kept separate from the application logic in the code and exposed at deployment or launch time
• As a corollary to the last item, secrets, such as passwords, credentials, etc. are not stored within the application code. These should be kept as configuration data, preferable in a secured vault facility, and obtained during runtime only
• Deployable units of application components (the combination of compiled code and configuration details) must be self-contained; whatever is needed to make that component run must be built into the container image deployed into CloudOne
• Shared infrastructure like databases, etc. are external to the application components. The information about these external, shared components, or "backing" services, is defined to application containers in configurations to enable an application component to attach to those "backing" services
• Application components are generally stateless and session-less, with quick startup and orderly shutdown
• Regardless of what network ports are used within an application component internally, no assumption is made that other services must know the specific port mappings within their code in order to reach the application component (i.e. avoid hard-coded port assignments)
• Multiple containers running the same application component must be safe to run concurrently
• Minimize the differences between deployment environments (i.e. keep technology the same even when details such as credentials are different for external dependencies like databases, message queue systems or other "backing" systems)
• Send logs always to standard output so they can be collected and exposed – do not log anything to local files within a container

For more details regarding design principles, see the later section detailing the design principles of a 12-factor application.

CloudOne conventions

Additional Do's:

• Do incorporate pool resilience: Where software layers like Spring Boot manage connection pools to external resources, include health checks and other properties to proactively manage connection pools and enable a retry capability in order to ensure recoverability in the event of temporary loss of connection to these external resources
• Do run in nn-privileged mde: Configure containers to run software in non-privileged mode (i.e. not as root) in order to minimize risk to platform security compromise as well as to minimize the risk of accidental modification of running containers that may go undetected
• Do expose metrics: Incorporate instrumentation into the application running in the container (e.g. Spring Boot Actuator) to allow for health checks and to expose other metrics that would be useful to monitor

Additional Don't's:

• Don't oversize threadpools: While it is perfectly Ok to build multi-threaded applications inside of containers, be mindful that thread capacity inside of a container is generally less that the thread capacity on a traditional server; therefore, avoid very high thread pool needs
• Don't oversize connection pools: Because containers can scale horizontally (meaning scaling comes from starting up additional containers), be mindful of connection pools to other shared external resources such as databases; avoid excessively large sized connection pools as these will add up and eventually reach the upper limits of connection capacity on those shared external resources
• Don't oversize containers: As a general rule of thumb, try to minimize the resource needs of any single instance (container) of an application, whether memory, threads or CPU demand; favor horizontal scaling over vertical scaling (i.e. more small containers is generally better than fewer large ones)

12-Factor Design Principles

1. Single codebase (in revision control system) for all environments
2. A single code base also means that every small application component (microservice, microapp, etc.) must have a separate and single code repository of its own
3. If there is any shared code, it should be maintained in one or more external code repositories and imported into the application component when needed and tracked as explicit dependencies
4. The code base is singular and identical across all deployments of the same application component across all environments (though there can be different versions deployed in different environments at any moment in time; these should be represented by different code branches or the same code base and repository)
5. Dependencies are isolated and explicitly declared
6. Every dependency on external or shared libraries must be explicitly declared
7. The application must not assume the prior existence of any specific external system tools in an environment; whatever tools may be needed must be explicitly declared with automation in place to ensure the installation or instantiation of such tools (this can be addressed outside of the application code, as part of image definition or deployment logic, but it must be explicitly defined)
8. All external dependencies that must be included within the application run-time environment should be built into that run-time environment
9. Dependencies are isolated in that the specific supporting libraries and versions built into the run-time environment for one application are completely separate and unimpacted by the supporting libraries and versions built into the run-time environment of another application
10. Configuration is stored in the environment
11. "Configuration" refers to attributes, properties and behaviors that would vary from one environment or deployment to another
12. Examples include credentials to external services, URLs and other references to integration points and properties that dictate variations in application logic and flow
13. Configuration must be stored outside of the code in order to keep configuration and code strictly separated
14. Configuration data can be defined via environment variables or dynamically retrieved language-specific properties served from an external component (e.g. a property server) or any mechanism that is external to the actual business logic code
15. Backing services are used as attached resources
16. A "backing service" is a functionality consumed by an application (it can be local or an external 3rd party service) in which the functionality is not included within the deployed application
17. Reliance on a backing service should make no assumptions about the location or configuration level access to that application (URLs, credentials, database names, etc).
18. The application attaches to the backing service as a resource using environment-level configuration for the details needed to access the service (i.e. these details should NOT be baked into the application code)
19. Strict separation between code build vs code deployment vs code execution

20. There is a strict separation and distinction made between the 3 phases of promotion of business logic to run-time execution, represented by the following 3 definitions:

    1. Build – A "build" is the compilation of source code, combined with a downloading and linking of any dependencies, to produce an executable program or bundle
    2. Release – A "release" is combination of a particular versioned build with the deployment configuration details of a given environment
    3. Runtime – A "runtime" is the execution of the particular version of an application in a given environment (the release)

21. Implications of these 3 stages being strictly separated include:

    1. It is not possible to apply changes to the code of an application at runtime
    2. Deployment tools generally have a mechanism to manage distinct releases of applications, enabling predictable deployment of the correct version of an application as well as a reliable mechanism for rolling back to prior versions if needed
    3. Every release should have a unique identification (whether a version number based on some nomenclature or a timestamp-based identification); releases are immutable meaning that once a release is created, it cannot be modified – updates require a new release to be generated
    4. While a build stage can be complex and error prone (due to the complex nature of software development) the process of execution of a runtime must be quick and reliable to enable automatic restarts and quick recovery from failures or shutdowns
22. Applications and application components are executed as stateless processes
23. Applications must be stateless – meaning that they persist no data within (they DO persist data, but only to external backing services such as databases or shared storage)
24. The internal memory or storage of an application (or a container within which the application executes) can be used only for transient data; the data is assumed to be short-lived for the length of single transactions and makes no assumptions about the presence of data stored locally based on previously executed transactions or activity
25. "Sticky sessions" (often a required assumption for certain web applications) where transactions are expected to be handled by the same processes that handled previous transactions for the same session are considered an anti-pattern and a violation of 12-factor design principles; a resolution to this CAN be the persistence of temporary session state data in a high-speed, low latency backing service (e.g. in memory caching service)
26. Services are exposed via port binding
27. An application that is exposed as a web service does not rely on an externally provided web container and web service; the exposure of the application on a network port is self-contained within the application
28. The application can expose itself on any local network port, but the mapping of that local network port to how it will be exposed to services and applications outside of its execution environment is handled external to the code at time of deployment
29. The exposure of an application via network ports is applicable to any protocols of communications and integration and is not limited to HTTP – these provide methods of integration between application components running in independent containers. This also means that any application can become the backing service of another applications.
30. Horizontal scaling
31. Concurrency can exist in the form of multiple threads running within the same deployed application process or container. However, the application should also be able to scale out horizontally via the execution of multiple processes running in independent containers concurrently
32. The application does NOT manage the processes or the number of processes to be run concurrently; it relies on the execution environment (whether a host server or container management system like Kubernetes or OpenShift) to manage the life-cycle and scale of concurrent processes and containers
33. Disposability – quick startup and graceful shutdown

34. Applications should be designed for:

    1. Rapid startup – minimized startup time, providing greater agility for scaling up the number of processes when needed or rapid recovery in the event of process or container failures
    2. Graceful shutdown – orderly completion of transactions in flight upon receipt of signals instructing the process to shut down or reliable dependence on backing services to roll back incomplete transactions
    3. Transaction queuing is often a method used to pass work between application components in such a manner that incomplete work results in a return of the unfished work to the queue to be consumed by another surviving process in the event of process failures
    4. Implied guidance here includes keeping HTTP-based transactions short in time and including retry and reconnect capabilities in HTTP (e.g. REST) calls
35. Keep all environments, from non-production through production, as similar as possible

36. Parity between development, testing, staging and production should be optimized by keeping to a minimum the 3 main factors in creating gaps between development and production:

    1. Time – minimize the time from development to final deployment into production to minimize the risks associated with changing external factors
    2. Personnel – Maintain a continuous process to minimize the impacts of different parties involved in the deployment of applications in different stages and environments
    3. Tools – keep the technologies as much as possible identical in all environments despite logical abstractions designed to make the swapping out of different backing services as transparent as possible

37. The most common approach to address the 3 gaps described above is a Continuous Deployment strategy that would, ideally, enable a rapid flow of new development through testing stages and into production with minimal intervention. Real-world circumstances may not enable a continuous flow with no intervention at all (e.g. manual approvals) to occur within a very short timeframe, but a 12-factor design for applications and their deployment will strive to:

    1. minimize the manual intervention steps
    2. rely on repeatable automation, and
    3. encourage as short a timeframe as possible between the development of application components, their subsequent testing and ultimate release to production (which is further enabled by the break-down of applications to smaller components in order to enable more rapid updates and validations and minimize the risks associated with logic changes and updates)
38. Logs as event streams
39. An application should not attempt to manage logs in local log files or make any assumption about the ultimate destination of logs
40. Applications should simply send their log data as unbuffered output streams to standard output (stdout) to rely on the execution environment to collect, archive and manage
41. Administrative/management tasks as one-off processes
42. Separation of administrative tasks from the "normal" logic of the application
43. Even though separated, administrative tasks should operate within the same runtime environment and using the same configuration
44. If and where applicable, administrative tasks logic should operate within the same codebase and, if and where applicable, be delivered with the same code release to prevent drift in functionality

References:

1. https://12factor.net ("The Twelve-Factor App")
2. DevOps with OpenShift, by Stefano Picozzi, Noel O'Conner and Mike Hepburn, published by O'Reilly Media, Inc., Released: July 2017
3. 10 Things to Avoid in Docker Containers, by Rafael Benevides, 2016, https://developers.redhat.com/blog/2016/02/24/10-things-to-avoid-in-docker-containers/
4. 9 Pillars of Containers Best Practices, by Marc Hornbeek and Eric Glasser, Trace3, 2018, https://containerjournal.com/topics/container-management/9-pillars-of-containers-best-practices/