NetApp logo
Documentation PortalBack to Self Assist PortalBack
Documentation Portal
Contents

Container-As-A-Service (CAAS) Application Stack - V 4.7

Container-as-a-Service (CAAS) Application Stack

The Container-as-a-Service (CAAS) application stack is used to deploy a pre-built container image into the CloudOne environment. This type of application stack is unique in that there is no software build involved, which also means there are no Git pull requests against application source code to trigger the execution of a deployment pipeline.

Despite no software to build, a pre-built container image still shares common requirements with other application stacks deployed to the CloudOne environment. These shared requirements include:

  • Security scanning for any vulnerabilities
  • Staging of the application in the appropriate Docker image repositories and tagged according to CloudOne conventions
  • Completion of Unit Testing and User Acceptance Testing
  • Gated approvals before allowing deployments into pre-production and production environments
  • Audit trail and history of deployment within the CloudOne CD Pipeline

Because of the above-listed requirements, the CAAS application stack is provided in order to support the deployment of pre-built container images in a manner that integrates with CloudOne requirements and processes. If the application is custom built, it is assumed to be built using a Continuous Integration pipeline prior to hand-off to the CloudOne CD pipeline. The flow of deployment still includes first a Continuous Integration stage of processing within the CD pipeline prior to deployment in the Continuous Deployment stages. The Continuous Integration stage focuses primarily on staging the image in the appropriate Docker image repository and scanning it for security vulnerabilities. However, no image build takes place for a CAAS application. The build of the image and its storage in a docker repository is a prerequisite to the start of the CloudOne pipeline.

Getting Started in the Azure DevOps environment

Refer to the following link to learn about getting started in the Azure DevOps environment: Getting Started in Azure DevOps Environment

Source Code Repository Structure

The structure of the source code repository for the CAAS application stack will contain one directory tree with details for deployment, as one of the following:

  • appCode-serviceName – (named as the AppCode followed by the application or service name) contains the Helm chart deployment materials

Also at the top of the repository is a file called azure-pipelines.yml. This file contains reference to the appropriate version of the CI/CD pipeline logic (the CAAS application stack is available only from version 3 and above) as well as YAML data structures providing key information about the environments into which to deploy the application and the sequence of events to complete the deployment (e.g. dependencies, additional steps to retrieve secrets to be passed to the deployed container, etc).

Additional items in this repository will generally not be modified and should not be changed to avoid risk of breaking the pipeline workflows.

Application Configuration into the CloudOne CD Pipeline

Once navigated to the appropriate repository, it can be cloned to a local development environment (i.e. local workstation) for the actual configuration work for the CAAS application to be deployed. This would consist primarily of defining the target workspace and hostspaces plus any dependencies between hostspaces or prerequisite deployment objects required. Once it is ready to be deployed into the CloudOne workspace, commit these changes back to Git and manually launch the pipeline.

Triggering the Continuous Delivery Pipeline

The CAAS application stack, because there is no source code updates to trigger builds, is not triggered by changes to a source code repository. Prior to version 3.6 of the CI/CD pipeline, the pipeline could be manually run from the Azure DevOps web interface and with version 4.7, the pipeline can be triggered by updates to the container stored in Artifactory, or still by manually running the pipeline.

As specified in the project's README.md file, prior to running the pipeline (or as a trigger to automatically start the pipeline, in the case of the Artifactory webhook described below), the new container image should be uploaded to Artifactory to the repository named docker-appCode (where appCode is the app code for the project), under an /uploads directory, named for the designated container image for the project and tagged with the appropriate version to be deployed.

Manually triggering the Continuous Delivery Pipeline

When relying on manual execution to run the CI/CD pipeline, it prompts for the version number of the container image to be deployed. This image is expected to be located in the same Artifactory as a container image that would have been built from any other application stack.

Automatically triggering the Continuous Delivery Pipeline

With version 4.7 of the CI/CD pipeline, an automatic trigger of the pipeline is possible as with version 3.6. It is implemented with a webhook within Artifactory, provisioned and defined by the CI/CD pipeline. This webhook will be set up automatically for newly created projects, but can be activated for existing projects by making the following changes in the project's azure-pipelines.yml file:

  1. The pipeline for the project must be pointed to release 4.0 or later
  2. A webhook resource for the pipeline must be defined
  3. The caasImageVersion variable must be set with no default value
  4. The caasImageVersion variable must be defined to accept a value from the webhook

*Note that the changes to the azure-pipeline.yml file described in this section must be applied to the Git repository on the master branch, in order for the webhook to take effect. Changes made to any other branch will not cause the webhook to activate.*

The pipeline for the project must be pointed to release 4.0 or later

In order to upgrade the project to use the version 4.7 CI/CD pipeline (or a later version, when available), update the ref: YAML value for the templates resource in the azure-pipelines.yml file to look like the following:

resources:
  repositories:
    - { repository: templates, type: git, name: devexp-engg/automation, ref: release/v4.7 }
    - { repository: spaces, type: git, name: spaces, ref: devops/v1.0 }

A webhook resource for the pipeline must be defined

Add a new resource under the resources: YAML section of the azure-pipelines.yml file to define the webhook, as follows:

resources:
  ...
  webhooks:
    - { webhook: webhook, connection: webhook-appCode, filters: [{path: data.image_name, value: caas_image_name}] }

Note that value defined for the resource in this example is set to caas_image_name and should be substituted with the correct name for the image as defined for the project. Also the connection should be changed to use the appropriate appCode value for the project.

If the webhook should be in a disabled state in order to not allow automatic triggering of the pipeline, use any value that will not match the image name, such as disabled_caas.

The caasImageVersion variable must be set with no default value

In the parameters: section of the azure-pipelines.yml file, the caasImageVersion variable must have the default: attribute defined but left with a blank value, in order to not have the default value override the webhook. The variable should be defined as in the following yaml code:

parameters:
- name: caasImageVersion
  type: string
  displayName: "Specify the CaaS image version to deploy"
  default:

The CaasImageVersion variable must be defined to accept a value from the webhook

Variable caasImageVersion will only prompt when the value is not supplied from the webhook and to accept the value from the webhook, when available, with the following YAML definition in the azure-pipelines.yml file:

extends:
  template: devops/caas.yml@spaces
  parameters:
    appVersion: ${{ coalesce(parameters.webhook.data.tag, parameters.caasImageVersion, 'version') }}

YAML Example for Webhook Trigger

A complete working example of the azure-pipelines.yml file with support for automatic triggering of the CI/CD pipeline would look like the following at the start of the file (using devexp for the appCode and caasimagename as an example image name):

parameters:
- name: caasImageVersion
  type: string
  displayName: "Specify the CaaS image version to deploy"
  default:
name: $(SourceBranchName)_$(BuildID)
trigger: none
pr: none
resources:
  repositories:
  - { repository: templates, type: git, name: devexp-engg/automation, ref: release/v4.7 }
  - { repository: spaces, type: git, name: spaces, ref: devops/v1.0 }
  webhooks:
  - { webhook: webhook, connection: webhook-devexp, filters: [{path: data.image_name, value: disabled_caas}] }
extends:
  template: devops/caas.yml@spaces
  parameters:
    appVersion: ${{ coalesce(parameters.webhook.data.tag, parameters.caasImageVersion, 'version') }}

Defining hostspaces for application deployment

Pipeline 4.7 simplifies the structure of azure-pipelines.yml to define all spaces(workspace,devint,db/hostspaces) as a single list. Older format is still supported but to continue using old format ref and k8s must be removed from spaces. It's strongly recommended to follow newer format. acceptanceSpace must be specified in the file. For example:

parameters:
  acceptanceSpace: devexp-ae103-stg
  spaces:
    workspace:
      helm:
        overrideFiles: |
          devexp-caas/values.workspace.yaml
    devexp-ae101-dnt:
      helm:
        overrideFiles: |
          devexp-caas/values.hostspace.yaml
    devexp-ae103-stg:
      helm:
        overrideFiles: |
          devexp-caas/values.hostspace.yaml
      dependsOn:
        - devexp-ae103-stg
    devexp-npc02-prd:
      helm:
        overrideFiles: |
          devexp-caas/values.hostspace.yaml

Application Logging

Although when deploying a pre-built container image there may be limited control over log message formats, certain guidelines will help to ensure the capture of logs. All logging from the application should go to standard output (also sometimes referred to as “console” in the context of common logging libraries). In order for log entries to be picked up and collected into the logging collection infrastructure (Splunk), the log entries must match a particular format. That format looks like the following:

[ISO8601-Date] logLevel=level pid=pid thread=thread class=logger-class message=free-style-message where:

  • level is the logging level (INFO, WARN, etc.)
  • pid is the process ID
  • thread is the thread ID
  • logger-class is the class calling the logger (indicates where in the code the log message was issued)
  • free-style-message is any addition logging content

The format can be further extended by adding additional key=value pairs before the free-style log message.

Continuous Delivery Pipelines

Please note that the document “CloudOne Concepts and Procedures” contains more details about the specific flow of the application through the CD pipelines

Once the CD pipeline for CAAS is started, there is no application build, however the container image is still subject to some quality checks that are prerequisite to deploying the image in CloudOne.

Details for navigating to, starting and examining the CD pipeline and analyzing its results can be found here: Continuous Integration and Continuous Delivery Pipelines

Upgrading to Pipeline v4.7

CloudOne DevOps Pipeline v4.7 introduces CyberArk Integration with Rancher. Conjur performs certificate-based authentication with each Rancher cluster and securely supplies secrets to each namespace/application within the cluster. Secrets are securely stored in the CyberArk vault. For more information refer Conjur Integration Document.

  1. Setup CyberArk integration (link)
  2. Remove ambassador.yaml file from templates folder.
  3. Add release 4.7 referance in azure-pipelines.yaml.
resources:
  repositories:
  - { repository: templates, type: git, name: devexp-engg/automation, ref: release/v4.7 }
  - { repository: spaces, type: git, name: spaces, ref: devops/v1.0 }

Note : Security scan stage was release as a minor update in v4.5 to read more about how this feature works please refer <a href="https://selfassist.cloudone.netapp.com/docs/secret-scanning-pipeline">New Security Stage of Pipeline: CrowdStrike Image Scanning and Gitleaks Secret Scanning</a>.

To transition to v4.7, we recommend using the dxpctl tool. - Access the latest tool here

Downloadable Version Docker Build/Run Locally

Pipeline Definition: azure-pipelines.yml file

The logic for the v4.7 CAAS CD pipeline is a common and shared code base for all applications; however the configuration of the pipeline that applies the common logic to the specific application is defined in the top level directory of the source code repository for the application in a file named azure-pipelines.yml.

The structure of the azure-pipelines.yml file is a YAML file using standard YAML syntax. For general information about YAML structures, there are many available resources, including the tutorial at this link: YAML Tutorial.

There are certain required data elements that must be defined within the azure-pipelines.yml file as a prerequisite to the CD pipeline running while other elements are optional and used to modify the standard behavior or the CD pipeline.

More details about the azure-pipelines.yml file can be found here: General Structure of the azure-pipelines.yml File

Configuration and Customization of the CAAS v4.7 Pipeline

Application-Specific Pipeline Configuration

The extends YAML object is a complex object consisting of additional YAML objects. This object is used to extend the pipeline logic (referenced by the repository defined in the resources object) by (a) referencing the correct appstack pipeline entry point (devops/caas.yml for the CAAS pipeline) and (b) passing a set of YAML objects as parameters to influence the behavior of the pipeline to meet an application teams specific needs.

The extends YAML object consists of 2 objects beneath it:

  • template
  • parameters

The template YAML object is a single value set to the initial entry point for the v4.7 pipeline for the CAAS appstack, so it should always be defined as follows:

extends:
  template: devops/caas.yml@spaces

The parameters YAML object is defined immediately following the template object and at the same indentation level. This is the object that requires the most attention and definition to be set up.

The parameters YAML object includes one YAML object for the CAAS application version which is interactively prompted and filled in at start time for the pipeline. Additional parameters would be defined here normally, however, as there are no software builds for CAAS, there usually are no additional parameters.

Horizontal Pod Auto-scaler

Pipeline v4.7 includes configuration of Horizontal Pod Auto-scaling for any deployment. Please refer to documentation for more details. To configure HPA use below template in values.yaml for the host space required. Below commented example starts the deployment with a single pod which can be scaled up to 3 pods when avarage cpu usage crossed 70%.

# If you are using Horizontal POD Auto Scaler, you must disable ReplicaCount
replicaCount: 2

# hpa:
#   min: 1
#   max: 3
#   cpu: 70

PVC for application stacks

Persistent Volume Claims (PVCs) for application stacks involve providing support for persistent storage in application stacks deployed in Kubernetes clusters. By enabling PVCs for application stacks, This feature will allow developers to define and configure persistent storage requirements for their applications using PVCs.

persistence:
enabled: true
#storageClass: ""
accessMode: ReadWriteMany
# subPath: appdata
size: 25Gi
mountPath: /data

Pod Anti-Affinity

Pipeline v4.7 enchances pod scheduling on the cluster with addition of Pod Anti-affinity. For multi-pod applications, this ensures pod scheduled scattered across the nodes within a cluster.

Detailed Pipeline Configuration

The remainder of the configuration work in the azure-pipelines.yml file focuses primarily on defining the target workspace and hostspaces for the CAAS application and providing details for these spaces, any additional tasks needed to prepare the deployments, and dependencies that will dictate the sequence of deployments in these spaces beyond the established Cloudone environment deployment approval processes. Details for configuring these elements of the pipeline can be found here: Pipeline Configuration Details

Kubernetes Deployment Objects

In order to deploy an application, including a pre-built container image as with a CAAS application, a number of Kubernetes objects must be defined and deployed. The definition of these objects is controlled through a set of files as Helm charts. Information about the contents and customization of these Kubernetes deployment objects can be found here: Kubernetes Deployment Objects

Troubleshooting

If something fails to deploy, the information about why the deployment failed (or was not even initiated) will be found in the logs of the CI/CD pipeline and can be tracked down using the methods described earlier in the “Continuous Integration and Continuous Delivery Pipelines” section.

However, additional information may be required either to better troubleshoot a failed deployment or to investigate the runtime behavior of an application that has been successfully deployed. In those cases, much of the information can be found in the rancher web console. Information about navigating and analyzing information from the rancher web console can be found here: Navigating the rancher Web Console

CONTENTS