Introduction

When a new version of the CloudOne CI/CD pipeline is released, each repository that uses this CI/CD pipeline must be updated in order to inherit the new functionality.

Some changes within each repository may be required as part of the pipeline upgrade. The changes can sometimes be as simple as pointing to a newer pipeline version from within the repositories azure-pipelines.yml file, or sometimes more complex like changing the structure and format of the azure-pipeline.yml file and multiple Helm chart files that define the application's deployment into Kubernetes.

This document describes an upgrade tool that automates as many changes as possible to the above-described repository files, in order to minimize the effort required to upgrade an existing repository. At a minimum, some manual review of the changes will be required after running the tool and some additional updates may be required in cases where the tool would not have enough information to apply changes automatically; however the intent is to minimize these manual updates for a more streamlined upgrade process.

This upgrade tool is provided as a Docker image for users to run on their local machines to upgrade their pipeline.

Upgrade Tool Usage

1. Install Docker on your machine.
2. If you don't have a local copy of the repository, clone the Azure DevOps Git repository to be upgraded.
3. Check-out the branch of the repository to be upgraded (the upgrade tool does not assume which branch should be updated; it will apply the updates to whatever branch is checked out).
4. Enure that there are no outstanding changes to be committed and that the local commits are up-to-date. The upgade tool will not update a repository that has uncommitted changes or commits that have not been pushed to the Azure DevOps Git repository.
5. Within your cmd/shell, cd to the directory containing the Git repository with the appropriate branch checked out.
6. Run the following commands for your architecture.

Windows (cmd):

Authenticate with your SSO:

docker login docker-dxp.repo-hio.cloudone.netapp.com

Make sure you have the latest image:

docker pull docker-dxp.repo-hio.cloudone.netapp.com/dxpctl

Run the upgrade utility:

docker run --rm -it -v "%cd%:/repo" docker-dxp.repo-hio.cloudone.netapp.com/dxpctl upgrade

Windows (powershell):

Authenticate with your SSO:

docker login docker-dxp.repo-hio.cloudone.netapp.com

Make sure you have the latest image:

docker pull docker-dxp.repo-hio.cloudone.netapp.com/dxpctl

Run the upgrade utility:

docker run --rm -it -v "${PWD}:/repo" docker-dxp.repo-hio.cloudone.netapp.com/dxpctl upgrade

Linux (shell):

Authenticate with your SSO:

docker login docker-dxp.repo-hio.cloudone.netapp.com

Make sure you have the latest image:

docker pull docker-dxp.repo-hio.cloudone.netapp.com/dxpctl

Run the upgrade utility:

docker run --rm -it -v "$(pwd):/repo" -u $(id -u):$(id -g) docker-dxp.repo-hio.cloudone.netapp.com/dxpctl upgrade

Under most circumstances, no additional parameters or options are needed; however, if there is a need to pass additional information, the following parameters are available:

Execution of the Upgrade Command

Note: Please make sure .hostspace.yaml and .workspace.yaml file names are updated as below, It is not must if you are following different naming conventions tool will skip the updates and you need to update the changes manually.

<values.hostspace.yaml> & <values.workspace.yaml>

The output of all of these phases look like the following:

Validate Phase Output

After verifying that the target directory contains a valid CloudOne project and is clean of any outstanding updates not committed to Git, the old and new pipeline versions are displayed as well.

In the event that one of the phases encounters an error and cannot continue, the upgrade tool will stop at the point of failure. Depending on the phase, some partial updates may have taken place and will need to be rolled back. In these cases, refer to the Rollback of Updates Prior to Commit section below.

SetVars Phase Output

Every variable assignment is listed and if a variable value cannot be determined, the user is prompted for the value

Prep Phase Output

The output from the Prep phase will indicate the path to the backup of the project repository preserved prior to applying any changes. Note that the directory under which this backup is saved is added to the .gitignore file for the project so that the backup will not be preserved as part of any commits to the Git repository.

Update and Template Phase Outputs

The Update phase actually includes the Template phase, which is actually the first step carried out, so the output will initially show the start of the Update phase, continue with the Template phase and finish again withe the Update phase.

If the retrieval of the Template files required for upgrade needs authentication from Git, the user will be prompted to complete the authentication.

Final Output after the Update Phase

Upon completion, a summary of all project files updated or added (or removed, in some cases) will be displayed. All modified (or added) files should be reviewed, after completion of the upgrade run, before the project repository is committed and pushed back to the Azure DevOps Git repository.

Follow-Up Steps After Tool Execution

It is important to be aware of the following when using the upgrade tool:

• The tool applies changes only to the local copy of the project repository; changes are not persisted to the main Git repository until committed and pushed from the workstation manually.

After the tool has completed, the final summary output will list all files that have been, in any manner, modified. These modifications will include new files and changes to existing files (in some cases, files may have even been removed).

Each listed file should be carefully examined to ensure its correctness and to add any additional updates that the tool lacked sufficient information to apply automatically. Once the files have been reviewed and are in their final upgraded form, the changes to the repository should be committed and pushed from the local workstation to Azure DevOps (e.g. via the git commit and git push commands).

The tool makes its best attempt at updating the files correctly; however, all changes should be reviewed by hand and carefully verified before committing and saving into Git. Make sure all your custom changes are not overwritten , also before committing make sure to resolve all conflicts .

Troubleshooting

Most problems that might occur during the execution of the upgrade tool will have self-explanatory error messages to indicate the steps to resolve them. The typical problems to be resolved will depend on the phase of execution of the upgrade tool as follows:

Validate Phase

• If the error indicates that the directory does not appear to contain a valid CloudOne project, check that the upgrade tool was operating on the correct directory and verify its contents
• If the error indicates uncommitted changes in the project repository, check using the git status command (or an equivalent step in a local IDE) to make sure there are no pending updates not yet committed to the repository
• If the error indicates that commits have not been merged and pushed to the Azure Devops centralized copy of the repository, then check the latest commits (e.g. via the git log command) and compare against what is found in Azure Devops (e.g. via the Web UI) and merge any outstanding commits as needed

SetVars Phase

• If an error indicates that this phase was unable to extract a value for a variable, this will usually come in the form of a prompt for the value; either provide the value when prompted or examine the file from which the variable would likely be derived and check for improper YAML syntax that may have prevented the value extraction from completing successfully

Template Phase

• If an error indicates that the retrieval of an application stack's template files failed, check that proper Git credentials were provided
• If the error indicates that despite having proper Git credentials, the application stack could not be retrieved, then check that the correct application stack was derived from the azure-pipelines.yml file

Update Phase

Failures in the Update phase may indicate unexpected conditions which the upgrade logic is not equiped to address. The steps to resolve will depend on the specific error message received and will need to be handled on a case-by-case basis.

Rolling Back

In the event that the upgrade to the project needs to be reversed, the steps to roll back the upgrade will depend on whether or not the updates have been committed to Git. However, in all cases, standard Git tasks and actions will be the method for rolling back.

Rollback of Updates Prior to Commit

In the event that the updates produced by the upgrade tool and/or by manual updates should be backed out, if the updates have not yet been committed to Git, the updates can simply be abandoned. A simple way is to use the git restore or git reset command (which one depends on the version of the git CLI on the local workstation). The git status command will reveal the outstanding updates that need to be reversed that may require, after performing the git restore or git reset command, additional file or directory deletions.

Rollback of Updates After Commit and/or Merge

In the event that the updates produced by the upgrade tool and/or by manual updates should be backed out, if the updates have already been committed to Git, then use the git log command to identify the commit that precedes the updates and the git revert command to roll the project back to that earlier commit level.