Route Definition

An application in Kubernetes can be exposed and made accessible using an "Ingress" routing layer. The primary way that this is provided in CloudOne is using Ambassador, an edge inbound proxy service. Up until version 3.7 of the platform, another method was the use of OpenShift Routes, which is a feature uniquely tied to the OpenShift platform of Kubernetes. However, as of version 3.7, OpenShift Routes are no longer supported, they are unique to the OpenShift platform while Ambasador mappings work across multiple Kubernetes platforms. Note also that Ambassador mappings have additional advanced routing configuration options, which will be described in this document.

The definition of routes or mappings is driven by settings in the Helm charts for an application that define YAML values. While these can be defined anywhere within the Helm chart structure, normally these will be defined in the values.yaml or one of the values.*.yaml files. As these files are associated with certain deployment environments or sets of deployment environments, these are the most appropriate places in which to define the settings.

Host part of URLs

If no additional customizations are applied to the definition of the URL route, the host part of the exposed URL is defined as follows:

• The host portion of the URL will be an extended service name composed of the application name, application code and space name all separated by hyphens

• The extended service name, referenced below as an extended-service-name, then looks like the following: AppName-AppCode-SpaceName, where

  • AppName is the name of the application or service
  • AppCode is the (usually) 3-letter unique application code
  • SpaceName is the namespace, i.e. name of the workspace or hostspace
• In the case of a Workspace, the host part will also have a unique build identifier (based on the git pull request) appended after the name of the application or service, which will then look like the following: AppName-BuildID-AppCode-SpaceName, which will be referenced below as an extended-service-and-branch-name)

After the extended service name, the rest of the fully qualified domain name (FQDN) will depend on the environment and whether the URL is exposed as an Ambassador mapping or as an OpenShift route.

Regardless of how the URL is exposed, the Azure DevOps CI/CD pipeline will populate a tab within the CI/CD pipeline web user interface listing the URLs exposed for the deployed service. The tab is named Extensions (located near the top of the web page) and, when selected, lists all URLs by environment (both Ambassador mappings as well as OpenShift routes).

Ambassador Mapping

When using Ambassador, there are two Ambassador proxy infrastructures available for web services:

• internal - for services to be exposed within the Netapp/CloudOne network
• external - for services to be exposed outside of the network

If the YAML value ambassador.internal.enabled is set to true in the Helm charts, then an Ambassador route mapping will be created on the internal network, under the subdomain: prd01i.cloudone.netapp.com for workspace and for the first stage and production spaces (stg-1 and prd-1) and under the subdomain: prd02i.cloudone.netapp.com for the second stage and production spaces (stg-2 and prd-2).

If ambassador.external.enabled is set to true then an Ambassador route mapping will be created and exposed on the external network, under the subdomain: prd01e.cloudone.netapp.com for the first stage and production spaces (stg-1 and prd-1) and under the subdomain: prd02e.cloudone.netapp.com for the second stage and production spaces (stg-2 and prd-2).

Note that URLs exposed in workspaces are only exposed internally. Externally exposing workspace services is not an available option.

Ambassador Mapping URLs

The following table summarizes the resulting URLs for Ambassador mappings of web services (where <extended-service-name> is the extended service name as described above):

Environment	Internal URL	External URL
workspace	<extended-service-and-branch-name>.prd01i.cloudone.netapp.com	N/A (workspace URLs are not exposed externally)
stg-1	<extended-service-name>.prd01i.cloudone.netapp.com	<extended-service-name>.prd01e.cloudone.netapp.com
stg-2	<extended-service-name>.prd02i.cloudone.netapp.com	<extended-service-name>.prd02e.cloudone.netapp.com
prd-1	<extended-service-name>.prd01i.cloudone.netapp.com	<extended-service-name>.prd01e.cloudone.netapp.com
prd-2	<extended-service-name>.prd01i.cloudone.netapp.com	<extended-service-name>.prd01e.cloudone.netapp.com

Ambassador Mapping Configuration

Additional custom configurations can be applied to an Ambassador mapping by modifying one of the following YAML values also included in the values.*.yaml files:

Ambassador YAML value	Description
prefix	The URI resource path under the host part of the Ambassador URL which will map to the Kubernetes service exposed. Though not strictly required, it is almost always best practice to define a prefix to include the trailing / to avoid matching on partial strings by accident.
domain	The domain under which the host header of a request must match in order to match, in turn, the Ambassador mapping (not generally subject to change)

Custom Ambassador Mapping Configuration Options

Any additional custom changes to the Ambassador mappings would require applying the changes directly to the Helm chart template at templates/ambassador.yaml. Additional mapping rules that can be added, if needed, might include the following:

Ambassador Mapping Attribute	Description
method	HTTP call method to limit matches to a given mapping (e.g. POST, GET, PUT, etc.)
retry_policy	a YAML structure to define a policy for automatic retries, where the response to trigger a retry, the number of retries and timeout period per retry (retryon, numretries and pertrytimeout respectively) are defined in the YAML structure
circuit_breakers	a YAML structure to define a circuit breaker where a certain number of failures will cause Ambassador to stop sending traffic to the selected mapping (often works with multiple mappings to alternate instances of the same service so that a tripped circuit breaker will simply ensure traffic continues to be sent to a surviving path)
cors	a YAML structure to define Cross-Origin Resource Sharing (CORS) filters and rules
headers	a YAML structure listing headers and matching values that will be required for a mapping to match an inbound request
prefix_regex	when set to true, treats the value of prefix as a regular expression to match URI patterns with greater flexibility
protocols	special values associated with mapping for alternate traffic protocols other than HTTP traffic
host_redirect	when set to true, will cause an HTTP 301 redirect to a URL specified in the value of service and if path_redirect is set, the URI resource path will be rewritten to the value specified for path_redirect
rewrite	when specified, replaces the URI resource path with the specified path
regex_rewrite	works like rewrite but (as a YAML structure) with a more powerful and flexible matching to a regular expression pattern specified in pattern and replacement pattern specified in substitution
timeout_ms	milliseconds to wait before timing out on attempt to receive response from mapped resource
connecttimeoutms	milliseconds to wait before timing out on attempt to connect to a mapped resource
shadow	when set to true the mapping is a "shadow" mapping that takes a copy of every request sent downstream to another service based on another mapping, copying all traffic of a subset based on the value of weight to set a percentage of traffic to shadow
weight	though normally set automatically and implicitly for canary-based deployments, this value defines a percentage of traffic to be accepted by a given mapping (used in conjunction with having multiple mappings defined over which to distribute traffic)
case_sensitive	normally the string matching for the prefix value is case sensitive; if this value is set to false then the matching will be case-insensitive
precedence	if set, will ensure that the mapping will be matched earlier than other mappings that otherwise match; precedence is in descending order so that higher integer values match earlier in the ordering algorithm

For more details on the advanced Ambassador mapping specifications, refer to the Ambassador documentation at: https://www.getambassador.io/docs/latest/topics/using/mappings/

OpenShift Route

If still using OpenShift routes (prior to verion 3.7 of the CI/CD pipeline), this is defined under the templates/route.yaml file of the Helm chart and enabled by having the YAML variable route.enabled set to true in an appropriate override file.

OpenShift Route URLs

The following table summarizes the resulting URLs for OpenShift routes (where <extended-service-name> is the extended service name as described above):

Environment	Exposed URL
workspace	N/A (workspace URLs are not exposed as OpenShift routes which are externally accessible)
stg-1	<ext-service-name>.east1.ncloud.netapp.com
stg-2	<ext-service-name>.npc-us-west-dc61.ncloud.netapp.com
prd-1	<ext-service-name>.east1.ncloud.netapp.com
prd-2	<ext-service-name>.npc-us-west-dc61.ncloud.netapp.com

Note that URLs exposed as OpenShift routes are externally accessible, and since services deployed into workspaces are not intended to be externally accessed, OpenShift routes are not created for services deployed into workspaces (Ambassador mappings will provide an internally-exposed option instead).

Custom OpenShift Route Configuration Options

Custom modifications and configuration settings can be set for the route following normal Kubernetes route definition rules in the templates/route.yaml file of the Helm chart.

Some examples of additional configuration options for OpenShift Routes include the following items, configured under the route's spec YAML structure:

OpenShift Route spec Attribute	Description
tls	YAML structure in which alternate TLS information (certificate and termination details) can be defined
port	YAML structure in which an alternative targetPort value can be specified for reaching an OpenShift service
to	YAML structure defining the target for the route and within which a weight can be defined to accept a portion of traffic sent to the route
alternateBackends	YAML structure defining a list of additional targets for the route where each alternate has a weight that can be defined to accept a portion of traffic sent to the route
wildcardPolicy	References a policy configured into an OpenShift router for handling of wildcard URLs to map to an OpenShift route

Advanced OpenShift Route Configuration Options

Additional advanced modifications and attributes can be set for the route in the templates/route.yaml file of the Helm chart by modifying the metadata YAML structure within the route YAML definition, with some examples as follows:

OpenShift Route metadata Attribute	Description
annotations	The annotations YAML structure essentially provides an extension ro the route definition to configure more advanced features of OpenShift routes (that are not typically required) and which are described in more details below
labels	If there are multiple routers configured for the OpenShift environment, selector labels can be defined within the Labels YAML structure to select the appropriate router

Some examples of the more advanced configuration features available for OpenShift routes under the annotations YAML include the following:

OpenShift Route metadata.annotations Attribute	Description
haproxy.router.openshift.io/balance	sets the load balancer algorithm - available values are: roundrobin, source or leastconn
haproxy.router.openshift.io/rate-limit-connections	set to true to enable rate limiting
haproxy.router.openshift.io/rate-limit-connections.concurrent-tcp	when rate limiting is enabled, sets limit on number of concurrent TCP connections per IP address
haproxy.router.openshift.io/rate-limit-connections.rate-http	when rate limiting is enabled, sets the maximum rate that an IP address can accept HTTP requests
haproxy.router.openshift.io/rate-limit-connections.rate-tcp	when rate limiting is enabled, sets the maximum rate that an IP address can initiate TCP connections
haproxy.router.openshift.io/timeout	sets a timeout for the route
haproxy.router.openshift.io/ip_whitelist	if set, limits access to the route to the specified list of IP addresses or address ranges (specified in CIDR notation)
router.openshift.io/haproxy.health.check.interval	set the interval for health checks of the backend service

For more details on the advanced OpenShift Route specifications, refer to the OpenShuft documentation at: https://docs.openshift.com/container-platform/3.11/architecture/networking/routes.html