EKS

Deployment instructions for NannyML Cloud on AWS EKS

NannyML Cloud is available on the AWS marketplace as a helm chart that can be installed on a Kubernetes cluster. This page provides deployment instructions and describes configuration options for customizing your NannyML Cloud instance.

If you prefer a video walkthrough, here's our AWS Setup YouTube guide:

Prerequisites

Before deploying NannyML Cloud in your AWS account, you will need to prepare:

• A few tools to configure the Kubernetes cluster

  • AWS CLI configured with appropriate credentials for the account you wish to use

  • eksctl to manage the EKS cluster

  • kubectl to read information from the EKS cluster

  • helm to install the NannyML Cloud helm chart

If you don't already have these tools installed locally, we recommend using AWS CloudShell. It comes with the AWS CLI and kubectl preinstalled and configured. Using AWS CloudShell you will only need to install eksctl and helm using the commands below to get started.

# Install helm
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash

# Install eksctl
PLATFORM=$(uname -s)_amd64
curl -L "https://github.com/eksctl-io/eksctl/releases/latest/download/eksctl_$PLATFORM.tar.gz" | tar -xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin

Note AWS CloudShell may ask you for confirmation to execute multiple lines when copy pasting this command. This can safely be accepted.

• An Amazon EKS cluster

  • The AWS documentation provides a step-by-step guide to create a cluster if you don't have one yet.

  • Node size is free to choose, but we recommend 2 nodes using t3.large or t3a.large instance as minimum.

  • The cluster can be shared with other applications if desired.

• Amazon EBS CSI driver on the cluster

  • The databases used by NannyML Cloud are run inside of the cluster. Data is persisted in Elastic Block Storage.

Just want to try things out using the default configuration? Use our quick start instructions to set up a cluster.

AWS Marketplace subscription

To access NannyML Cloud, you will need to subscribe to one of our offers on the AWS marketplace. Currently we provide two offers:

• NannyML Cloud Trial: NannyML Cloud configured to use a 30-day scale trial license. It is free to subscribe and use, but once the trial license expires you will need to purchase a license to continue using the software.

• NannyML Cloud: NannyML Cloud using a contract license model. To subscribe you need to purchase a license in the AWS marketplace.

To get started, click the Continue to Subscribe button on the offer of your choice.

Subscription page on AWS Marketplace

Configure license

If you've opted for the NannyML Cloud offer with free trial, you can skip this step. Otherwise you will be shown a contract page where you can configure your license by specifying the number of units you want to purchase:

• Starter, Scale or Enterprise license: the license required to run NannyML Cloud. One license is required for each instance of NannyML Cloud you want to run. See our pricing table to understand what's included.

• Extra models: the enterprise license includes capacity for monitoring 10 models. If you want to monitor more models, enter the number of additional model monitoring licenses you want to purchase. These licenses are floating, meaning if you have 2 NannyML Cloud instances, they can both use the available licenses as models are added to the application.

Don't know yet how many models you will need? Don't fret, you can always come back and edit your license in the AWS marketplace or add a new license contract.

We advise to renew your license automatically every month. Should you choose not to, your NannyML Cloud instances will stop working when the licenses expire. You can then purchase a new license to restore your existing NannyML Cloud instances.

Contract page on AWS Marketplace

Identify software version

Select the Helm chart option for NannyML Cloud and find the software version you wish to use. We recommend using the latest version available for the best user experience. Then follow the instructions in launching the software to create your NannyML Cloud instance.

Configuration page on the AWS Marketplace

Launching the software

Step 1: Create an AWS IAM role and Kubernetes service account

To deploy NannyML Cloud from AWS Marketplace, you need to create a Kubernetes service account with appropriate IAM permissions for communicating with AWS License Manager. When launching the software it will attempt to acquire a license from AWS on start-up. In case no license is available, the software will fail to start.

To set up the service account you'll need to execute these steps:

• Create an IAM role with AWS-managed AWSLicenseManagerConsumptionPolicy.

• Create a K8s service account named nannyml-cloud-service-account in your Amazon EKS cluster.

• Set up a trust relationship between the created IAM role with nannyml-cloud-service-account.

• Modify nannyml-cloud-service-account annotation to associate it with the created IAM role.

Your Amazon EKS cluster needs to have an IAM OIDC provider enabled to associate a Kubernetes service account with an IAM role. If you've installed the Amazon EBS CSI driver prerequisite this should already be enabled. See Creating an IAM OIDC provider for your cluster for more information.

We recommend doing this via eksctl. The commands below automate these steps. Start by defining the EKS cluster name.

CLUSTER_NAME=nannyml-cloud-demo

Then run the command below to create a K8s service account with access to AWS License Manager.

eksctl create iamserviceaccount \
    --name nannyml-cloud-service-account \
    --namespace nannyml-cloud \
    --cluster $CLUSTER_NAME \
    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AWSLicenseManagerConsumptionPolicy \
    --approve \
    --override-existing-serviceaccounts

Step 2: Download NannyML Cloud helm chart

Define which version you would like to install. The list of available versions can be found on the configure step in AWS Marketplace. We recommend installing the latest available version for the best user experience.

NANNYML_CLOUD_VERSION=0.24.2

Login to grant helm access to the images hosted on AWS Marketplace, then pull the appropriate image.

NannyML Cloud (paid license)

aws ecr get-login-password \
    --region us-east-1 | helm registry login \
    --username AWS \
    --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com

helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/nannyml/nannyml-cloud/nannyml-cloud-operator-aws \
    --version ${NANNYML_CLOUD_VERSION}

NannyML Cloud Trial

aws ecr get-login-password \
    --region us-east-1 | helm registry login \
    --username AWS \
    --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com

helm pull oci://709825985650.dkr.ecr.us-east-1.amazonaws.com/nannyml/nannyml-cloud-trial/nannyml-cloud-operator-aws-trial \
    --version ${NANNYML_CLOUD_VERSION}

Step 3: Deploy NannyML Cloud with attached IAM role

Optionally define a custom URL where you want to make the application available. If provided, we'll configure an ingress and attempt to get a certificate for the domain. Once the application is deployed, you will need to make a DNS registration so that the URL points at the appropriate AWS elastic load balancer. If you do not provide a URL, we'll use a self-signed certificate to enable HTTPS on the AWS elastic load balancer URL.

NANNYML_CLOUD_HOSTNAME=

Deploy NannyML Cloud using the following command:

NannyML Cloud (paid license)

helm upgrade -i nannyml-cloud nannyml-cloud-operator-aws-${NANNYML_CLOUD_VERSION}.tgz \
    --namespace nannyml-cloud \
    --create-namespace \
    --set operator.app.spec.hostname=${NANNYML_CLOUD_HOSTNAME} \
    --set operator.app.spec.server.serviceAccountName=nannyml-cloud-service-account

NannyML Cloud Trial

helm upgrade -i nannyml-cloud nannyml-cloud-operator-aws-trial-${NANNYML_CLOUD_VERSION}.tgz \
    --namespace nannyml-cloud \
    --create-namespace \
    --set operator.app.spec.hostname=${NANNYML_CLOUD_HOSTNAME} \
    --set operator.app.spec.server.serviceAccountName=nannyml-cloud-service-account

The serviceAccountName argument provided has to match the service account name created earlier. If you used a different name in step 1, you will also need to update the argument here.

It usually takes around 2 minutes for the helm command to complete. When completed the kubernetes cluster will spin up all the required containers and allocate block storage on AWS. It will take another 2-3 minutes for the NannyML Cloud instance to be fully operational.

Step 4: Accept trial license

If you opted to purchase a license for NannyML Cloud, it will already be active and you can skip this step. If you're using a trial license, the license is issued to you when the software is first deployed. Depending on the policy configured for your AWS account, it may need to be accepted in the AWS environment before it can be used.

Access the list of licenses granted to your account on AWS License Manager. You should see a license for NannyML Cloud issued by NannyML as shown in the screenshot below. When the license is newly issued, it will be in Pending acceptance status.

Granted trial license in AWS License Manager

It may take a few minutes for the license to become available as it is granted when your NannyML Cloud instance has started.

Click the License ID column to view details for the license. Next click the Accept & activate license button in the top right and follow the instructions in the pop-up to activate your trial license.

Trial license details

The trial license comes with a few limitations:

• Only one trial license is allowed per account. Once it has expired, you will need to purchase a license.

• Delaying the license activation does not change the expiration date. It will expire 30 days from when it was issued, regardless of activation.

• Only one NannyML Cloud instance can be active at a time using the trial license. When purchasing a license, you can opt to buy multiple licenses, enabling multiple instances.

Step 5: Access your NannyML Cloud instance

Your NannyML Cloud instance should now be active on your EKS cluster. It is automatically exposed using an Amazon ELB (Elastic Load Balancer) with a dynamically generated URL. Run the command below to identify the ELB URL for your instance.

kubectl get ingress nannyml-cloud-ingress \
    -n nannyml-cloud \
    -o=jsonpath={.status.loadBalancer.ingress[0].hostname}

Custom URL

If you entered a custom URL in step 3, you should now set up a DNS CNAME record to direct your chosen URL to the ELB URL generated by AWS. Exact steps for this depend on your DNS provider. If you're unfamiliar, please check your DNS provider's documentation or ask your IT administrator for help.

Once the CNAME record has been created a certificate will be generated by Let's Encrypt and stored on the kubernetes cluster. This process could take a few minutes. Once completed, your NannyML Cloud instance is available at your configured URL using a secured HTTPS connection.

Generated URL

Access the application at the returned URL, e.g. https://a424a15820a2e40b9b2395b54299d674-444730227.eu-central-1.elb.amazonaws.com/

⚠️ When accessing the dynamically generated URL you may see a security warning such as shown below. This is expected because the application is using a self-signed certificate for that URL. You can proceed to the application by bypassing the warning in your browser.

Security warning when accessing application using self-signed certificate

Unfortunately we cannot get a certificate from a certificate authority for these URL's as they are ephemeral. If you were to shut down your NannyML Cloud instance, AWS may assign that URL to someone else. If you had received a certificate for it, you would now hold a certificate to someone else's website which is of course undesired.

The solution to this issue is setting a custom URL in step 3 when configuring your NannyML Cloud instance. With a URL you control we can obtain a certificate from Let's Encrypt and have a secure HTTPS setup.

Updating the software

Once NannyML Cloud has been deployed on EKS, you can update to a new version by deploying a new helm chart on the kubernetes cluster. To install an update:

1. Make sure you have an environment available with the kubernetes tools described in the prerequisites section, or install them into AWS CloudShell.

2. Download the helm chart for the new version as described in step 2.

3. Deploy the new version as described in step 3.

   • Specify the same hostname as in the original deployment. If you specify a different hostname, the application will be exposed using the new URL. You may then need to change how you access the application as described in step 5.

4. Optionally verify the new version is active in the UI. The helm chart version should match the version number displayed.

   

   Version info in NannyML Cloud

After a few minutes the new version should be operational. If you used the same URL as in the original deployment, you can continue to use your NannyML instance without any changes.

Your NannyML Cloud instance will remain available using the old version during the update process. Once the new version is ready, kubernetes will switch over network traffic to the new version without any downtime. However, any active NannyML runs at the time of update will be cancelled. Those runs will then need to be restarted after the update.

We try to ensure all updates are backwards compatible so that all of your data will be retained. However, some updates may require significant changes that are not fully backwards compatible. This will be clearly specified in the release notes, including a description of how to update your existing models in NannyML Cloud so that no data is lost.

Getting a new license when your (trial) license expires

When your (trial) license expires your NannyML Cloud instance will shut down. All data will be preserved and remain unchanged from the moment the license expires. You will then need to purchase a new license for your AWS account to restore operation.

Subscribe to the NannyML Cloud offer on the AWS Marketplace, then follow the instructions to configure a new license. Once an appropriate license is available in your AWS account, your existing NannyML Cloud instance will recover and become available again. Re-deploying using instructions for the paid NannyML Cloud offer is possible, but not required as the software contained in both offers will behave identical.