# Getting Started

NannyML Cloud SDK is a Python package that enables programmatic interaction with NannyML Cloud. It allows you to automate all aspects of NannyML Cloud, including:

* Creating a model for monitoring.
* Logging inferences for analysis.
* Triggering model analysis

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1uRY74m_HwAjgwE4NmzlFjPh4TAS81tHl?usp=sharing)

If you prefer a video walkthrough, here's our YouTube guide:

{% tabs %}
{% tab title="Azure" %}
{% embed url="<https://youtu.be/P5XppzokFQ0>" %}

{% endtab %}

{% tab title="AWS" %}
{% embed url="<https://youtu.be/6b9TDjyHqTc>" %}
{% endtab %}
{% endtabs %}

## Installation

Currently, the package hasn't been published on PyPI yet, which means you cannot install it via the regular Python channels. Instead, you'll have to install directly from the [repository](https://github.com/NannyML/nannyml-cloud-sdk).

```sh
pip install git+https://github.com/NannyML/nannyml-cloud-sdk.git
```

## Authentication

To use the NannyML Cloud SDK, you need to provide the URL of your NannyML Cloud instance and an API token to authenticate. You can obtain an API token on the [account settings](https://docs.nannyml.com/cloud/v0.21.0/product-tour/account-settings) page of your NannyML Cloud instance.

<figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/giwDdSYwmis4hq6XejdC/image.png" alt=""><figcaption><p>Link to account settings from the header</p></figcaption></figure>

<figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/jIYs02KEXOV6XwwR1Z2i/image.png" alt=""><figcaption><p>API tokens on account settings page</p></figcaption></figure>

After clicking the create button you'll be presented with a prompt to enter an optional description for the API token. We recommend describing what you intend to use the token for so you know which token to revoke later when you no longer need it. Copy the token from the prompt and store it in a secure location.

<div><figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/eg6uw30ZtsjnsKF8xsGe/image.png" alt=""><figcaption><p>Create API token prompt</p></figcaption></figure> <figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/NBL8nNZRL05lDQYJROdT/New%20api%20token.png" alt=""><figcaption><p>Generated token prompt</p></figcaption></figure></div>

Once you have an API token you can use it to authenticate the NannyML Cloud SDK. Either by inserting the token & URL directly into the python code:

```python
import nannyml_cloud_sdk as nml_sdk

nml_sdk.url = "https://beta.app.nannyml.com"
nml_sdk.api_token = r"api token goes here"
```

Or using environment variables:

```python
import nannyml_cloud_sdk as nml_sdk
import os

nml_sdk.url = os.environ['NML_SDK_URL']
nml_sdk.api_token = os.environ['NML_SDK_API_TOKEN']
```

{% hint style="info" %}
We recommend using an environment variable for the API token. This prevents accidentally leaking any token associated with your personal account when sharing code.
{% endhint %}

## Example

The following snippets provide an example of how you can set up the monitoring data and create a model in NannyML Cloud to start monitoring it.&#x20;

To run the example, we will use the synthetic dataset included in the NannyML OSS library, where the model predicts whether a customer will repay a loan to buy a car. Check out [Car Loan Dataset](https://nannyml.readthedocs.io/en/stable/datasets/binary_car_loan.html#dataset-synthetic-binary-car-loan) to learn more about this dataset.

### Step 1: Authenticate and load data

{% code overflow="wrap" %}

```python
import nannyml_cloud_sdk as nml_sdk
import os
import pandas as pd

nml_sdk.url = os.environ['NML_SDK_URL']
nml_sdk.api_token = os.environ['NML_SDK_API_TOKEN']

# Load a NannyML binary classification dataset to use as an example
reference_data = pd.read_csv('https://github.com/NannyML/nannyml/raw/main/nannyml/datasets/data/synthetic_sample_reference.csv')

analysis_data = pd.read_csv('https://github.com/NannyML/nannyml/raw/main/nannyml/datasets/data/synthetic_sample_analysis.csv')

target_data = pd.read_csv('https://github.com/NannyML/nannyml/raw/main/nannyml/datasets/data/synthetic_sample_analysis_gt.csv')
print(reference_data.head())
```

{% endcode %}

### Step 2: Set up the model schema

We use the `Schema` class together with the `from_df` method to set up a schema from the reference data.

In this case, we define the problem as <mark style="color:purple;">'BINARY\_CLASSIFICATION'</mark> but other options like <mark style="color:purple;">'MULTICLASS\_CLASSIFICATION'</mark> and <mark style="color:purple;">'REGRESSION'</mark> are possible.

More info about the `Schema` class can be found in its [API reference](https://nannyml.github.io/nannyml-cloud-sdk/api_reference/monitoring/schema/).

```python
# Inspect schema from dataset and apply overrides
schema = nml_sdk.monitoring.Schema.from_df(
    'BINARY_CLASSIFICATION',
    reference_data,
    target_column_name='work_home_actual',
    ignore_column_names=('period'),
    identifier_column_name='identifier'
)
```

### Step 3: Create the model

We create a new model by using the `create` method. Where we can define things like how the data should be chunked, the main monitoring performance metric, etc.&#x20;

```python
# Create model
model = nml_sdk.monitoring.Model.create(
    name='Example model',
    schema=schema,
    chunk_period='MONTHLY',
    reference_data=reference_data,
    analysis_data=analysis_data,
    target_data=target_data,
    main_performance_metric='F1',
)
```

More info about the `Model` class can be found in its [API reference](https://nannyml.github.io/nannyml-cloud-sdk/api_reference/monitoring/model/).

In case you are wondering why we need to pass the `reference_data` twice —once in the schema and another in the model created— the reason is that both steps are treated differently. In the schema inspection step, we transmit only a few rows (100 to be precise) to the NannyML Cloud server for deriving the schema. While when creating the model, the entire thing is uploaded, so that will take a bit more time.

### Step 4: Ensure continuous monitoring

The previous three steps allow you to monitor an ML on the analysis data previously set. But once new production data is available, you might want to know how your model is performing on it.

You can load any previously set model by searching for it by name. Then, it's a matter of loading the new model predictions, adding them to the model using the method `add_analysis_data`, and triggering a new monitoring run.

```python
# Find the previous model in NannyML Cloud by name
model, = nml_sdk.monitoring.Model.list(name='Example model')

# Add new inferences to NannyML Cloud
new_inferences = pd.DataFrame()
nml_sdk.monitoring.Model.add_analysis_data(model['id'], new_inferences)

# Trigger analysis of the new data
nml_sdk.monitoring.Run.trigger(model['id'])
```

`new_infererences` can be a dataset with several new model inferences:

<figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/6pVrN5qzcfwAgAQBBYLX/Frame%2094.png" alt=""><figcaption></figcaption></figure>

&#x20;or even a single observation:

<figure><img src="https://content.gitbook.com/content/vipr4qR9MrP243sDPhAQ/blobs/mCh83gWZaXvmPcYbnFzN/Frame%2094%20(1).png" alt=""><figcaption></figcaption></figure>

It is also worth noting that you can trigger a monitoring run whenever you want (e.g., after adding 1000 observations) by calling the `trigger` method from the [Run](https://nannyml.github.io/nannyml-cloud-sdk/api_reference/monitoring/run/) class.

### Step 5 (optional): Add delayed ground truth data

If ground truth becomes available at some point in the future, you can add it to nannyML Cloud by using the method `add_analysis_target_data` from the [Model](https://nannyml.github.io/nannyml-cloud-sdk/api_reference/monitoring/model/) class.

<pre class="language-python"><code class="lang-python"><strong># If you have delayed access to ground truth, you can add them to NannyML Cloud
</strong># later. This will match analysis &#x26; target datasets using an identifier column.
delayed_ground_truth = pd.DataFrame()
nml_sdk.monitoring.Model.add_analysis_target_data(model['id'], delayed_ground_truth)

# Trigger analysis of the new data
nml_sdk.monitoring.Run.trigger(model['id'])
</code></pre>