# Monitoring a text classification model In this tutorial, we will use nannyML cloud to monitor a sentiment analysis text classification model where the model's goal is to predict the sentiment (Negative, Neutral, Positive) of a review left on Amazon. ## The model and dataset We will use a model trained on a subset of the [Multilingual Amazon Reviews Dataset](https://huggingface.co/datasets/amazon_reviews_multi). The trained model can be found in the [nannyML's hugging face hub](https://huggingface.co/NannyML/amazon-reviews-sentiment-bert-base-uncased-6000-samples). For details of how this model was produced, check out the blog post: [Are your NLP models deteriorating post-deployment? Let’s use unlabelled data to find out](https://huggingface.co/blog/santiviquez/performance-estimation-nlp-nannyml). ## Reference and analysis sets To evaluate the model in production, we have two sets: * **Reference set** - which contains all model inputs along with the model’s predictions and labels. This set establishes a baseline for every metric we want to monitor. Find the reference set: * **Analysis set** - which contains all model inputs extracted from a production set with the model’s prediction, and in this case, labels. The analysis set is where NannyML analyzes/monitors the model’s performance and data drift of the model using the knowledge gained from the reference set. Find the analysis set: ## Monitoring with nannyML Cloud ### Step 1: Add a new model Click the Add model button to create a new model on your nannyML cloud dashboard.

### Step 2: Define the problem type and main metric Each review that we are classifying can be Negative, Positive, or Neutral. For this reason, we will set the problem type as **Multiclass classification.** We will be monitoring the model's F1-score on a weekly basis.

### Step 3: Configure the Reference set Select "Upload via public link".

Use the following public URL to link the Reference dataset:

### Step 4: Define the reference dataset schema 1. Select the column **timestamp** as the Timestamp column 2. Select the column **predicted\_sentiment** as the Prediction column 3. Select the **real\_sentiment** as the Target column 4. Flag the columns **negative\_sentiment\_pred\_proba, neutral\_sentiment\_pred\_proba, positive\_sentiment\_pred\_proba** as Prediction Scores.

### Step 5: Configure the Analysis set Select "Upload via public link".

Use the following public URL to link the Analysis dataset:

### Step 6: Start monitoring

--- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.nannyml.com/cloud/v0.22.0/model-monitoring/tutorials/monitoring-a-text-classification-model.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.