Concept shift

The Concept drift dashboard enables you to analyze the impact and magnitude of concept shift. To calculate these results, NannyML Cloud requires the analysis set to include a ground truth column.

Note: Concept shift detection is currently supported only for binary classification use cases. For custom support in regression use cases, please contact us on our website.

If you prefer a video walkthrough, here is our guide explaining how to use the concept shift page:

Here, you can find detailed descriptions of various elements on the concept shift page:

Concept shift dashboard.

The Concept shift dashboard consists of three main components:

1. Filters

1.1 Metrics

Select which metrics you want to have displayed on the performance impact graph. Additionally, a magnitude metric illustrates the magnitude of concept drift at specific periods.

Concept shift metrics filter.

1.2 Alert status

Choose which metrics to display based on whether there are no alerts, alerts in any or only the last chunk, alerts in performance metric with the main tag, or include all charts regardless of when and if any alerts occurred.

Alert status filter.

1.3 Tags

Filter charts by the previously specified tags.

Tags filter.

2. Visualizations

2.1 Sort by

You can change the order of charts based on the metric name, number, or recency of the alerts.

Sort by window.

2.2 Date range

You can select a specific period of interest which applies to all charts.

Date range window.

2.3 Date reset

To reset a previously set date period, whether using the date range or slider, simply press the "Reset" button.

Date range reset.

2.4 Date slider

Similar to selecting a date range, you can choose a specific period of interest by simply moving the date slider.

Date slider.

2.5 Chart

The charts are interactive, allowing you to hover over them for more details. Red dotted lines indicate the thresholds, while the blue line shows the metric during the reference period. The light blue line represents the metric during the analysis period. The lightly shaded area around the performance impact estimation is the metric's confidence band.

You can also zoom in on any part of a chart. Simply press and hold your mouse button, then draw a square over your area of interest. To reset the zoom, just double-click on the chart.

Concept shift chart.

3. Plots

3.1 Plot format

There are two types of plot formats: line and step. A line plot smoothly connects points with straight lines to show trends, while a step plot uses sharp vertical and horizontal lines to show exact changes between points clearly.

Plot format.

3.2 Datasets

Select datasets to zoom in on reference, analysis, or create a separate subplot for both.

Datasets plots.

3.3 Plot elements

Toggle on or off some components on the charts, like alerts, confidence bands, thresholds, and legends.

Plots elements.