NannyML Cloud
HomeBlogNannyML OSS Docs
v0.24.2
v0.24.2
  • ☂️Introduction
  • Model Monitoring
    • Quickstart
    • Data Preparation
      • How to get data ready for NannyML
    • Tutorials
      • Monitoring a tabular data model
      • Monitoring with segmentation
      • Monitoring a text classification model
      • Monitoring a computer vision model
    • How it works
      • Probabilistic Adaptive Performance Estimation (PAPE)
      • Reverse Concept Drift (RCD)
    • Custom Metrics
      • Creating Custom Metrics
        • Writing Functions for Binary Classification
        • Writing Functions for Multiclass Classification
        • Writing Functions for Regression
        • Handling Missing Values
        • Advanced Tutorial: Creating a MTBF Custom Metric
      • Adding a Custom Metric through NannyML SDK
    • Reporting
      • Creating a new report
      • Report structure
      • Exporting a report
      • Managing reports
      • Report template
      • Add to report feature
  • Product tour
    • Navigation
    • Adding a model
    • Model overview
    • Deleting a model
    • Model side panel
      • Summary
      • Performance
      • Concept drift
      • Covariate shift
      • Data quality
      • Logs
      • Model settings
        • General
        • Data
        • Performance settings
        • Concept Drift settings
        • Covariate Shift settings
        • Descriptive Statistics settings
        • Data Quality settings
    • Account settings
  • Deployment
    • Azure
      • Azure Managed Application
        • Finding the URL to access managed NannyML Cloud
        • Enabling access to storage
      • Azure Software-as-a-Service (SaaS)
    • AWS
      • AMI with CFT
        • Architecture
      • EKS
        • Quick start cluster setup
      • S3 Access
    • Application setup
      • Authentication
      • Notifications
      • Webhooks
      • Permissions
  • NannyML Cloud SDK
    • Getting Started
    • Example
      • Authentication & loading data
      • Setting up the model schema
      • Creating the monitoring model
      • Customizing the monitoring model settings
      • Setting up continuous monitoring
      • Add delayed ground truth (optional)
    • API Reference
  • Probabilistic Model Evaluation
    • Introduction
    • Tutorials
      • Evaluating a binary classification model
      • Data Preparation
    • How it works
      • HDI+ROPE (with minimum precision)
      • Getting Probability Distribution of a Performance Metric with targets
      • Getting Probability Distribution of Performance Metric without targets
      • Getting Probability Distribution of Performance Metric when some observations have labels
      • Defaults for ROPE and estimation precision
  • Experiments Module
    • Introduction
    • Tutorials
      • Running an A/B test
      • Data Preparation
    • How it works
      • Getting probability distribution of the difference of binary downstream metrics
  • miscellaneous
    • Engineering
    • Usage logging in NannyNL
    • Versions
      • Version 0.24.2
      • Version 0.24.1
      • Version 0.24.0
      • Version 0.23.0
      • Version 0.22.0
      • Version 0.21.0
Powered by GitBook
On this page
  • Introduction
  • Prerequisites
  • Configuration
  • Triggers and payloads
  1. Deployment
  2. Application setup

Webhooks

This page shows how to integrate NannyML to external applications by using webhooks.

PreviousNotificationsNextPermissions

Introduction

Webhook integration in NannyML Cloud enables real-time communication with external systems. When specific events occur in NannyML Cloud, it triggers webhooks—simple HTTP requests sent to URLs you define. This setup is key for automating tasks, syncing data, or triggering actions in other applications seamlessly.

To integrate webhooks, you'll set up the target URLs for these requests and, if needed, configure additional request headers, e.g. for authentication to access a secured endpoint.

Prerequisites

  • You have a server application that can handle HTTP POST requests with a JSON body of a specific format, hence named the webhook server.

  • The webhook server can be reached over a (private) network by your NannyML Cloud instance.

  • Basic auth credentials or an authentication token in case the webhook server endpoint is secured.

Configuration

  1. In the Webhook section of the application setup wizard, select the Custom webhook option.

  2. In the custom webhook configuration screen, provide values for the following settings:

    1. Address: this is the full path to the webhook server endpoint that will handle the POST request sent by NannyML Cloud.

    2. Additional headers: in case additional headers are required for routing, authentication, or any other purpose, their keys and values can be configured here. After providing a key-value pair in the designated input fields, click the "plus" button to add them to the configuration. Not clicking the "plus" button will result in these values getting lost when you progress the wizard. Any added key-value pair can be removed by using the "minus" button.

  1. Use the Validate webhook button to send a test request using the provided configuration. If the webhook server returns a HTTP status code of 200, the webhook is considered valid. After validation, the Next button is enabled, and you can progress to the next section of the wizard. If any other HTTP status code is returned, the webhook is considered invalid. The error message returned will be shown on the screen. To aid with debugging, we've included the validation request on the page. You will not be able to progress to the next section of the wizard when webhooks are enabled, but the configuration cannot be validated.

Triggers and payloads

In NannyML Cloud a webhook is considered to be a notification channel. A notification will be sent, using the configured channels, when certain events occur in NannyML Cloud. Each notification sent is of a certain notification type, associated with the triggering event. Notifications will differ based on the channel they are sent over (e.g. email, webhook, ...) and the notification type. For webhooks each notification type is associated with a certain payload. This is essentially a JSON representation of data relevant to the trigger event.

Trigger

This notification triggers whenever a run is completed. This includes runs that fail.

Payload example

{
  "header": {
    "model_name": "test",
    "model_id": 28740,
    "notification_type": "RUN_COMPLETED"
  },
  "payload": {
    "run": {
      "started_at": "2024-01-25T14:04:10.157236"
    },
    "performance_alerts": {
      "cbpe": [
        {
          "metric_name": "specificity"
        }
      ]
    },
    "other_alerts": {
      "univariate_drift": [
        {
          "metric_name": "country"
        }
      ]
    },
    "success": [
      "unseen_values",
      "missing_values"
    ],
    "report_link": "https://test.app.nannyml.com/model/1"
  }
}

Custom webhooks configuration.
Custom webhook configuration values.
A validated custom webhook configuration.