# Defaults for ROPE and estimation precision ## When are ROPE and Estimation Precision automatically calculated? ROPE and estimation precision are typically business-related. ROPE is defined as a region with a similar business impact to the expected impact. Precision should be determined so that it limits the probability of getting a false positive (falsely accepting the hypothesis, that is - claiming the population performance is within ROPE) or false negative probability down to the business-required level. It is sometimes difficult to provide those parameters, yet it is still worth running a Probabilistic Model Evaluation. For those cases, NannyML provides reasonable defaults for ROPE and estimation precision. ## Default ROPE Default ROPE reflects the hypothesis that an ML model performance *is no worse than the one from reference data*. In practice, default ROPE spans from the left 95% HDI edge from reference performance posterior to the maximum value of the metric possible (1 for the metrics currently supported). Figure 1 shows the default ROPE.

Figure 1. Default ROPE based on reference data performance metric posterior.

## Default precision The default precision is calculated to ensure the experiment's power is 0.8, with the experiment's goal to get a conclusive answer (to accept or reject the hypothesis). At the default precision, the experiment will yield a conclusive answer with 80% probability. The process of estimating the default precision is the following: 1. Assume that the hypothesis is correct: the performance metric is within ROPE. 2. Sample performance metric uniformly from ROPE. 3. Generate *n* observations of data that we could observe given the performance metric sampled (similarly to predictive posterior sampling). 4. Get posterior from the sampled observations. 5. Calculate the HDI of the posterior from 4. 6. Check whether HDI is fully within the ROPE. 7. Repeat steps 2-6 multiple times. Store the result from step 6. 8. Check if \~80% of the experiment results (step 6) give a conclusive answer. 9. Repeat steps 2-8 to find n, for which 80% of experiments give a conclusive, positive answer. 10. Repeat the process with the assumption that the hypothesis should be rejected (that is - sample performance metric from outside of ROPE) to find *n* for this assumption. Pick the larger *n*. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.nannyml.com/cloud/probabilistic-model-evaluation/how-it-works/defaults-for-rope-and-estimation-precision.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.