Summary
In this code pattern, learn how to use the Cortex Certifai Toolkit to create scans to evaluate the performance of multiple predictive models using IBM Watson Studio.
Description
Explainability of AI models is a difficult task that is made simpler by Cortex Certifai. The Cortex Certifai Tookit evaluates AI models for robustness, fairness, and explainability, and allows users to compare different models or model versions for these qualities. Certifai can be applied to any black-box model including machine learning models and predictive models, and works with a variety of input data sets.
Data scientists can create model scan definitions, which are comprised of trained models that you want to evaluate for the following parameters:
Performance metric (for example, Accuracy)
Robustness: How the model generalizes on new data
Fairness by group, which measures the bias in the data
Explainability, which measures the explanations provided for each model
Explanations, which display the change that must occur in a data set with given restrictions to obtain a different outcome
Business decision makers can view the evaluation comparison through visualizations and scores to select the best models for business goals and to identify whether models meet thresholds for robustness, fairness, and explainability. Data scientists can use the evaluation results for analysis to provide more trustworthy AI models.
This code pattern demonstrates how to use the Certifai Toolkit to create scans to evaluate the performance of multiple predictive models using the IBM Watson Studio platform.
Flow
Log in to IBM Watson Studio powered by Spark, initiate IBM Cloud Object Storage, and create a project.
Upload the .csv data file to IBM Cloud Object Storage.
Load the data file in the Watson Studio notebook.
Install the Cortex Certifai Toolkit in the Watson Studio notebook.
Get visualization for explainability and interpretability of the AI model for the three different types of users.
Instructions
Find the detailed steps in the README file. Those steps explain how to:
Create an account with IBM Cloud.
Create a new Watson Studio project.
Add data.
Create the notebook.
Insert the data as DataFrame.
Run the notebook.
Analyze the results.