15  Self study - Session 2

CautionStill under construction

This section is still under construction and will be completed in the near future. Please do not go beyond this point for now.

In this session, we will transfer what we learned in Chapter 13 to a new dataset, the Explainable AI (XAI) Drilling Dataset by Wallsberger et al. (2023). We will explore it in depth, splitting our analysis into four milestones, each building on the previous one.

15.1 Milestone 1: Data Acquisition and Understanding

Exercise 15.1 (Register and download the dataset)  

  1. Register on Kaggle.
  2. Read the description of the Explainable AI (XAI) Drilling Dataset.
  3. Download the dataset.
  4. Load the dataset into your environment and inspect its structure: number of rows, columns, data types, and any obvious issues.
  5. Identify the target variable(s) and any class imbalances.
  6. Check the consistency of the labels: does the main failure flag coincide with the union of the subgroup failure flags? Are there samples with more than one subgroup failure?
  7. Run the same check on the AI4I dataset from Chapter 13. What do you find, and how would you handle inconsistent labels when training a model?

15.2 Milestone 2: Exploratory Data Analysis

Exercise 15.2 (Explore the dataset)  

  1. Pretend that you did not read the Subgroup failures paragraph in the description. Explore the dataset and find insights related to the different failure types.
  2. Do your findings align with the description given in the Subgroup failures paragraph?
  3. Compute summary statistics for each feature grouped by failure type.
  4. Create at least two visualizations (e.g., histograms, box plots) that reveal differences between failure types.

15.3 Milestone 3: Deep Dive Visualization

Exercise 15.3 (Visualize failure patterns with plotly)  

  1. Create a parallel coordinates plot to explore relationships between multiple features and failure types.
  2. Use dropdown widgets to filter the plot by failure type and observe how feature relationships change.
  3. For each failure type, use the brushing feature (select a range on an axis) to isolate the failing samples and read off approximate thresholds of the features involved. Note these thresholds down, you will need them in milestone 4.
  4. Document any new patterns or anomalies discovered through the interactive exploration.

15.4 Milestone 4: Distilling Findings into Rules

Exercise 15.4 (From exploration to a rule-based failure model)  

  1. Translate the thresholds from milestone 3 into explicit rules: write one Python function per failure type that predicts whether a sample fails.
  2. Evaluate your rules against the labels: compute the confusion matrix, precision, and recall for each failure type. Inspect the misclassified samples. Are your thresholds slightly off, or is a relevant feature missing?
  3. Split the data into a training and a test set and train a shallow decision tree per failure type on the training set (as in Section 13.1.6.2). Compare the trees and your hand-crafted rules on the test set. Which one generalizes better, and which one would you rather explain to a process engineer?