10  Self study - Session 1

This class contains self-study sessions to reinforce your understanding of the material and explore other aspects of the topic in a self-guided manner.

In this specific session, we will get hands-on with some essential tools for reporting: uv, JupyterLab, seaborn, and Quarto.

Each exercise focuses on a single topic.

Exercise 10.1 (Setting up an isolated environment with uv)  

uv is a fast, modern package and environment manager. It installs and manages Python versions and packages in isolated environments (venv).

  1. Install uv following the official instructions.
  2. In the project directory, initialize a new project and create a virtual environment using uv.
  3. Install the following packages into the environment: numpy, matplotlib.
  4. Verify the environment works by creating a simple Python script that creates a simple plot of \(f(t) = (16 (\sin(t))^3, 13 \cos(t) - 5 \cos(2t) - 2 \cos(3t) - \cos(4t))\) using the numpy and matplotlib package. (Hint: Use numpy.linspace to create a range of values for t, the function produces the \(x\) and \(y\) coordinates of the plot.)

After this exercise you should be able to

  1. Create and manage isolated Python environments using uv.
  2. Install and manage Python packages within these environments.
  3. Additionally be able to explain what uv does and why it is useful.

Now that you have a working environment, you can use it to install additional packages as needed for the exercises below.

The data set we will rely on for this session is the AI4I predictive maintenance dataset, which is a synthetic dataset that tries to mirror real predictive maintenance data encountered in industry.

As we are only using the data to explore a couple of libraries and tools, we are not going into the details of the dataset itself in this session.

Exercise 10.2 (Working with JupyterLab)  

JupyterLab is a web-based interactive development environment (IDE) for Jupyter Notebooks. It provides a flexible interface for interactive data exploration and reporting.

  1. Launch JupyterLab from your uv environment by adding it to your environment first or alternatively, using VS Code, you can also use the Jupyter extension instead.
  2. Create a new notebook called ai4i_explore.ipynb.
  3. Load the AI4I dataset into a pandas DataFrame using a code cell. Keep only the columns UDI (unique identifier which can be used as \(x-\)axis for plotting), Torque, Rotational speed and Air temperature.
  4. Display the first 10 rows and print basic statistics.
  5. Create a line plot of Torque over UDI using matplotlib.
  6. Experiment with different cell types: code, markdown, and raw cells.
  7. Export the notebook as HTML and PDF.

After this exercise you should be able to:

  1. Create and organize notebooks in JupyterLab.
  2. Differentiate between code, markdown, and raw cells.
  3. Export notebooks to other formats (HTML, PDF).
  4. Structure a notebook with self-contained, readable code cells.

matplotlib is a widely used Python library for creating static visualizations. Now we will introduce seaborn, which is a statistical data visualization library built on top of matplotlib, providing a high-level interface for drawing attractive and informative statistical graphics.

Exercise 10.3 (Visualizing the AI4I data with seaborn)  

In this exercise you will explore the AI4I dataset (columns UDI, Torque, Rotational speed and Air temperature) using seaborn.

Basic plots

  1. Load the AI4I dataset (keeping only Torque, Rotational speed and Air temperature).
  2. Create a line plot of Torque over UDI.
  3. Create a histogram of each column to observe the distribution of values.
  4. Create box plots of each column to identify outliers and understand the spread of the data.
  5. Create violin plots of each column and discuss how they differ from box plots.

Relationships between variables

  1. Create a scatter plot of Torque vs Rotational speed using sns.scatterplot. Add Air temperature as the hue parameter to see how the third variable influences the relationship.
  2. Create a pair plot (sns.pairplot) of all three numeric columns to observe pairwise relationships and distributions on the diagonal.
  3. Create a correlation heatmap using sns.heatmap on the correlation matrix of the three columns. Discuss which variables are most strongly correlated.

Advanced plots

  1. Create a joint plot of Torque vs Air temperature with a regression line overlay.
  2. Create a strip plot or swarm plot of Torque colored by a binned version of Rotational speed (e.g., low/medium/high using pd.cut).
  3. Create a count plot showing the distribution of data points across bins of Torque (e.g., binned into 10 intervals).

Observations

  1. Based on all the plots above, answer the following:
    • Are there any obvious outliers in the data?
    • Which two variables seem most related?
    • Are there any non-linear patterns visible?
    • How are the distributions shaped?

After this exercise you should be able to:

  1. Create a wide range of static statistical plots with seaborn.
  2. Use parameters to add additional dimensions to your plots.
  3. Compare and discuss different plot types for different analytical questions.
  4. Draw meaningful observations from statistical visualizations.

Congratulations, you are now able to create and manage isolated Python environments, work with JupyterLab, and visualize data using seaborn. The next step is to integrate your code and visualizations into a comprehensive report which you can hand over to stakeholders using Quarto.

Exercise 10.4 (Generating a report with Quarto)  

Quarto is an open-source scientific and technical publishing system. It allows you to create dynamic documents, reports, and presentations from your code and narrative text. The material of this course itself is rendered using Quarto.

  1. Create a Quarto Markdown (.qmd) file in your project.
  2. Embed the code and plots from the previous exercise into the Quarto document.
  3. Use code chunk options to control output (e.g., echo=FALSE to hide code, fig-width to set figure dimensions).
  4. Render the document as HTML and PDF.
  5. Explore collapsing code chunks in your document (see Quarto docs).
  6. Quarto also allows to render Jupyter notebook files (.ipynb) directly to HTML and PDF (see Quarto docs):
    • Prepare the notebook you created in the previous exercise for rendering with Quarto by adding additional information that makes a nice presentation.
    • Render the notebook you created as HTML and PDF using Quarto.

After this exercise you should be able to:

  1. Create a Quarto document that integrates code, output, and narrative.
  2. Render Quarto documents to HTML and PDF formats.