Waldo Blog nbconvert

This is a custom parser for turning jupyter notebooks into eleventy templates. I wasn't satisfied with the parsing options in the canonical nbconvert so I took it upon myself as an opportunity to actually write something in Rust. I'm pretty happy with it!

Installation

1. Clone the repo
2. cargo build --release

Usage

Note

This README is made by running nbconvert README.ipynb, so you can compare the two files to see what nbconvert does.

There's just one command:

nbconvert filename.ipynb

This turns filename.ipynb into filename.md. Markdown blocks are rendered as-is. Code blocks have the following transformations applied:

Source code and magic comments

Source code is rendered as-is with python code fences, except for magic comments. Magic comments are comments at the start of a code cell, taking the form

#| key: value

Right now, there are four magic comments:

• echo
  • true (default): the code cell is processed and added to the markdown file.
  • false: the code cell is skipped.
• fig_cap and alt_text
  • Can be provided multiple times in the same cell.
  • The Nth image checks the Nth time either of these options was provided. If the Nth option is fig_cap, the image is rendered as a figure with the value as the caption text. If the Nth option is alt_text, the image is rendered as a markdown image with value for alt text. If there are fewer than N options, the image is rendered as a picture with no alt text and a warning is emitted.
• suppress_text
  • true: Do not render any text results from this cell (i.e., only images).
  • false (default): Text and images are both rendered.
  • Note that most notebook parsers only render text if no images are rendered. The default behavior of this option may change to match that in the future.

So for example, this block:

#| suppress_text: true
#| fig_cap: A figure
#| alt_text: A picture

import matplotlib.pyplot as plt
plt.plot(range(10))
plt.show()

fig, ax = plt.subplots()
ax.plot(range(10))
fig

is rendered as:

import matplotlib.pyplot as plt
plt.plot(range(10))

fig, ax = plt.subplots()
ax.plot(range(10))

A figure


---

Images are saved to a notebook-images dir, created if it doesn't exist. Note that [<matplotlib.lines.Line2D at 0x136840c20>] (returned from ax.plot()) is not rendered.

Text outputs

Text outputs of code cells are rendered all together, even if they are separated by an image in the raw stream. Text outputs are surrounded by a nbresponse shortcode. So, the code block

print("Hello world!")

produces the markdown between the dividers:

---

print("Hello world!")

{% nbresponse %} Hello world!

{% endnbresponse %}

---

Exceptions have their own shortcode, nberror. The shortcode takes the exception class as an argument.

print("Next we'll raise an exception")
raise AttributeError("Bad attribute")
print("Never executed")

---

print("Next we'll raise an exception")
raise AttributeError("Bad attribute")
print("Never executed")

{% nbresponse %} Next we'll raise an exception

{% endnbresponse %}

{% nberror "AttributeError" %} Bad attribute {% endnberror %}

---

Tracebacks are not included in any output.

Config

In some posts, you might find yourself using the same magic comment in every codeblock. You can create an nbconvert.toml file in the same dir as the notebook to set the default values for the whole notebook. For example, if nbconvert.toml is:

suppress_text = true

then nbconvert will default to not printing cells' text output. You can still use #| suppress_text: false to turn text suppression off for individual cells. You can also set image_dir to something other than "notebook-images" in nbconvert.toml.