MyST-NB Usage#

The (probably) most useful function of this library is the myst_nb_metadata_injector. This allows you to write magic-comments inside of your Jupyter code cells to inject metadata into those cells and configure the execution/rendering behavior of the MyST-NB library.

For example, with a simple comment like # remove-input you can remove the code cell in your rendered documentation and only show the output without having to fight with setting the metadata tags of your code cell. See the following example:

Source (what you would write inside of your Jupyter notebook)

# remove-input
print("My code cell is removed")

Rendered output of MyST-NB:

My code cell is removed

As you can (can’t ?) see the output of the cell was rendered but the code-cell itself is hidden on the generated page.

The main motivation for this functionality is to keep the configuration local and not hide it behind an metadata window inside of the Jupyter Notebook UI. This is especially frustrating for users that use UIs that do not support configuring the code cell metadata, like VSCode.

Enabling the Metadata Injector for MyST-NB#

To enable MyST-NB to use the metadata injection preprocessor by default, you have to add the following code to your conf.py:

nb_custom_formats = {
    ".ipynb": [
        "common_nb_preprocessors.myst_nb_metadata_injector",
        {"prefix": "#", "delimiter": "=", "extra_tags": []},
    ]
}

The current version requires you to specify the prefix that indicates the start of a potential magic comment. Usually, this should be comment symbol(s) of the programming language. By default, and used in the example above, the prefix=# as this is the comment symbol for the Python programming language. The delimiter may be a character/string of your choice and will be used for separating key-value pairs. If you would like to add additional tags (when writing custom CSS rules, for example), you can also specify these as a list of strings in the extra_tags option.

To see a list of all possible configuration, see the API documentation of mystnb_nb_metadata_injector.

Important

To overwrite the default ipynb extension parser, you are required to use MyST-NB>=0.14.0! At the time of writing, the jupyter-book project (v0.13) fixes the MyST-NB version to 0.13.1 and cannot overwrite the default ipynb format.

You can rename your notebooks with a different extension (only characters after the last . are considered the extension) and use that option for the custom parser instead. The main issue with that approach is that some tools have a hard time working with a different extension for Jupyter notebooks (like VSCode).

Understanding the examples#

To show the effects of the different options, the article will be structured as follows:

The jupyter-notebook cell’s source code is shown. The visual indicator of the source code of the code cell, is that the stylized code-cell has no border around it.
The jupyter-notebook cell’s source code is displayed after applying the preprocessor. This will remove the magic comment and inject the metadata into the cell’s code (not shown here). The processed code cell is stylized with a white border and a green left-border. This is the rendered code cell you would be seeing in your documentation.
The output of the executed code-cell is shown with a white bordered cell.

And one last visual example to clarify the different stages:

Source:

# I am stage 1 and show the 'source' code of
# the code cell

Rendered:

# magic-comment
"""
I am stage 2 and show the post-processed code cell.
If the magic comment matches one of the MyST-NB tags
it would be removed from this cell and injected into the
metadata field.
"""
print("I am the rendered code cell output")

I am the rendered code cell output

The MyST-NB library provides two ways to configure the execution/rendering process of a code cell. One is to add specific strings to the tags metadata list. See Tag configuration for details on what is possible with this configuration method. The other configuration method is to set key-value pairs inside of the mystnb metadata tag of the code-cell. See Myst-NB Metadata Configuration for details. Note that the things you can configure are mostly different between both of these methods! You should take a look at both! I’ve also spend quite a lot of time to create some syntax sugar for settings nested key-value pairs. See Syntax Sugar for more details.

If the last paragraph confused you, don’t worry. It is probably not that important for you as it discusses the conceptual difference between the two configuration options. The important thing to know is that the tag based configuration only allows you to add a string, while the other allows you to set a key-value pair. You will see how this effects the configuration in the following sections.

Tag configuration#

At the time of writing, the following configuration keywords can be set in the tags metadata entry of a Jupyter code-cell:

remove-input
hide-input
remove-stderr
remove-output
hide-output
remove-cell
hide-cell
skip-execution
raises-exception

For more details, see the Cell tags section of the MyST configuration documentation.

no-match example#

Example showing what happens with code cells that do not include magic comments/are normal code cells.

Source:

# unknown
print("Example showing what happens if the cell doesn't include a magic comment")

Rendered:

# unknown
print("Example showing what happens if the cell doesn't include a magic comment")

Example showing what happens if the cell doesn't include a magic comment

Not so surprisingly, nothing much happens with those code-cells. They are simply executed and rendered as-is.

remove-input#

Removes the source code of the cell. Useful if you are only interested in the output.

Source:

# remove-input
print("My code cell is removed")

Rendered:

My code cell is removed

hide-input#

Hide the source code of the cell behind a collapsible button. Useful if you want to draw the attention of the reader away from the code but still allow them to view the code if interested.

Source:

# hide-input
print("My code cell is hidden behind a collapsible button.")

My code cell is hidden behind a collapsible button.

remove-stderr#

Only remove the standard error output of the code cell. This is useful if an underlying library uses stderr to print progress or other debug messages that would only pollute the output for the documentation.

Source:

# remove-stderr
import sys
print("The stderr output is removed", file=sys.stderr)
print("But not the default output")

Rendered:

import sys

print("The stderr output is removed", file=sys.stderr)
print("But not the default output")

But not the default output

remove-output#

Remove the output of the code cell (duh). Useful to hide some warning during important without having filter the messages or to hide visual output.

Source:

# remove-output
import sys
print("My output is removed")
print("including the error output", file=sys.stderr)

Rendered:

import sys

print("My output is removed")
print("including the error output", file=sys.stderr)

hide-output#

Hide the output of the code cell behind a collapsible button. This affects the entire output (stderr and stdout, like remove-output).

Source:

# hide-output
print("My output is hidden behind a collapsible button.")

print("My output is hidden behind a collapsible button.")

remove-cell#

Remove the input as well as the output of the code cell.

Source:

# remove-cell
print("This entire cell is removed")

Rendered:

As you can (can’t ?) see the cell is missing from the documentation page.

Note

The code-cell is still executed under the hood!

hide-cell#

Hide the input as well as the output of the code cell behind a collapsible button.

Source:

# hide-cell
print("The input and output is hidden behind a collapsible button.")

raises-exception#

The following code raises an exception and has raises-exception set to signal that the exception is expected and continues with the execution and build process of the documentation. This is helpful, if you are showing examples of failing code and want to include the trace-back.

Source:

# raises-exception
1 / 0

Rendered:

1 / 0

---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
Cell In [11], line 1
----> 1 1 / 0

ZeroDivisionError: division by zero

skip-execution#

The following code would raise an exception but the execution of the cell is skipped and not exception is raised.

Source:

# skip-execution
1 / 0

Rendered:

1 / 0

Notice how no output is produced and no exception is raised.

Myst-NB Metadata Configuration#

At the time of writing, the following configuration keyword-value pairs can be set in the mystnb metadata entry of a Jupyter code-cell:

remove_code_source
remove_code_outputs
code_prompt_show
code_prompt_hide
number_source_lines
output_stderr
merge_streams
text_lexer
error_lexer
markdown_format
image
figure

In contrast to the tag-based configuration, the mystnb-based configuration uses a key-value pair! The delimiter is used to separate the key-value pairs. Note that spaces around the delimiter are allowed/ignored during the parsing process. The value is parsed as a yaml-string! This allows you to even set nested structures in a single line and is required for coercing the data to the correct type.

Important

The value of the key-value pair is parsed as a yaml-string to allow correctly coercing the data to the desired type and to allow nested structures within a line.

This library only allows setting the value on the same line. This means that you cannot write the value across multiple lines!

remove_code_source#

Removes the code cell… (Surprising, right?)

Source:

# remove_code_source = true
print("My code cell is removed")

Rendered:

My code cell is removed

remove_code_outputs#

Removes the output of the cell, including the output to standard error.

Source:

# remove_code_outputs = true
import sys
print("My output is removed")
print("including the error output", file=sys.stderr)

import sys

print("My output is removed")
print("including the error output", file=sys.stderr)

code_prompt_show#

Prompt text that is displayed to inform the user that the dialog can be expanded. The optional {type} placeholder will be replaced with content, source, or outputs, depending on the hide tag (hide-input, hide-output, hide-cell).

Note

Also requires the code cell to specify one of the hide-X options.

Mainly included for completeness. This would be an ideal example of a configuration value that should be set globally instead of individually within each code cell.

Source:

# hide-input
# code_prompt_show = "Click here to see my absolutely awesome {type}"
print("Setting code_prompt_show")

Setting code_prompt_show

code_prompt_hide#

Prompt text that is displayed to inform the user that the dialog can be collapsed/minimized. The optional {type} placeholder will be replaced with content, source, or outputs, depending on the hide tag (hide-input, hide-output, hide-cell).

Note

Also requires the code cell to specify one of the hide-X options.

Mainly included for completeness. This would be an ideal example of a configuration value that should be set globally instead of individually within each code cell.

Source:

# hide-input
# code_prompt_hide = "Click here to minimize my absolutely awesome {type} (What is wrong with it?! :( )"
print("Expand the button above to see the text!")

Expand the button above to see the hide prompt!

number_source_lines#

Add source line numbering to the current cell.

Source:

# number_source_lines = true
# line 1
# line 2
print("I love my husky.")

Rendered:

# line 1
# line 2
print("I love my husky.")

I love my husky.

output_stderr#

Define the behavior for standard error output. Can be set to one of:

show: (Default) Show the stderr as a separate output
remove: Hide stderr
remove-warn: Hide stderr but log a warning to Sphinx
warn|error|severe: Log the error with a specific logging level to Sphinx

Source:

# output_stderr = show
import sys
print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

Rendered:

import sys

print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

stdout 0
stdout 1

stderr

Source:

# output_stderr = remove
import sys
print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

import sys

print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

stdout 0
stdout 1

For the other options, see the official MyST-NB documentation for the stderr configuration.

merge_streams#

Merge standard output and standard error into a single output cell.

Error

This doesn’t work!

I will have to investigate why this isn’t working. It seems like it also doesn’t work by manually setting the tag.

Source:

# merge_streams = true
import sys
print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

Rendered:

import sys

print("stdout 0")
print("stderr", file=sys.stderr)
print("stdout 1")

stdout 0
stdout 1

stderr

text_lexer#

Pygments lexer applied to standard out, standard error and text/plain outputs.

Source:

# text_lexer = python
print("# Render output with Python code cell lexer")

Rendered:

print("# Render output with Python code cell lexer")

# Render output with Python code cell lexer

error_lexer#

Pygments lexer applied to the error/traceback outputs. Note, that to be able to correctly build the documentation, the following example also uses raises-exception to let the executor now that we expect an exception to be raised.

Source:

# error_lexer = python
# raises-exception
raise Exception("# The last line be lexed as a Python comment, everything else will be broken")

Rendered:

raise Exception(
    "# The last line be lexed as a Python comment, everything else will be broken"
)

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In [21], line 1
----> 1 raise Exception(
      2     "# The last line be lexed as a Python comment, everything else will be broken"
      3 )

Exception: # The last line be lexed as a Python comment, everything else will be broken

markdown_format#

The format to use for text/markdown rendering.

Can be one of:

commonmark (Default)
gfm
myst

Source:

# markdown_format = myst
from IPython.display import Markdown

Markdown(r"""
Term 1
: Requires `deflist` extension to be loaded
""")

Rendered:

from IPython.display import Markdown

Markdown(
    r"""
Term 1
: Requires myst and `deflist` extension to be loaded
"""
)

Term 1: Requires myst and deflist extension to be loaded

In contrast to the default with

Source:

# markdown_format = commonmark
from IPython.display import Markdown

Markdown(r"""
Term 1
: Won't render as `deflist` with commonmark
""")

from IPython.display import Markdown

Markdown(
    r"""
Term 1
: Won't render as `deflist` with commonmark
"""
)

Term 1 : Won’t render as deflist with commonmark

image#

Options for image outputs.

To see a list of all possible configuration values, take a look at the official image documentation.

Source:

# image = {"width": 20%, "align": "right", "alt": "A cute comic-style fish with a party hat"}
from IPython.display import Image
Image("fish.png")

from IPython.display import Image

Image("fish.png")

A cute comic-style fish with a party hat

figure#

Options for figure outputs. To see a list of all possible configuration values, take a look at the official figure documentation.

Source:

# figure = {"caption": "I am a top caption!", "caption_before": true}
from IPython.display import Image
Image("fish.png")

from IPython.display import Image

Image("fish.png")

_images/8805bb905c918293d73323a7663fc62884a9e0f108cf85be829e86e24d69fbdf.png — I am a top caption!#

Syntax Sugar#

I’ve spent way too much time (and made the library significantly complexer) to allow nested key setting. This is especially useful for nested options such as figure and image. So instead of writing a long line like the following:

Source:

# figure = {"caption": "I am a top caption!", "caption_before": true}
# image = {"width": "10%", "alt": "Fish with party hat"}
from IPython.display import Image
Image("fish.png")

from IPython.display import Image

Image("fish.png")

Fish with party hat — I am a top caption!#

You could also write it in such a way that each option is on their own line and access the next level with using the dot (.) character.

# figure.caption = "I am a top caption!"
# figure.caption_before = true
# image.width = "10%"
# image.alt = "Fish with party hat"
from IPython.display import Image

Image("fish.png")

from IPython.display import Image

Image("fish.png")