Jupyter Notebooks are gone from my scientific workflow

TL;DR: I’ve just learned that the text editor Sublime Text can display images within Markdown files. Gone therefore is my need to use Jupyter Notebooks.

---

I was never a true convert to Jupyter Notebooks. I used them for several years, and saw their appeal, but they just didn’t quite feel right to me.

Most complaints against Notebooks are technical ones: they’re awkward to version control, they’re hard to debug, and they promote poor programming practices. But these issues are tangential to my complaints against Notebooks, which are are less concrete:

• I’m always scrolling. It’s inefficient.
• I don’t want to do work in a browser. Maybe it’s a weak reason, but I like keeping my scientific and programming tools separate from the browser.
• Editing and navigating Notebooks feels clumsy. Maybe it’s a lack of practice, but I’d rather leverage the time I’ve invested in learning and setting up my text editor than spend time learning a bunch of new shortcuts specific to Notebooks.

My old workflow using Notebooks

Until recently, I had a conventional workflow involving Notebooks. I would typically keep one Notebook for each paper I’m writing. Each had about a dozen figures interspersed with the code to generate them.

Except, the next step is maybe unusual. To run code, I wouldn’t use the Notebook itself. I prefer to work in a proper text editor, so I’d copy the code to the editor, run it in a console, iterate through the necessary changes, and then copy and paste the final code back into the Notebook.

My Notebooks were acting almost like a version of record. Their value to me was that whenever I needed to find code to produce a certain figure, I could search for that figure directly, rather than wading through a folder full of scripts—whose filenames might’ve been meaningful when I created them, but were less so when coming back weeks or months later.

I thought that I had a pretty clever set up. For each figure, I’d have two cells:

• A Markdown cell with a title and a PNG copy of the figure
• A cell with the associated code

With this setup and a few extensions, I could navigate somewhat quickly. First, I would collapse away all the code so that I was scrolling past only the figures. Second, by adding a toolbar on the side with a table of contents, I could jump to a given heading, which was sometimes quicker than scrolling.

Here’s what that looks like (without the table of contents):

If this approach appeals, see my Jupyter Notebooks post from 2018.

Long story short, I was using Jupyter Notebooks as an elaborate Markdown editor that just happened to have the ability to fold away code and show images. Notebooks are way more capable than this, but they were overkill for my workflow.

Wait? My favourite text editor can show images?

My workflow changed once I realised that my text editor can show inline images. The functionality is limited and the figures don’t look amazing, but it does the job.

Now, instead of having a Notebook with the figures and code, I have a single Markdown file with

• A PNG copy of the figure
• A file path pointing to the associated code

Here it is in action:

The setup is easy. Install the main package: Markdown Images and set “img_maxwidth”: 400 in MarkdownImages.sublime-settings so that figures don’t take up the whole screen. And also install OpenURL so that you can open links to files with a single shortcut (by default: Ctrl + Alt + u). And that’s it.

Should you try this?

You’ll have a workflow that works for you. And I’m sure it’s not the same as mine. But perhaps you’re a Sublime user. Perhaps today is the day you learned that your editor can show images. If so, my post has served its purpose.

If you’re not a Sublime user, you’re probably a Notebook user—otherwise you would’ve stopped reading long ago. Or maybe you’re a JupyterLab user who could point out all the cool new tools that make Notebooks seem quaint. Again, your workflow should work for you.

My approach here suits my workflow for a few reasons:

• I generate some figures on my laptop and others on more powerful, remote computers. It’s nice to have a master file that merely links to the figure generation scripts, because then I can keep those scripts in whatever location I need them to be.
• I use Matlab for some of my work. For this work, the Matlab kernel helps me a lot. But I mostly use it to run m-files (i.e. regular text files). Debugging and defining functions aren’t as easy when running code that is contained within cells in a Notebook.
• “Much of my data analysis is solely for me and the output is presented and discussed in other forms” (to quote from a similar post about Notebook alternatives).

If nothing else, I hope this post will “challenge you to think about how your tools limit you” and “how different workflows could make your life easier” (to quote Joel Grus from his I don’t like Notebooks talk.) Or, as someone on Reddit put it, users who stick to Notebooks “are missing the wonderful life of living with the terminal or the benefits of using an editor like vim (or even PyCharm or VSCode), which hampers their growth potential.”