Connecting to your notebook kernel using Jupyter console

Jupyter notebooks are a great way to explore data using Python (and other languages as well). Having a visual representation of your code and output, along with documentation and formatting in one view can be extremely helpful. However, there are some things that are just much better to do in a console session. In this post, I’m going to show you how to connect to the kernel that your notebook is using when you want to make use some of the convenience and speed of using a console.

Jupyter basics

First, let’s just establish a few basic details about Jupyter and its components. It’s good to know that the Jupyter project is a large project with a number of components. For now, we’ll just concentrate on two main portions: user interfaces and kernels. If you’ve always only used notebooks, you may not realize that the Jupyter projects supports a number of diffent user interfaces to the underlying kernel that is executing code. For now, we’ll look at just two user interfaces: notebooks and the console.

Notebook basics

I’m assuming you already know how to run a Jupyter notebook. If not, check out the basic docs.

Now, let’s assume you’re in a running Jupyter notebook. I’m writing this sentence now in a notebook, so I’ll first create a code cell and execute it.

x = 5

At this point, you should know that there is a kernel executing that the notebook front end is connected to. In that IPython kernel, there is now a variable x that has a value of 5. Now if you want to inspect that variable, you can create another cell, type x in it, and then execute it.

x
5

Why would I want to connect with the console?

There are a lot of great things about running code in a notebook, but there are a number of cases where notebooks just don’t cut it. One example is when you’re deep into debugging some isolated code. In this case, using a quick Read-Eval-Print Loop (REPL) environment like a console allows you to work much faster. Maybe you want to execute a function with 5 different parameters quickly and see the return values. If you’re using a notebook, you’ll have to either create or edit a cell five different times and then execute it. If you’re in a console, you can just quickly execute those lines and see the results.

A second area that a console shines is your command history. In the notebook, once you change a cell, you don’t see the previous value. In the console, you can see (and edit) the history of code that you’ve executed. It’s a big win to quickly and efficiently edit and re-execute those lines, especially when you’re making small changes and honing in on a solution.

Another example is when exploring values that are already defined in your session. In a notebook, you have to edit a cell and execute it each time you want to see anything. In a terminal, you can very quickly just view the values by executing a line with that variable to inspect it.

I’d recommend looking through the IPython tutorial for some more examples of using IPython. Much of this functionality will be helpful both in a console and notebook, but some features, like traversing command history, make using a console very helpful. 

Getting connected

Now, I’ll show you how you can connect to this kernel with another front end, a Jupyter console instead of a notebook. So what do you need?

First, make sure you have a console running using the same virtualenv as your Jupyter notebook instance. You can do this by starting a new terminal session and choosing your virtualenv as you did to launch the notebook instance above. Another option is to browse to the Jupyter home page on your local machine (probably something like http://localhost:8888) and launch a new terminal from the dropdown in the upper right corner.

Next, you can run this command: jupyter console --existing

At this point, you should connect to your existing kernel. Type x and hit return, and you should see that the variable already exists and has a value of 5. You’ll even see that the prompt history number should be the next number. So if the last command you ran in the notebook had a prompt of Out[5]:, then the console should be Out[6].

What if I’m running lots of notebooks?

Many times you’ll be running multiple notebooks, and you want to connect to a specific instance. To do this, you need to know which specific kernel to connect to. In your notebook, run the %connect_info magic, and a helpful message will show you how to do this.

{
  "shell_port": 57293,
  "iopub_port": 57294,
  "stdin_port": 57295,
  "control_port": 57297,
  "hb_port": 57296,
  "ip": "127.0.0.1",
  "key": "c2dc5435-fc2c47faa20a16212f0c92ac",
  "transport": "tcp",
  "signature_scheme": "hmac-sha256",
  "kernel_name": ""
}

Paste the above JSON into a file, and connect with:
    $> jupyter <app> --existing <file>
or, if you are local, you can connect with just:
    $> jupyter <app> --existing kernel-b5e78585-d011-4876-97cd-cb130fbd26ff.json
or even just:
    $> jupyter <app> --existing
if this is the most recent Jupyter kernel you have started.

So in our case, we’ll run the command jupyter console --existing kernel-b5e78585-d011-4876-97cd-cb130fbd26ff.json.

Once we’re connected, we can modify the value of x, and see that the change makes its way back to our notebook frontend by inspection (in a new cell or by re-executing an earlier cell).

One word of warning, notebooks are already an easy way to make a big mess of your code. If you start executing code in your notebook and your console session in a non-linear order, and then try to reproduce that output at a later time, you’ll find yourself very frustrated. Make sure that you only use this technique when you’re drilling into a specific issue where having console access is going to help you solve a specific problem. And when you have solved that problem, it usually makes sense to ensure all code changes are persisted to your notebook and saved, and if possible, you should restart your kernel and run the notebook from the top.