Notebooks

Notebooks give analysts and admins Python in the browser next to the SQL workspace: pandas, charts and longer analysis.

You need: an analyst or admin login. Vendors are refused.

Open it

In the web app, choose Notebooks in the menu (shown when the platform has the notebook address configured). It opens JupyterHub in a new tab and signs you in with the same account.

On a development setup the address is http://localhost:18000 through the SSH tunnel (see Local development). Use localhost, not 127.0.0.1, or the login redirect is refused.

The first time, the hub starts your own JupyterLab and copies start.ipynb into it. This file is yours; change it freely.

Query the lake

The starter notebook connects to Trino as you:

import os, pandas as pd
from sqlalchemy import create_engine

user = os.environ['JUPYTERHUB_USER']
eng = create_engine(f"trino://{user}@{os.environ['TRINO_HOST']}:{os.environ['TRINO_PORT']}/lake")
def q(sql): return pd.read_sql(sql, eng)

q("SHOW TABLES FROM lake.gold")
q("SELECT * FROM lake.gold.vendor__project__folder LIMIT 20")

Always use LIMIT while exploring: tables can be huge and your notebook has a memory cap. Bronze and silver have the lineage columns (_source_file, _ingested_at), which show which files the rows came from.

What is in the environment

One shared, read-only Python environment: JupyterLab, pandas, pyarrow, matplotlib, SQLAlchemy and the Trino driver. You cannot pip install into it. To add a package, an admin adds it to requirements-notebook.txt and re-runs install step 18.

Limits

Limit	Default
Memory per user	4 GB (the kernel is stopped, not the server)
CPU per user	2 cores
Idle time	60 minutes, then your server stops. Your files stay
Disk	your home folder is kept between sessions
Shell and sudo	none

See JupyterHub for how it is installed and how an admin manages it.