Python notebooks

A Python notebook runs cells in an interpreter inside the server, with the Shell scope available as the shell module. Cells share one namespace, so a variable defined in one cell is there in the next. This page shows the module, how to query the graph, how names map from C#, and how to turn a query into a DataFrame. For what a notebook is and what it can reach, see the overview.

Open Shell, select New tab, and choose Notebook (Python).

The interpreter

The interpreter is Computerwelt, a Python interpreter written in C# that runs in the server process. It runs Python code with its own standard library subset, numpy and pandas emulations, and the limits below. It is not CPython: packages with native code, pip install and the parts of the standard library that touch the host are not available. The Computerwelt Python pages list what is supported. Because it is an interpreter, loops in a cell are much slower than in a C# notebook: see How Python notebooks run, and how fast.

• No host access. Code cannot read the server's files, open a network connection or start a process. open() works on a scratch file system in memory that belongs to the notebook and ends with its kernel.
• A budget per cell. A cell can run 20 million bytecode instructions. A loop that needs more fails with RuntimeError: instruction limit exceeded. Move bulk work into the graph or into pandas operations, which run in C#.
• Output. A cell shows what it printed and the value of its last expression. A trailing ; on the last line suppresses the value.

import math
xs = [1, 2, 3]
print("hello from python")
sum(xs) * math.pi

hello from python
18.84955592153876

Names from the shell scope

Before the first cell runs, the notebook runs from shell import *, so the scope's members are in scope under Python names:

The reference lists every name with its signatures. The notebook also adds dataframe() and query_rows(), described below, and imports the flint module for charts.

The names set up this way are not listed as variables in the Variables panel.

Naming: Python style, C# spelling accepted

Everything the scope hands out, and every member of the objects it returns, is named in snake_case: StartAt is start_at, AsEnumerable is as_enumerable, a keyword argument userUID is user_uid. Completion lists only the Python names. The C# spelling still works when you type it, so code copied from a C# cell runs:

q().start_at("Ticket")   # Python style
q().StartAt("Ticket")    # C# spelling, same call

A name ending in _async still waits for its result. get_secret_async("Api") returns the secret, not a task.

Query the graph

q() returns the same query object as Q() in C#. Its methods are the graph query language in snake_case.

q().start_at("Ticket").emit("N")

A cell that ends in a query shows the query's results in the viewer the Shell's Results pane uses.

To work with the nodes in Python, read them with as_enumerable():

open_tickets = [
    n for n in q().start_at("Ticket").as_enumerable()
    if n.get_string("Status") == "Open"
]
len(open_tickets)

Each item is a node. get_string("Field"), get_int("Field") and the other typed getters read its fields, and uid and type give its identity.

Values cross between the interpreter and the server as follows:

Going the other way, None is null, a str becomes a string, a UID, an enum member or any type that reads from text, and a list becomes an array or a list. When no method matches the arguments, the error lists each overload that was tried and why it did not fit.

Turn a query into a DataFrame

dataframe(query, *fields) builds a pandas DataFrame with a row per node:

df = dataframe(q().start_at("Ticket"))
df.head()

                      UID    Type  Status Priority
0  RoUTCKNvDA3FDoseGtct4y  Ticket    Open     High
1  35bXwC3MCdVVwNbLcCxjGy  Ticket  Closed      Low
2  7Fq2xWkLpR9aTz1mNcYvDe  Ticket    Open      Low

The columns are UID, Type and every field. Name fields to take only those, in that order:

df = dataframe(q().start_at("Ticket"), "Status", "Priority")
df.groupby("Status").size()

Status
Closed    1
Open      2
dtype: int64

query_rows(query, *fields) returns the same rows as a list of dictionaries, for code that does not need pandas. Both accept a list of nodes in place of a query:

nodes = list(q().start_at("Ticket").as_enumerable())
rows = query_rows(nodes, "Status")

Fields hidden by the schema are left out, node types that never leave the server (secrets, tokens) are dropped, and a query of more than 100,000 nodes fails with a message to narrow it. A node that lacks a field has None there.

Work with the result

A DataFrame displays as an HTML table. numpy and pandas work on the rows in memory:

import pandas as pd

df = dataframe(q().start_at("Ticket"), "Status", "Priority")
by_status = df.groupby("Status").size()
by_status

To plot it, pass the DataFrame to the flint module. See charts and tables.

Errors

An exception in a cell shows its traceback, with the cell number in the file name, and the notebook carries on with its variables intact:

Traceback (most recent call last):
  File "<cell 3>", line 1, in <module>
ZeroDivisionError: division by zero

Errors from the server arrive as Python exceptions you can catch:

A stop is not an exception you can catch. It ends the cell with KeyboardInterrupt.

Stop a running cell

Stop cancels the cell's token. The interpreter stops at its next instruction, and so does a query the cell started through the scope. The notebook keeps its variables.

import time
time.sleep(60)

KeyboardInterrupt

Completion

The editor completes names in scope, members of modules and members of objects. It follows a chain of calls on the scope by the types the methods declare, without running anything, so these all complete:

q().                               # the members of the query
q().start_at("Ticket").            # the members of what start_at returns
graph.                             # the members of the graph
flint.                             # the functions of the module

Completion does not run your code, so the result of a function you wrote (f().) is not completed.