Repository navigation
Notebook
This page is about writing a pipeline as a notebook: DeepSharp.Verso.Notebooks, an extension for
Verso. Every block of the notebook holds one step of the pipeline, written as the
step's own JSON, and the blocks in the order they stand are the pipeline. The one rule to remember: a block holds no
data. What a block shows is worked out from the steps above it every time it is asked for, and saving the notebook
saves the steps and nothing else.
| You want to… | Write |
|---|---|
| Read a comma-separated file in the first block | { "step": "read.csv", … } |
| Write every step of a course still missing | press Course for a table or Course for a series |
| See the rows as they stand at a block | press Show the data here on the block's card |
| Leave a column out, or take it back | untick or tick its box in the grid's header |
| Pick columns, kinds and the output from one list | press Choose the columns on the schema block's card |
| Run every step, fitted on the training rows | press Run the pipeline |
| Save the pipeline file the C# chain writes | press Export the pipeline |
| Bring saved column decisions back into the blocks | press Take over the saved columns, then tick Apply |
| Read the pipeline in a C# cell | Variables.TryGet<string>("deepsharp.pipeline", out var text) |
| Hand a model's predictions back to the report block | Variables.Set("deepsharp.predictions", trained.Measures!.PredictionsToJson()) |
| Turn steps a format kept as text back into blocks | press Make blocks of the steps |
| Show a notebook in the browser with nothing else installed | deepsharp-serve Samples/titanic.verso |
| Host notebooks in an application of your own | new OpenNotebooks(grace: TimeSpan.FromMinutes(1)) |
The rest of this page explains each one.
DeepSharp.Verso.Notebooks is an extension for Verso, the open-source notebook that runs in VS Code and in the
browser. Install it from Verso's Extensions panel with the notebook open.
Verso loads a package installed there for a notebook that names it among the extensions it needs. Installing it into
the open notebook writes that notebook's name for it, so it is loaded there from then on. Install it the same way into
each notebook that does not name it yet, the sample Samples/titanic.verso among them.
It runs in Verso's VS Code extension, in the browser editor verso serve opens, in DeepSharp's own server, and inside
an application of your own — see where it runs.
A notebook is usually a script with a memory: change the second cell, run only the fifth, and the fifth now works on something the page no longer says — which is exactly the leak PDD exists to remove.
This one is not. Its blocks hold no data. Each block is one step of one declaration, what a block shows is worked out from the steps above it every time it is asked for, and saving the notebook saves the steps and nothing else.
Every block holds one step, written as the step's own JSON — exactly what a pipeline file holds for it — and the blocks, in the order they stand, are the pipeline.
A block added to a notebook starts empty, as Verso's editors add every cell. In DeepSharp's own host —
deepsharp-serve, or an application through DeepSharp.Verso.Api — a pipeline block arrives as the step of the course
that belongs where it is added, and a new notebook starts with one block, the step that reads a CSV file.
The first block may read a comma-separated file, a Parquet file, a sheet of an Excel workbook or a JSON file:
read.csv, read.parquet, read.excel or read.json, as a pipeline file names them. The notebook brings those
readers with it, as it brings the indicators and the charts, in every place it runs; no host can add another.
A window landed from a live source is a comma-separated file like any other, so a block reads it with read.csv. The
notebook has no block that lands, since its blocks are read again at every gesture and landing asks an exchange — see
Live sources.
A first block may also be read.join, which reads two comma-separated files as one source:
- The notebook keeps what it shows of a source by a fingerprint of the bytes it was read from. For a join that is a fingerprint of both files, in order: change only the second file and the view is read again, change neither and it is not.
- The block's form lists the two files and the columns the join is made on, and
unmatchedis a choice with no default.
A block is edited in either of two ways, and both write the same text.
As text. The editor offers the verbs after "step", the keys the verb takes, the words a key takes, and — once
the data has been shown — the columns there are. Hovering over a key says what it means. Verso's editor does not
underline a fault as you type; the block names every fault, each at its line, when it runs.
Field by field, in Verso's properties panel:
- A column is picked from the columns that are really there at that block.
- A list of columns is a switch per column.
- A schema is a pick per column of the source — its kind, or not taken. For a column taken there is a switch for whether it may be absent. For a timestamp taken there is a field for the format its moments are written in, left empty for ISO 8601. For every column taken there is a field for the value that stands for a gap in it, left empty for none.
- A column not taken stays in the schema with its kind, so taking it again brings it back as it was, and a step below that reads it says so at its own block.
- A number is typed with a full stop, whatever language the machine speaks, because a decimal comma would be read as something else in the file.
- The test share of a split is shown and never typed: it is whatever the others leave.
- A verb can be swapped only for one that does the same kind of thing, keeping every value the other verb takes.
- A change in the panel waits for a change on the grid that is already on its way, and is not made when that change rewrote the block: the panel then shows the block that stands in its place.
The rules are the pipeline's own, and a block that breaks one says so at the block: a fill written above the split is not hidden, it is shown with the rule it breaks.
The pipeline is the longest run of blocks from the top that makes a declaration, and every block below it says which block stops it.
Two buttons in the toolbar, Course for a table and Course for a series, write every step of a course the notebook does not hold yet, each as a block that waits for what only you know, in one turn.
A block that waits does not read as a step, so it names the key it waits for and nothing else, and it takes no part in the pipeline the blocks above it make. The blocks below it are not judged, so no rule is shown broken on a block nobody has touched.
Fill the blocks in from the top: a block starts to read the moment the last key it waits for is said, and then its properties panel takes it over.
A course goes on from where the blocks stand. It never writes over a block, never moves one, and puts its steps right after the last block that is a step. It is offered to no notebook whose steps stand in another order than the course teaches, or that holds the whole of it already.
In DeepSharp's own host, the add button between two blocks does the same one block at a time: a pipeline block arrives as the step of the course that belongs where it is added.
Starting from a course has both courses step by step, and why each step stands where it does.
Running a block shows its card: the stage it belongs to — read, columns, order, features, clean, split, learned from the training rows, evidence, output — and what its step does, or every fault, each at its line. Running a block does not run the pipeline.
Show the data here, on a block's card, does. It runs the pipeline down to that block and shows the rows there, fifty at a time:
- Every row says which part it lands in. The split is found wherever it is declared, so above the split every row already knows the part the split below will put it in.
- The cells are coloured per column, from blue at the smallest value to red at the largest, taken from the training rows alone.
- A gap has a colour of its own, and so does a value in validation or test outside the training range — precisely the rows where you have to decide what a value beyond the training range should do.
- A category is one colour, and darker for a value the training rows never held — one an encoder fitted on them would not know.
- A column that is not in is black, and its values are not on the page at all: ticking its box brings them back.
- Under the rows: a count of the rows in each part, and how many there are in all.
Showing the same block again, or another page of it, runs nothing. The notebook keeps the last view it worked out, under the steps it came from and the bytes of the file it read, and a view is those two and nothing else.
The rows as the file holds them are kept too, keyed by the file's bytes, which took close to half off every show on files of five and eleven megabytes.
The toolbar has the other two:
- Run the pipeline runs all of it, fitting every step on the training rows, and shows the last block.
- Export the pipeline saves it as the same file the C# chain writes, named after the notebook — with what the run learned, while that is the run of the steps the blocks declare now, over the file as it is now. Pressed while a change on the grid is on its way, it waits for it and saves the blocks as that change left them.
Every column in the grid's header has a box saying whether it is in, and every column the schema takes a second box saying whether it is a category. Ticking and unticking change the declaration, never the data.
-
Unticking a declared column no step names leaves it out in the schema, which keeps its kind. That holds for a
category the
encode.categoriesblock turns into columns of its own too —pclassandsexin the sample — though only the columns made of it reach the end. -
Unticking a column a step reads, or one a step makes, writes a
drop.columnsblock below the last step that reads it and the step that makes it, or adds the column to one that stands there already. - Ticking brings a column back as it was. It takes in a column the schema does not name as the list would — as the saved file declares it, else as its cells propose, else as text — where the source has it. Only the source's own header says that, so a column ticked before anything read the source shows the source first and takes nothing in.
- Category makes a column a category and remembers the kind it was; unticking it gives that kind back.
A box whose change would break a rule is drawn but cannot be clicked:
- the answer cannot be left out;
- nor the last column a schema takes;
- nor the last category
encode.categorieshas to encode; - a column a later step scales as a number cannot become a category;
- a category that does not say what it was cannot be unticked.
The notebook writes the block, runs it, and shows the grid again. A box sends the state it is in, so the same click twice does the same thing once.
A change a rule refuses — sent from a grid drawn before the blocks changed, say — is not made: the block says which rule, above the data as it was. A grid that a change made stale is cleared rather than left showing what is no longer so.
In a layout that cannot add or remove a block, such as a dashboard, the change is refused, and the block says to show the notebook in its notebook layout.
Choose the columns, on the schema block's card, lists every column of the source — and every column the schema names that the source lacks — as one row: its name, its first ten values, whether it is in, its kind, and what it is to the output.
Nothing runs to draw it: the rows are the source's as it reads them, and a column the saved file never showed is marked new.
The row's box is the grid's box. Ticking takes the column in — with the kind the schema keeps for it, else as the saved file declares it, else as its cells propose, else as text — and unticking leaves it out.
Where the cells decide, the row says so beside its kind: category, proposed, 3 values for embarked, a category
offered beside whole numbers such as sibsp, and the format moments written another way than ISO 8601 are read by,
which a tick takes in with them.
A select offers every kind the rules let the column take. Walking it with the keyboard sends every kind it passes; the list commits the one it ends on, as if that kind had been picked once from where the walk started. So a category walked away from and back to still remembers the kind it was.
- Above the rows a select picks the kind of output the rows' boxes make — one column, a distribution, labels, an answer read rows ahead — and picking it commits nothing.
- A row's output box puts its column into that output or takes it out. A column the schema does not take is taken in first, in the same change.
- A box of its own takes the output away.
- The output's own values — how many rows ahead, the value or a return, how many ones a row holds, what the shares are shares of — are selects offering only the values the rules keep, with "not said" where a file may leave one out.
- A value no rule bounds is set in the output block's form.
A pick above the rows switches between ticking one column and ticking a range. In a range the first tick says where it starts and changes nothing. The second takes in every column between — or, for an output whose answer is many columns, makes them the answer — with one kind picked for the range: seventy bands of a flock are two ticks.
A band the schema holds as text, or as any kind the output cannot read, becomes the range's kind when it is made the answer.
When the blocks or the source's bytes have changed since a list was drawn, other than by the walk of the select in use, a click on it draws the list again and changes nothing. In a session that has not read the source, the list asks to be chosen again.
Every box and select that can change the blocks — the grid's, the list's, Apply — waits three tenths of a second before it reads them, so text still on its way from the editor lands first.
What the blocks decide about their columns is saved beside the notebook, in a file named after it —
titanic.columns.json beside titanic.verso: the schema, with the columns it leaves out and their kinds, the columns
dropped, and the output.
- It is written after every change the blocks accept and every run of the whole pipeline, while every block is in the pipeline.
- Showing a block writes nothing, and the same decisions again leave the file as it was.
- A file that cannot be read is never written over, and one that cannot be written says so at the block.
- A notebook never saved has nowhere beside it, and saves none.
Take over the saved columns, on the toolbar, brings such a file's decisions back into the blocks — into a notebook written again over next month's rows, say. The button is there for a saved notebook whose blocks make one pipeline, with columns saved beside it.
It lists first and changes nothing. At the schema's block it shows:
- every column whose decision would change, from how it stands to how it would stand — its kind, what a category was, how a timestamp's moments are written, whether the source may lack it, which value stands for a gap;
- the output, the schema's order and what the schema does with the columns it does not name, when they would change;
- every saved drop the blocks cannot make, and why;
- the source's columns the file never showed.
A drop the blocks cannot make is never made: the saved file forgets it the next time it is written. When the blocks already hold everything else, the list says so, names those drops, and offers nothing to apply.
Otherwise under the list is one box, Apply, and ticking it makes exactly what was listed. Blocks that changed since the list was shown are left as they are — take over again — and a take-over whose blocks would break a rule is listed with every rule and offers no box.
Two blocks are about the data itself, and both are measured on the rows the split trains on: evidence.profile and
evidence.correlation. A third, evidence.likeness, compares the parts the split makes, and evidence.report names
what a trained model will be held to.
An evidence.profile block gives, per column, the rows, the gaps, the values that are not numbers, the distinct
values and, for numbers, the smallest, the largest, the mean and the middle. It counts the rows that are there more
than once.
For each thing it finds it says how it is answered:
- a gap — with a step the column's kind takes;
- a value that is not a number — with
fill.nan; - words — with an encoder;
- a column that never changes, hands a model the answer, says again what another says or only names its row — by leaving it out;
- a value that is how the file writes that nothing is known — by saying so in the schema.
An alert whose answer changes the columns has a box beside it, leave it out or say so, which gives the answer when ticked.
An evidence.correlation block draws a heatmap of the columns it names against each other, over the training rows
that hold a number in every one of them, and says how many rows that is. With the answer in it, a column that gives
the answer away — the Titanic set carries alive, which is survived in words — lights up at once.
It is drawn by DeepSharp.Charts, with MatPlotLibNet, which draws every chart
DeepSharp has.
- Its
coefficientsays whether the coefficients are Pearson's or Spearman's, which is Pearson's of the ranks. -
shownsays whether they are drawn or written as numbers. - A column that never changes has no coefficient with any other, so its cells are blank in the picture and
n/ain the table, with a sentence under either naming it.
The numbers are the pipeline's own, worked out once, so the picture, the table and the profile agree.
An evidence.likeness block, below the split, sets out how alike training, validation and test are: a row for each
column and a cell for each two parts. A cell holds the Kolmogorov–Smirnov distance of a column of numbers or the total
variation of a column of words, in bold where it is above the bound the block says, beside the rows each part had.
It reads the features of every part and never the answer, and it warns and never refuses.
An evidence.report block names what a trained model will be held to — which measures, on which parts, shown
how — below the output.
The notebook trains nothing, so the block's measures come from the C# cell that trains a model and hands its predictions back, below. "Show the data here" at the block then measures them on the notebook's own run of its blocks and draws them under the grid, as the block says they are shown — the numbers, the charts or both.
With nothing handed back, the block names its measures and says where they come from. Shown again or paged, it measures nothing again while the blocks, the file and the predictions stay as they were. Networks has the report itself.
A block may name the network the notebook's pipeline trains. A learn.network block holds the step a pipeline file
holds for it — the layers, what moves them, what judges them, when the run stops and the engine it runs on, with the
seed, the epochs and the batch size of the run — and stands below the output, as the step does in any pipeline.
The notebook's catalog knows the verb, the block's form draws a part's name and the settings that name takes, and typing a part offers the names there are. Pipeline has the step and Networks the words.
The block's form draws these as well, each optional:
| Part | Settings |
|---|---|
| a dense layer |
bias, weights, biasStart, l1, l2, maxNorm, learns
|
| a normalisation |
scale, shift, learns
|
| an optimizer |
weightDecay, dampening, nesterov, amsgrad, centered
|
| the step |
clipValue, stepEvery, average
|
A setting left at what leaving it out means is not written into the block's JSON, so a block that names none of them is the block it was.
The block trains nothing, since the step changes no column and runs nothing: it names the network, and a C# cell trains it, as a C# cell hands the report block its predictions.
The sample notebook, Samples/titanic.verso, holds such a block last, after its output block and its report block,
naming the network the passenger list trains. Its C# cell trains exactly that network with Train() rather than
writing one of its own.
A C# cell in the same notebook reads what the blocks declare, as text, under deepsharp.pipeline, and the notebook's
folder under deepsharp.folder — the folder a relative path in the pipeline is read from. The cell reads it through a
catalog that knows every verb a block can hold, so it brings the packages the notebook brings:
#r "nuget: DeepSharp.Pipelines.Indicators"
#r "nuget: DeepSharp.Pipelines.Parquet"
#r "nuget: DeepSharp.Pipelines.Excel"
#r "nuget: DeepSharp.Pipelines.Json"
#r "nuget: DeepSharp.Learners.Networks"
using DeepSharp.Learners.Networks;
using DeepSharp.Pipelines;
if (Variables.TryGet<string>("deepsharp.pipeline", out var text))
{
var folder = Variables.TryGet<string>("deepsharp.folder", out var saved) ? SourceFolder.Of(saved) : SourceFolder.WorkingDirectory;
var catalog = StepCatalog.BuiltIn().WithIndicators().WithParquet().WithExcel().WithJson().WithNetworks();
var declaration = PipelineDeclaration.FromJson(text, catalog);
var prepared = new Pipeline(declaration, rows: null, folder).Run();
}It is text rather than a pipeline because the notebook's types and a cell's are loaded apart, and are not the same
types even when their names are. Read it with TryGet, because it is often not there.
"Show the data here" and the toolbar's run hand it over — after the run, with what the run learned. It is taken back whenever the blocks may no longer make what it holds:
- after Verso's Run All;
- after a change made in a block's form;
- after a block run by hand with other text;
- when the blocks make no whole pipeline.
A block added, taken away or moved, a cell turned into another kind, or text typed into a block reaches the notebook at the next gesture, since Verso's editors tell it nothing of those. A grid the blocks no longer make is cleared then, and the pipeline is taken back unless they still make it. An application that changes cells itself tells the notebook at once, and so does DeepSharp's own server.
The way back is text too. The cell that trains a model hands back what the pipeline's report measured, under
deepsharp.predictions, as trained.Measures!.PredictionsToJson() writes it: the fit the predictions were made
behind, and for each part the report names the key of each row, what was predicted for it and what the model said it
learned nothing about. The cell writes it with
Variables.Set("deepsharp.predictions", trained.Measures!.PredictionsToJson()).
Only what the model gave crosses: what it is measured against, and the way back to the answers' own units, are the notebook's own run's.
Predictions are measured only beside the very fit they were made behind:
- a hand-back from before a block was edited — a feature or the answer scaled another way, say — is refused at the report block, saying to run the cell that trains again;
- so are predictions for other rows, or of another output;
- predictions from a learner that ran on a pipeline with steps left out for it are measured on a run of every step.
A cell that ends with trained.Measures!.Report() shows the same report itself, and one that ends with
trained.History!.LossCurve() the loss curve, as Verso shows the text of any picture.
The sample notebook, Samples/titanic.verso, does all of it: its blocks declare the Titanic pipeline and the network it
trains, its C# cell trains that network on what they hand over and hands the predictions back, and its report block
then draws the measures. Where the cell above runs the declaration with .Run(), the sample's does it with .Train(),
which trains the network the declaration names.
A notebook of blocks is saved as a .verso file, the one format of Verso's that keeps a block a block. Saving it there
works in every place the package runs — Verso's VS Code extension among them, where 0.3.0 and 0.4.0 refused every such
save, since Verso names its own format there by a second name the guard did not know.
Saving one in any other format is refused:
- Verso's Jupyter writer keeps what kind a cell is only in a mark of Verso's own, so every other program that opens the file shows each block as a raw cell;
- its Markdown writer keeps a block as text in a Markdown cell with the block's text fenced in;
-
.dibis one Verso reads and never writes.
The refusal works the other way too: a file of another format whose cells turn out to be steps — in either of those forms, or as code cells, as a Jupyter file another program wrote holds them — is refused when it is opened, rather than opened with every block turned into text that runs nothing.
Both refusals are the host's to ask for, and from Verso 1.2.3 every host of Verso's asks: its VS Code extension, its
browser editor verso serve, its command line's convert, run and repl, and DeepSharp's own server and an
application on DeepSharp.Verso.Api ask every extension before they write a file and after they read one.
One way around them is known, and written down rather than trusted: an install from the Extensions panel is loaded only after a file of another format has been opened, so the refusal on the way in holds only for an install at the top of Verso's extensions folder.
Tests pin the order Verso opens things in and the steps it takes when it opens and saves a file, so a Verso that changes either is noticed.
Make blocks of the steps, on the toolbar, is the way back, in every editor the notebook runs in. It is on wherever a cell holds a step a format kept the text of — a raw cell or a code cell whose whole text is a step, or a fence of the block's language in a cell of text — and pressing it puts a block in each such step's place.
The text around the fences stays text, in its place, and every other cell stays as it was. The blocks are written as any change to them is, and nothing runs.
A Jupyter copy made by a Verso that still asked no extension, opened in verso serve, made blocks again and saved
there, left a .verso file that held every block of the notebook it came from, and ran. Verso was asked to run the
guard in its browser editor too (#113) and to say so before a
save writes over a file of the same name (#114), and it does both
now.
The same package runs in four places, and it is the same code in each:
- Verso's own editor, in VS Code;
- Verso's own editor, in the browser;
- DeepSharp's own server,
DeepSharp.Verso.Serve; - an application of your own, with
DeepSharp.Verso.Apihosting it.
deepsharp-serve shows a notebook in your browser with nothing else installed — not Verso, not VS Code:
dotnet tool install --global DeepSharp.Verso.Serve
deepsharp-serve Samples/titanic.verso
That opens the sample, run from the repository's folder. Run it beside a notebook, or in a folder of them to have them listed.
It prints the address it serves and opens it; --port names the port, and --no-browser opens nothing, for a computer
you reach from another. The notebook runs code as you, so the server listens on this computer alone and answers only
the address it printed, token and all — Security says what shuts it.
Its page does what Verso's editor does. Each tab holds one connection to its notebook, as Verso's editor holds one, and everything the page draws with comes with it — Mermaid and KaTeX among it — so it fetches nothing from anywhere.
Opening a notebook runs nothing of it: HTML a cell shows, which the notebook's file keeps from the last time it ran, is drawn with whatever in it would run taken out.
It draws each block's card, the grid and the list as Verso's editor does, and sends a box or a select once, on its change. What a cell shows is drawn as that editor draws it:
- a failure with its name and where it happened;
- standard error labelled;
- JSON as a tree, CSV as a table, progress as a bar;
- Mermaid diagrams and the formulas in Markdown;
- a widget in a sandboxed frame of its own, as the state it saved.
Outputs set hidden say how many are hidden, and outputs cut short keep to the cell's lines. The dashboard and the presentation are drawn as the engine arranges them, the dashboard's tiles moved by their bar, resized by their corner and run by their button.
A cell is chosen by a click or by writing in it, and a Markdown, Mermaid or HTML cell shows its text while it is chosen and its rendering once it is left.
A cell is added under the last one or after the chosen one, of any kind the notebook has. Its bar turns it into another kind, moves it up or down, folds a code cell's text to its first lines, and deletes it once you say yes — each only where the notebook's layout allows it. The engine's own buttons on it, such as clearing what the cell shows, are there only where they can be pressed, as the export menu holds only what can be exported now.
What you type is sent as you type it, and what you type while the connection is down waits, marked on its cell, until it is back. Ctrl+Space — or a dot, or a quote — lists what the cell's kernel offers there, a block's verbs, keys and columns among them, and a line under the text says what the word at the cursor means until you leave the text.
The keys are Verso's:
| Key | Does |
|---|---|
| Shift+Enter | runs the cell and goes to the next |
| Ctrl+Enter | runs it and stays |
| Alt+Enter | runs it and inserts a code cell below |
| Ctrl+Alt+Enter | runs them all |
| Tab, Shift+Tab | indent and outdent |
| Escape | leaves the text |
The parameters form draws itself, and adds, sets and takes away parameters, as it does in Verso's editor.
The panels are Verso's: Metadata, where the notebook's title is changed; Properties, the chosen cell's fields, where the layout has them; and View, which switches the layout and the theme, the page drawn in the notebook's theme once it chose one.
Beside the notebook's name stand its kernel and how many cells it has, and a status says what the kernels do. Save carries a dot while anything is unsaved, in every tab.
A cell runs from its bar or with its keys only while nothing runs, as in Verso's editor, and the Stop then stands where Run All stood. A run that waits for a C# run in another notebook says so — on its cell, or on the page's own line for a toolbar button — until it runs or you stop it. A cell that a click on a block runs, or one a stop left behind, shows running with nothing to stop.
The page says things as Verso's editor does:
- a notice that goes after three seconds, such as "Saved to titanic.verso";
- a line that stands while what it says holds;
- an error in a banner you dismiss.
A dropped connection comes back by itself, keeping what you were writing. While it is down, the page says so, and that what is unsaved is lost if the server has stopped.
Close goes at once, even while a run is under way, and asks first only when something would be lost. Run in a folder, the list of its notebooks, shown even when there is one, makes a new notebook. A notebook no page has shown for a minute, with nothing running in it and nothing unsaved, closes by itself.
Ctrl+C in its terminal ends it at once, whatever a notebook runs: a run under way is stopped as its Stop does. It carries a build for .NET 8 and one for .NET 10, and keeps starting on the newer runtime once .NET 8 is gone.
Both are Verso's own editor, so everything on this page works alike in them, the guard on saving and opening included: from Verso 1.2.3 the browser editor asks it as the VS Code extension does — see saving. In both the package is installed from the Extensions panel, loaded for a notebook that names it.
They do run on different versions of .NET: the VS Code extension on the newest one installed, the browser editor that
verso serve opens on .NET 8 for as long as .NET 8 is there. The package carries a build for .NET 8 and one for
.NET 10, and each editor loads the one for the runtime it is on, so there is nothing to set. The Extensions panel keeps
one install for each runtime, so the two find their own on one machine.
Verso's engine is on NuGet as Verso, and
Verso's guide to embedding it
shows how an application opens a notebook and runs its cells.
Reference DeepSharp.Verso.Notebooks as well, and the engine finds the blocks on its own: it reads every assembly
beside the application that references Verso's abstractions. A program published as a single file has nothing beside
it, and then the engine finds no blocks at all, without a word.
DeepSharp.Verso.Api opens a notebook for you instead. It registers the blocks' parts itself before the engine looks
for any, reads the file through the serializer for its format, runs the guards that follow reading, and refuses an
extension a notebook asks for rather than fetching it.
It opens and saves a notebook as Verso's browser editor does:
- nothing the engine only falls back on — a kernel, a layout, a theme — is written into it;
- a layout the engine lacks is shown in the notebook's own and kept by its name;
- what the layouts arranged and the parts keep as settings goes back into the file at every save.
One file is one notebook however many views show it, and its cells come back as plain values.
Saving is Verso's own, with what a block shows left out and when it was saved stamped in. Saved under another name — through the holder of the notebooks, which knows which files are open — the notebook is that file from then on.
A new notebook is made as one block that reads a CSV file, written whole before it appears and never over a file that is there already.
What is done to it — typing a cell's text, running a cell, a click on a control a block drew, handed over as the control carries it — takes its turn one at a time, in the order it came. A click from a card whose block a change rewrote still reaches the part, which knows the block it became, and what a part answers a click with is what the cell shows next.
A cell is added after another or at the end, of any kind the engine has — code, Markdown, a pipeline block and the rest, each saying whether a person writes it and whether it is shown rendered once it has run. It starts empty, but for a pipeline block in DeepSharp's own host, which starts as the step of the course that belongs where it is added. It is taken away, moved past the neighbour it passes, or turned into another kind, keeping its text.
Each is refused where the notebook's layout does not allow it, as Verso's own editors refuse it, and a cell's properties panel is refused where the layout has none, as in the dashboard and the presentation.
As a cell's text is typed, its kernel offers what may come next and says what a word means, as Verso's editors ask it. These, and a cell's panel, are answered beside the notebook's turn, so they come back while a run is under way. One that a start afresh overlapped says nothing, even when the kernel failed as it was put away, since the kernel it asked is gone. Typing into a block tells the notebook at once, so what was worked out from the block as it was is taken back.
The layouts and the themes the engine has are listed:
-
SwitchLayoutAsyncandSwitchThemeAsyncswitch them as the notebook's choice; -
InteractAsynchands what a person did to what a layout drew — a tile moved, resized or run — to the layout's own part; -
RetitleAsyncchanges the title, which names what the notebook is exported as.
A run exists from the moment it is asked, numbered, and Stop(run) names the run it means, so a stop sent again after
that run ended stops no other.
Stop ends a run that never would, with a fresh kernel — which clears the notebook's variables. The kernel is the one
that ran at the stop, found as the engine finds it:
- for a cell, the kernel of the cell's type, else of its language, else the notebook's default;
- none for a cell that only draws, which takes no C# turn either;
- for code a button runs in no cell, its language, else the default.
A stop while nothing runs, between two of a button's cells, starts none. A run left behind that ends while its cell runs again leaves the stop of the run under way whole.
A stop is one step. Unless the run already ended by itself, it takes the run's end, marks the run, tells the notebook — so what the run left behind writes nothing more, and the notebook takes its next change at once — and only then takes what runs as the kernel to start afresh, once everything let through before the stop has landed. Whether a kernel starts afresh never depends on when a thread happens to run.
The run itself goes on in the background, and should it end later, Verso's C# kernel puts the process's console back as it found it, which can take from another C# run what it prints at that moment.
C# runs take their turn across the process, and a C# run waiting for its turn is told on every version as waiting;
Stop ends the wait, so it never runs.
The toolbar comes with every version: every button the engine has, each saying whether it can be pressed then — a button on a cell's toolbar for each cell, as Verso's editors ask it. One whose part cannot say is not pressable, and says why.
A press is one run. Stopped, the cell under way is left behind and no cell the button would still run begins, as Verso's own browser editor stops a Run All. A file a button hands over, such as the exported pipeline, comes back to whoever pressed it, unless the press was stopped, and nothing is written beside the notebook.
A cell's properties panel comes back as a section from every part that has one for it, and a changed field goes to that part. A look — a button asked whether it can be pressed, a panel asked to draw its section — is refused every verb, so what is only looked at does nothing to the notebook.
A change — a click on a control a cell drew, a changed field — runs DeepSharp's blocks as its own. Anything else it asks to run is a run of its own, told and stopped as any run is, and once that run is stopped nothing else the change asks is done, and what it would answer or hand over is dropped.
Every view of the notebook is told what changed: the notebook as it stands when the view begins, then each change after it, numbered — the cells that came or changed as they now stand, and every id in order when cells came, went or moved. A clear and a form's change are told like any other, though the engine itself says nothing of them.
What a C# cell displays while it runs reaches a view before the run ends. Every version says:
- which run is under way, from its start — the cell that runs or waits, none between a button's cells, since when, and
the number
Stopnames; - what the engine runs that no run owns — a block a click runs, what a stop left behind;
- whether the notebook differs from its file;
- what became of its kernels;
- which layout and theme it is shown in, and what the layout draws of its own;
- what the notebook says of itself: its title, its default kernel, its dates and its format.
What a cell shows carries a failure's name and where it happened, and whether a text came on standard error. A view that reads slowly is kept one change behind and never holds the notebook up.
Given a grace, the holder closes a notebook by itself once no view has shown it for that long, nothing runs in it —
nothing a stop left behind either — and nothing in it differs from the file it was last saved to; what a block shows
never counts. A notebook with changes not yet saved stays open until it is saved, or until
notebooks.CloseAsync(notebook) closes it at once.
No close waits for a run: the run under way is stopped as Stop stops it, a change under way finishes, and closing the
holder stops every notebook's run before any of them closes. Put together:
await using var notebooks = new OpenNotebooks(grace: TimeSpan.FromMinutes(1));
var notebook = await notebooks.OpenAsync("titanic.verso");
var card = await notebook.RunAsync(notebook.Cells[0].Id);
var clicked = await notebook.GestureAsync(new HostedGesture(card.Id, extensionId, action, payload));
var exported = await notebook.RunToolbarAsync("io.github.xkqg.deepsharp.notebooks.export");
await notebook.SaveAsync();
await notebooks.SaveAsAsync(notebook, "titanic-2.verso");
using var view = notebook.Subscribe();
Draw(view.Snapshot.Cells);
await foreach (var change in view.ReadAllAsync())
{
Redraw(change.Order, change.Cells);
}Drawing what a view is told is the application's own.
Verso's editor is not published for other applications, so an application that hosts Verso's engine without Api does for a block what the editor does:
-
Draw what a block shows. It is HTML.
-
Pass a click on. A control in that HTML carries the gesture in
data-actionand the part that answers it indata-extension-id. A button carries what it concerns indata-payload; a box carries none, and is passed on with its state —trueorfalse— and a select carries none either, and is passed on with the value it is at, as Verso's editor passes them. Hand the three to that part, with the block and the notebook, the way the notebook's own tests do:var part = extensionHost.GetInteractionHandler(extensionId); var gesture = new CellInteractionContext { CellId = block.Id, ExtensionId = extensionId, InteractionType = action, Payload = payload, Region = CellRegion.Output, Variables = scaffold.Variables, Notebook = scaffold.NotebookOps, NotebookModel = scaffold.Notebook, }; await part.OnCellInteractionAsync(gesture);
-
Draw again. The scaffold's
OnCellOutputUpdatednames a block whose output changed, and a gesture that changed the blocks themselves — a box ticked in the grid or the list, a select that commits, or a take-over applied — comes back withStateChangedset. -
Offer the toolbar. Run, Export, Take over the saved columns and Make blocks of the steps are toolbar actions, found with
GetToolbarActions(). They ask the application for the notebook's blocks, its variables, where it is saved, and a way to hand over a file. -
Save it. Save through the serializer the engine holds for
.verso, fromextensionHost.GetSerializers(), or throughnew VersoSerializer(extensionHost.GetCellTypes()). Either knows that a block keeps what it shows out of the file; aVersoSerializermade without the cell types cannot tell a block from any other cell, and saves a page of the data with it.
An application that wants only the pipeline, without the grid, needs no editor at all. It reads each block's text with
a catalog's ReadStep — StepCatalog.BuiltIn().ReadStep(text) — and makes the declaration from the steps, new PipelineDeclaration(steps).
Or it has Verso's command line write the pipeline file, given the folder the package was published into:
verso export titanic.verso --format "Export the pipeline" --extensions <that folder> -o titanic.pipeline.json
That file holds the steps and not what they learn, because nothing has run them there; running the pipeline learns it, from the training rows.
The pipeline itself stays exactly what it is: a program that writes Pdd.Create() without any notebook runs the same
file the notebook exports. The notebook is one more way of writing the declaration and looking at it — see Pipeline
for every verb it can hold.