Skip to content

Latest commit

 

History

History
796 lines (645 loc) · 24.5 KB

tutorial.md

File metadata and controls

796 lines (645 loc) · 24.5 KB

Learn Aether: the tutorial

This tutorial is the story told alongside the path through the examples. The path is the code: one file per idea, each saying how to run it and ending with what it prints. This page says what each step shows and why it comes where it does. The first two chapters, the language and actors, also list the mistakes newcomers make and end with exercises and their answers. A quick reference of the syntax closes the page.

Read a chapter, then run its examples in order from the repository root and change them:

ae run examples/01-language/01-hello.ae

Every program on this page is compiled by CI, and the ones followed by their output are run and compared, so what you read is what Aether does today. Getting started installs the toolchain; the glossary defines the words this page uses; the language reference has everything else.

  1. The language
  2. Actors
  3. The standard library
  4. C and other languages
  5. Containing code
  6. Tools
  7. Programs

1. The language

Aether compiles to C: ae run turns a file into a native program and runs it. A program is a set of functions, and it starts at main(). There are no classes and no methods: structs are plain data, behaviour is functions, and the one stateful thing with an inside and an outside is the actor (chapter 2).

Values, strings and control flow

01-hello, 02-values-and-types, 03-strings, 04-control-flow

A line ends a statement, and ${...} puts any value into a string. A local takes its type from the first value it is given and keeps it. Conditions need no parentheses, every body takes braces, and if is also a value:

main() {
    temps = [12, 19, 25, 31]
    for i in 0..temps.len {
        t = temps[i]
        feel = if t < 15 { "cold" } else if t < 28 { "mild" } else { "hot" }
        println("${t} C is ${feel}")
    }
}
12 C is cold
19 C is mild
25 C is mild
31 C is hot

A string is immutable text that knows its length; == compares contents. What takes strings apart and builds new ones is in std.string, called through the module's name: string.length(s), string.to_upper(s).

Functions

05-functions

Parameters can have defaults and calls can name their arguments. A function can return several values, which the caller takes apart, and a function can be written as clauses, each for one shape of argument:

min_max(xs: int[]) -> (int, int) {
    lo = xs[0]
    hi = xs[0]
    for i in 1..xs.len {
        if xs[i] < lo { lo = xs[i] }
        if xs[i] > hi { hi = xs[i] }
    }
    return lo, hi
}

describe(0) -> "zero"
describe(n) when n < 0 -> "negative"
describe(n) -> "positive"

greet(name: string, greeting: string = "Hello") -> string {
    return "${greeting}, ${name}"
}

main() {
    lo, hi = min_max([7, -2, 9, 4])
    println("${lo} .. ${hi}")
    println("${describe(0)} ${describe(-5)} ${describe(3)}")
    println(greet("Ada"))
    println(greet(greeting: "Welcome", name: "Grace"))
}
-2 .. 9
zero negative positive
Hello, Ada
Welcome, Grace

Data: structs, enums, sum types and match

06-structs, 07-enums-and-sums, 08-match

A struct is a record held by value: assigning or passing one copies it, and a function that should change the caller's takes its address, &p. An enum is a set of named constants. A sum type is a value that is exactly one of several structs, and match over it must cover every one, so adding a variant shows every place that has to learn it:

struct Deposit { amount: int }
struct Withdraw { amount: int }
struct Close {}

type Command = Deposit | Withdraw | Close

apply(balance: int, c: Command) -> int {
    return match c {
        Deposit -> balance + c.amount
        Withdraw -> balance - c.amount
        Close -> 0
    }
}

main() {
    let cmds: Command[3] = [Deposit { amount: 50 }, Withdraw { amount: 20 }, Deposit { amount: 5 }]
    balance = 0
    for i in 0..cmds.len {
        balance = apply(balance, cmds[i])
    }
    println("balance: ${balance}")
}
balance: 35

Inside an arm, the value is that variant, so c.amount reads its field. match also takes literals, ranges, strings, list patterns such as [head | tail] and struct patterns; it is an expression, so it has a value.

Absence and failure

09-optionals, 10-errors

Two different things, two different tools. A value that may be absent is an optional, T?: none or a value, read with ?? for a fallback. An operation that can fail returns its value and an error string, "" when it worked; -> T! names that pair, and or gives a fallback:

import std.string

first_even(xs: int[]) -> int? {
    for i in 0..xs.len {
        if xs[i] % 2 == 0 {
            return xs[i]
        }
    }
    return none
}

parse_percent(text: string) -> int! {
    n, err = string.to_int(text)
    if err != "" {
        return 0, "'${text}' is not a number"
    }
    if n < 0 || n > 100 {
        return 0, "${n} is not a percentage"
    }
    return n
}

main() {
    println(first_even([3, 8, 5]) ?? -1)
    println(first_even([3, 5]) ?? -1)

    v, err = parse_percent("120")
    println("${v} [${err}]")
    println(parse_percent("42") or 0)
    println(parse_percent("lots") or 0)
}
8
-1
0 [120 is not a percentage]
42
0

A panic is neither: it is for a bug. It unwinds to the nearest try, and outside one it ends the program. A bad index, a broken contract and a none forced open with ! are panics.

Arrays, slices and collections

11-arrays-and-slices, 12-collections

[1, 2, 3] is a fixed array, a value like a struct. A slice, int[], is a view of elements stored elsewhere, so a parameter typed int[] takes any array without copying it. Every index is checked. For things that grow, std.list, std.map and std.set give handles (below).

Functions as values, and modules

13-closures, 14-modules

|x| -> x * 2 is a function value, and a closure sees the variables around it. A call followed by a block passes the block as its last argument: the trailing-block form that most of std's builders use. A module is a directory under lib/ with a module.ae whose exports(...) line names what other files may use.

Contracts, defer and ownership

15-contracts, 16-defer, 17-ownership

requires and ensures state what a function needs and what it promises, and a broken one is a panic naming the clause. defer runs its statement however the block ends.

Ownership is the last step of the chapter because it explains the others. Aether has no garbage collector, and it does not ask you to free everything either. Three rules cover it:

  • Counted values: strings, closures, string sequences, and the structs, tuples, optionals, sum values and fixed-size arrays made from them. The compiler counts them and frees them; you write no free.
  • A slice's buffer (make([]T, n), or one push grows) has one owner. A local that holds it releases it when its scope ends, as does a local, a caller or a @keep parameter it moves to. Put in a struct's field, a module var, a tuple or a message, or held by a local a closure captures, it is yours to give back with free (free(s.items) empties the field).
  • A handle, a ptr a module gave you (a list, a map, a JSON document, a socket), is the program's to free, through its module, and defer keeps the free next to the new:
import std.list

main() {
    names = list.new()
    defer list.free(names)
    list.add(names, "ada")
    list.add(names, "grace")
    println("${list.size(names)} names")
}
2 names

Common mistakes

  • Freeing what the compiler frees, or not freeing what it leaves. A string needs no free, nor does a make buffer a local holds; freeing a parameter the function only borrows is refused (E0901). A buffer stored in a struct's field is not freed with the struct: free it.
  • Forgetting a handle. A list.new() with no list.free leaks it. Write the defer on the next line.
  • Changing a local's type. count = 0 then count = "many" is a compile error, not a new variable.
  • A panic for an expected failure. Bad input from a user is an error to return (T!), not a panic; panics are for bugs.
  • Reading an element of a list or map as a string without its getter. Elements come back untyped; list.get_string and map.get_string say when one is not a string.

Exercises

1. FizzBuzz. Print 1 to 15, but "Fizz" for multiples of 3, "Buzz" for multiples of 5, and "FizzBuzz" for both.

An answer
fizzbuzz(n: int) -> string {
    if n % 15 == 0 { return "FizzBuzz" }
    if n % 3 == 0 { return "Fizz" }
    if n % 5 == 0 { return "Buzz" }
    return "${n}"
}

main() {
    line = ""
    for i in 1..16 {
        line = "${line}${fizzbuzz(i)} "
    }
    println(line)
}
1 2 Fizz 4 Buzz Fizz 7 8 Fizz Buzz 11 Fizz 13 14 FizzBuzz 

2. The largest. Write largest(xs: int[]) -> int?, none for an empty slice, and print the largest of [4, 11, 7] and of [], with "nothing" for the second.

An answer
largest(xs: int[]) -> int? {
    if xs.len == 0 {
        return none
    }
    best = xs[0]
    for i in 1..xs.len {
        if xs[i] > best { best = xs[i] }
    }
    return best
}

show(v: int?) -> string {
    return match v {
        none -> "nothing"
        some(n) -> "${n}"
    }
}

main() {
    println(show(largest([4, 11, 7])))
    println(show(largest([])))
}
11
nothing

3. A result that can fail. Write divide(a: int, b: int) -> int! that fails with "division by zero", and print divide(9, 3) and, with or and a fallback of -1, divide(1, 0).

An answer
divide(a: int, b: int) -> int! {
    if b == 0 {
        return 0, "division by zero"
    }
    return a / b
}

main() {
    q, err = divide(9, 3)
    println("${q} [${err}]")
    println(divide(1, 0) or -1)
}
3 []
-1

2. Actors

Aether's concurrency is actors: a piece of state that messages reach. Other code sends an actor a message, and the actor changes its own state, one message at a time, so there are no locks to take. The scheduler runs actors on every core, so many actors use the whole machine.

A first actor, its state, and asking it

01-first-actor, 02-state, 03-ask-reply

message declares what can be sent, actor what an actor does with each kind, spawn starts one, ! sends without waiting, and ? sends and waits for the handler's reply:

message Add { n: int }
message Total {}

actor Counter {
    state total = 0
    receive {
        Add(n) -> { total = total + n }
        Total() -> { reply total }
    }
}

main() {
    c = spawn(Counter())
    c ! Add { n: 5 }
    c ! Add { n: 7 }
    int t = c ? Total {}
    println("total: ${t}")
}
total: 12

The ask sees both adds: messages from one sender reach an actor in the order they were sent. wait_for_idle() is the other way to wait: it returns once every actor has nothing left to do, receive timeouts that are counting down included, so main can go on knowing what they did. (The end of main waits for the messages already sent anyway.)

Actors that talk to each other

04-actor-refs, 05-messages, 06-pipeline, 07-worker-pool

What spawn returns is an actor_ref, a value that goes into messages and state, so an actor can be told whom to answer; self is an actor's own. A message owns the strings and arrays in it, so the sender can reuse its variables at once; a ptr in a message hands the handle over. With refs come the shapes of concurrent programs: a pipeline of stages, and a pool of workers whose results a collector gathers. Order holds between one sender and one receiver, never between two senders.

Shapes of an actor's life

08-state-machine, 09-timeouts

An enum as an actor's state, with a transition table, makes a state machine that applies one event at a time. receive { ... } after N -> { ... } does something when no message has come for N milliseconds: a watchdog.

When actors fail

10-crash-isolation, 11-ending, 12-links, 13-supervisor

A panic in a handler ends that actor and nothing else. An actor also ends when it stops itself (actors.stop()) or something kills it (actors.kill(ref)), and whoever monitors it gets a Down with the reason. A link ties two actors so that one's failure ends the other, unless that one traps exits and takes the Down instead. A supervisor is built from those: it links its children, and when one fails it restarts it in place, so every ref to the child reaches the new one.

Holding a producer back

14-bounded-mailbox, 15-streams, 16-sharded-map

A send never blocks, so an actor that receives faster than it handles grows its mailbox without bound. @mailbox(bound: N) caps it: past N, a send is refused at once, and try_send says so. std.stream bounds a flow between two actors with credits, so a fast producer never has more than a window of chunks in flight. And when one actor's mailbox is the bottleneck, its work can be split over several by key.

Common mistakes

  • Reading an actor's state while it runs. A reference reaches an actor's state (c.total), but read while the actor is handling messages it races them. Ask it (?), or read after wait_for_idle().
  • Expecting order between two senders. Main and another actor sending to the same actor interleave in any order. Wait for one before the other, or route both through one sender.
  • Calling wait_for_idle() inside an actor. It waits for every actor, this one too, so it can never return there; it panics instead. Call it from main.
  • Blocking in a handler. A handler that sleeps, waits on a socket or computes for long stalls every actor that shares its core. Keep handlers short, and hand blocking work to std.worker, a thread pool outside the scheduler.
  • Using a handle after sending it. A ptr sent in a message belongs to the receiver now; the sender must not touch it again.

Exercises

1. A bank account. An actor with a balance takes Deposit { n } and Withdraw { n }, refusing a withdrawal larger than the balance, and answers Balance {}. Deposit 100, withdraw 30, then try to withdraw 500, and print the balance.

An answer
message Deposit { n: int }
message Withdraw { n: int }
message Balance {}

actor Account {
    state balance = 0
    receive {
        Deposit(n) -> { balance = balance + n }
        Withdraw(n) -> {
            if n > balance {
                println("refused: ${n} is more than ${balance}")
            } else {
                balance = balance - n
            }
        }
        Balance() -> { reply balance }
    }
}

main() {
    a = spawn(Account())
    a ! Deposit { n: 100 }
    a ! Withdraw { n: 30 }
    a ! Withdraw { n: 500 }
    int b = a ? Balance {}
    println("balance: ${b}")
}
refused: 500 is more than 70
balance: 70

2. A two-stage pipeline. A Doubler doubles each number it gets and sends the result on to a Printer, which prints it. Send 1, 2 and 3 through.

An answer
message Num { n: int }
message Connect { next: actor_ref }

actor Printer {
    receive {
        Num(n) -> { println("got ${n}") }
    }
}

actor Doubler {
    state next: actor_ref = null
    receive {
        Connect(next: n) -> { next = n }
        Num(n) -> { next ! Num { n: n * 2 } }
    }
}

main() {
    printer = spawn(Printer())
    doubler = spawn(Doubler())
    doubler ! Connect { next: printer }
    for i in 1..4 {
        doubler ! Num { n: i }
    }
    wait_for_idle()
}
got 2
got 4
got 6

3. The standard library

01-files · 02-json · 03-processes · 04-http · 05-http-server · 06-http2-tls · 07-reverse-proxy · 08-proxy-pool · 09-logging · 10-tcp · 11-cryptography

The standard library follows two rules from chapter 1 everywhere. A call that can fail returns its value and an error string, so a program checks err and goes on. What a module hands out as a ptr (a JSON document, a server, a socket) is a handle the program frees through that module.

  • Files and processes. std.fs reads, writes and walks the file system. std.os runs a program from an argument list, with no shell in between, and reads what it prints and its status.
  • Data. std.json parses into a tree the program frees once, and builds documents to encode.
  • The network. std.http serves by route and fetches: the same server API carries middleware, health probes and metrics, HTTP/2 and TLS, and std.http.proxy forwards to upstreams. std.tcp is the socket itself, the layer HTTP runs on: a byte stream, so a protocol says where a message ends.
  • Logs and secrets. std.log writes levelled lines. The std.cryptography modules are written in Aether: hashes, HMACs, signatures, and the pure-Aether TLS.

The standard library reference lists every module and call.


4. C and other languages

01-libc · 02-own-c · 03-callbacks · 04-embedding · 06-java-host · 07-limits

Aether compiles to C, so C is one declaration away in both directions. extern declares a C function to call; @source ships a C file with a module; @c_callback hands an Aether function to C as a function pointer. The other way, ae build --emit=obj (or --emit=lib) makes an Aether file a library a C program links, each function an aether_<name> export.

A program that runs Aether code it does not control bounds it: every export runs in an execution frame, so a panic, a deadline, a cancel from another thread or a memory budget ends the call and gives the host an outcome. A deadline or a cancel lands at the call's next safe point, a loop head, so code with no loop (straight-line code, or a long C call) runs on until it returns. Other languages (Lua, Python, Tcl, JavaScript, Go and more) run under the sandbox of chapter 5 through aether-guest-languages, a separate repository. C interoperability and emit=lib are the references.


5. Containing code

01-sandbox · 02-trusted-calls · 03-audit · 04-spawn-sandboxed · 05-capsicum · 06-casper

A sandbox is a list of grants: files it may read or write, environment variables, programs, network hosts. sandbox.enforce(grants) callback { ... } runs a block under them, and anything not granted is refused, the refusal recorded where std.audit can read it back. A trusted call lets the block use one service with your authority. A child process can be held to grants at its own libc calls, and on FreeBSD, Capsicum has the kernel enforce it. The containment sandbox is the reference.


6. Tools

01-testing · 02-mutation · 03-project · 04-lib-path · 05-ae-help

std.spec writes tests (describe, it, assertions) and ae test finds and runs them; std.mutation checks that the tests would notice a change to the code. A project is a directory with an aether.toml, its program in src/ and its modules in lib/. ae help explains what the compiler found in a script and suggests the fix. Testing and the formatter cover the rest of the everyday tools.


7. Programs

01-word-count · 02-chat-room · 03-build-supervisor · 04-calculator

The last part puts the others together: a word counter (files, sorting, errors), a chat room (actors keeping each other in state), a build supervisor (a process with a deadline and nothing left behind) and a terminal calculator (a builder DSL with closures). Read them for how the pieces fit, then write your own.

The path ends with Rosetta Code tasks: classic problems (FizzBuzz, the N queens, quicksort, Levenshtein distance) solved in Aether, with their length beside the Go and Python solutions. They are good exercises too: try one before reading its file.

Quick reference

// Values
x = 42                         // a local's type comes from its value
long big = 3_000_000_000       // or is written before the name
const LIMIT = 10
name = "Ada"
println("${name} is ${x}")     // interpolation

// Control flow
if x > 10 { ... } else if x > 5 { ... } else { ... }
kind = if x > 10 { "big" } else { "small" }
while x > 0 { x = x - 1 }
for i in 0..10 { ... }         // 0 to 9
for item in xs { ... }

// Functions
add(a: int, b: int) -> int { return a + b }
square(x: int) -> x * x
split(n: int) -> (int, int) { return n / 2, n % 2 }
greet(name: string, greeting: string = "Hello") -> string { ... }
fact(0) -> 1
fact(n) when n > 0 -> n * fact(n - 1)

// Data
struct Point { x: int y: int }
p = Point { x: 1, y: 2 }
enum Color { Red, Green }
type Shape = Circle | Rect
area = match s { Circle -> ... Rect -> ... }
int? maybe = none
v = maybe ?? 0

// Failure
parse(s: string) -> int! { ... return 0, "why" }
n = parse("7") or 0
try { ... } catch msg { ... }
defer list.free(items)

// Actors
message Add { n: int }
message Total {}
actor Counter {
    state total = 0
    receive {
        Add(n) -> { total = total + n }
        Total() -> { reply total }
    }
}
c = spawn(Counter())
c ! Add { n: 5 }               // send
int t = c ? Total {}           // ask
wait_for_idle()

Where next