diff --git a/book/src/SUMMARY.md b/book/src/SUMMARY.md index 1d47e9ce0..e92c24f13 100644 --- a/book/src/SUMMARY.md +++ b/book/src/SUMMARY.md @@ -61,6 +61,7 @@ - [Errors](super-sql/types/error.md) - [Named Types](super-sql/types/named.md) - [Nulls](super-sql/types/null.md) + - [None and Missing](super-sql/types/none.md) - [Operators](super-sql/operators/intro.md) - [aggregate](super-sql/operators/aggregate.md) - [assert](super-sql/operators/assert.md) diff --git a/book/src/formats/model.md b/book/src/formats/model.md index 3c0d198de..98d775469 100644 --- a/book/src/formats/model.md +++ b/book/src/formats/model.md @@ -77,16 +77,22 @@ type `T` or null is a [union](#25-union) of `T` and `null`, e.g., a nullable string has type `string|null`. The _missing_ type is a primitive type whose only value is `missing`, -which represents the absence of a value, e.g., in a record type formed by -[type fusion](../super-sql/type-fusion.md), the value of -a field that is absent from the original record. - -The _none_ type is a primitive type whose only value is `none`. Its -primary purpose is to be cast to an [option](#29-option) type, yielding -that option's "none" value. It is also the element type of an empty array +which marks a field that is absent from some of the values described by +one type, e.g., a field of a record type formed by +[type fusion](../super-sql/type-fusion.md) that is absent from some of +the original records. + +The _none_ type is a primitive type whose only value is `none`, which +means there is no value, e.g., the result of +[ok](../super-sql/functions/generics/ok.md) applied to an error. +Cast to an [option](#29-option) type, `none` yields that option's "none" +value. It is also the element type of an empty array or set and the key and value type of an empty map whose types are not otherwise known. +See [None and Missing](../super-sql/types/none.md) for how SuperSQL +treats these two values. + ### 1.2 All and Any The _any_ type is the top type of the type system: every type conforms diff --git a/book/src/super-sql/expressions/comparisons.md b/book/src/super-sql/expressions/comparisons.md index 85230a6d0..c2c271627 100644 --- a/book/src/super-sql/expressions/comparisons.md +++ b/book/src/super-sql/expressions/comparisons.md @@ -103,12 +103,10 @@ values {isNull:this is null,isNotNull:this is not null} 1 null 2 -missing # expected output {isNull:false,isNotNull:true} {isNull:true,isNotNull:false} {isNull:false,isNotNull:true} -{isNull:false,isNotNull:true} ``` --- diff --git a/book/src/super-sql/types/error.md b/book/src/super-sql/types/error.md index 89b8f4fc5..9f414e5e7 100644 --- a/book/src/super-sql/types/error.md +++ b/book/src/super-sql/types/error.md @@ -146,6 +146,17 @@ values x error("no such field x") ``` +SuperSQL does have a type called `missing`, but it plays a different role. +When values of different shapes are described by one type, e.g., after +[type fusion](../type-fusion.md), the type must say which fields are absent +from some of the values. A field of type `int64|missing`, for instance, +is an `int64` in the records that include the field and absent from records +that do not include it. +So `missing` describes the shape of data rather than being a value that a +query produces: reading such a field where it's absent is still an error, +as above. +See [None and Missing](none.md) for how `missing` relates to `none`. + ## Examples --- diff --git a/book/src/super-sql/types/intro.md b/book/src/super-sql/types/intro.md index 6824a6995..3bb2a8d82 100644 --- a/book/src/super-sql/types/intro.md +++ b/book/src/super-sql/types/intro.md @@ -8,6 +8,7 @@ comprising [sum types](union.md), [named types](named.md), the [null type](null.md), +the [none and missing types](none.md), and first-class [errors](error.md) and [types](type.md). diff --git a/book/src/super-sql/types/none.md b/book/src/super-sql/types/none.md new file mode 100644 index 000000000..d60909017 --- /dev/null +++ b/book/src/super-sql/types/none.md @@ -0,0 +1,130 @@ +# None and Missing + +SuperSQL has two types for "nothing here," each with just one value: +* `none`, a value meaning that there is no value, and +* `missing`, which marks a field that's absent from some of the values + described by one type. + +They answer different questions. A `none` says _the answer is nothing_, +while `missing` says _the field isn't there_. You compute with `none`. +You don't compute with `missing`: SuperSQL reports it so you can say +what you want, usually by converting it to `none` with +[ok](../functions/generics/ok.md). + +## None + +A plain `none` is the result of an expression that produces nothing, e.g., +* [ok](../functions/generics/ok.md) of an error, +* `?.` applied to `none`, +* a [type assertion](../expressions/type-assertions.md) that fails, +* a [match](../expressions/match.md) with no matching case and no `else`, + or +* a pipe subquery that produces no values. + +A `none` is handled quietly rather than reported: +* it's dropped from the output of the [values](../operators/values.md) + operator, so it's never passed to a downstream operator, +* it disappears from a record expression, an array, a set, or a map, +* [aggregate functions](../aggregates/intro.md) skip it, +* `??` replaces it with another value, and +* `?.` passes it through a field access. + +For example, +```mdtest-spq +# spq +values {a:this, b:ok(10/this)}, ok(10/this) ?? "bad value" +# input +2 +0 +# expected output +{a:2,b:5} +5 +{a:0} +"bad value" +``` + +The `none` of an [option type](../../formats/model.md#29-option), e.g., +`none::int64`, isn't a plain `none`. It's a value of its option type, and does +it does not disappear from record fields or from array/set/map values like +a plain `none` does. + +## Missing + +A `missing` never comes from an expression. It appears only in data whose +different shapes are described by one type, e.g., the records produced by +the [fuse](../operators/fuse.md) operator: +```mdtest-spq +# spq +fuse +# input +{a:1} +{b:"x"} +# expected output +{a:1::(int64|missing),b:missing::(string|missing)} +{a:missing::(int64|missing),b:"x"::(string|missing)} +``` +Here, the type `int64|missing` says that `a` is absent from some of the +values, and each record holds `missing` for the field it doesn't have. + +Reading a field where it's absent is an [error](error.md), just as it is for +a field that doesn't exist at all: +```mdtest-spq +# spq +fuse | values b +# input +{a:1} +{b:"x"} +# expected output +error("no such field b") +"x"::(string|missing) +``` + +When the types of a query's input are known, a field that may be absent is +reported before the query runs: +```mdtest-spq fails +# spq +values {a:1},{b:2} | values a ?? 0 +# input + +# expected output +field may be missing (consider ok()) at line 1, column 29: +values {a:1},{b:2} | values a ?? 0 + ~ +``` +Note that `??` and `?.` handle `none`, not `missing`. To handle a field +that may be absent, convert it to `none` with +[ok](../functions/generics/ok.md), then drop the `none` or replace it with +`??`: +```mdtest-spq +# spq +values {a:1},{b:2} | values {input:this, ok:a.ok(), with_default:a.ok() ?? 0} +# input + +# expected output +{input:{a:1},ok:1,with_default:1} +{input:{b:2},with_default:0} +``` + +## Why Two Types? + +A `none` from `ok()` or a failed type assertion is something the query asked +for, so it's safe to drop it or replace it without comment. + +A field that may be absent is different. It usually means the data doesn't +look the way the query expects, e.g., the field name is misspelled, or only +some of the input has the field. If an absent field quietly became `none`, +such mistakes would silently drop values from the output. + +Keeping the two apart lets SuperSQL drop `none` quietly while still catching +every read of a field that may not be there. Writing `ok()` makes the choice +explicit: where the field is absent, treat it as nothing. + +This resembles the distinction between `null` and `missing` in +[SQL++](https://asterixdb.apache.org/docs/0.9.7.1/sqlpp/manual.html), but in SuperSQL a field that isn't +there is an error rather than a value (see [Errors](error.md#missing)), +and `missing` only marks such fields in the types and values of data. + +>[!NOTE] +> The [SUP](../../formats/sup.md#26-null-missing-and-none-values) format spells +> the missing value `missing` so that data with absent fields can be written +> and read, but `missing` isn't intended for use in queries.