Syntax

The surface syntax of blr source files (.blr). This page is a description only — rationale, research, and decisions live in the knowledge vault (knowledge/design/syntax); the migration plan lives in knowledge/plans/syntax-rewrite.

Comments

• // starts a line comment, running to end of line.
• A // comment placed directly above an item, with no blank line between, is the item’s doc comment.

Identifiers

One class of identifiers. Case is a convention enforced by lint, not by the grammar:

• snake_case — values, variables, functions, modules, record fields, parameters.
• PascalCase — user-defined types and sum case labels.
• 'a — type variables (OCaml style); ' followed by a lowercase identifier.

Keywords

pub, fn, type, let, import, match, extern, resource, linear, select, from

true and false are built-in identifiers (constants of bool), not keywords.

Punctuation

Items

Items appear at the top level of a file. A .blr file is a module: items plus at most one top-level expression. The top-level expression, when present, is the module’s main (the script form); pub fn main is the named form of the same entry point.

Modifiers precede the item keyword; the order is fixed: pub extern fn, never extern pub fn.

Types

In a sum definition, case names are written bare; they are defined as T::Case for the sum T. A label that is a blr keyword is written with a leading backtick: [ Let unit, Match unit ].

DataFrame<'a> is a data frame typed by the row (record) type of its data; the row parameter is a phantom — a frame value is the same opaque resource handle regardless of its row type. CachedDataFrame<'a> is the typed form of a cached data frame: a materialized table that serves fresh frames without re-executing the pipeline that built it. DataTable<'a> is the materialized columnar result of consuming a frame. The untyped handles behind these — Frame, CachedFrame, and Table — are ordinary named resources declared by the df module itself (see Frames and caches).

Named resources

A module can declare its own resource types with a top-level resource declaration:

pub resource Widget
pub resource linear Ticket

A resource declaration is an item, like an extern fn or a native fn, not a type alias: the declaration itself is the type’s identity, and the name is what identifies the resource to the host (converted to kebab-case, so Widget crosses the boundary as the WIT widget resource). A declared name may still be used anywhere a type is written (an extern fn signature, an alias body), but it cannot be re-aliased to a different type.

A named resource is an opaque handle that only has meaning across the component boundary: the host module that provides the extern fns naming the type owns the representation. The type is nominal — two declarations with the same name in the same module are the same type. A resource value cannot be constructed in blr; it only arrives from an extern fn result and flows back to one.

A bare resource is shared: values of the type may be passed around and re-used freely (at the boundary they cross as borrow handles in parameter position and own handles in result position). Adding linear opts the type into use-exactly-once, exactly like frames: each value must be consumed by one extern fn call.

The df standard-library module declares its own boundary resources — a linear Frame and shared CachedFrame and Table — with exactly this syntax; they are not special to the compiler. resource and resource linear therefore cover every resource, including the data-frame family.

Linearity

Linearity is a property of types, not of names: a value is linear iff its type contains a frame. Linear values must be used exactly once; using one zero times or two times is a compile-time error.

Rules, checked after type inference:

1. Exactly once. Each variable bound to a linear type occurs exactly once in its binding scope. let bindings and function parameters alike.
2. Whole-value movement. Linear values may be constructed (in lists, records, sums), passed as arguments, and returned freely — these move the value whole.
3. Destructuring.
   • Projection (r.f) of a record that contains a linear field is legal only when the rest of the record is statically empty; projecting {f1: DataFrame<'a>, n: i64} as r.f1 is an error, because n’s sibling fields are discarded without being consumed.
   • Branching (match) on a sum containing a linear field is forbidden: the unchosen variant would be silently discarded.
4. No linear captures. A linear value may not be referenced inside a nested closure. (Full linear-closure support — where a closure’s linearity depends on its captures — is a deliberate later phase.)

// error: `f` is used twice
let f = select {a, b, c} from df::fromcsv("test.csv");
df::count(f) + df::count(f)

// error: `f` is never used
let f = select {a, b, c} from df::fromcsv("test.csv");
0

Lists

A list is a fixed-size, element-major sequence built with [v, ...]. The std::list module provides the basic operations:

map and fold take closures, which may be inline (fn (x) -> x * 2) or function items (double), so generic functions are monomorphized per list element type as with any application.

l[i] is bounds-checked at runtime: an out-of-range index (negative or >= length) aborts the program with a runtime error (see below).

Indexing a list of linear elements is a compile-time error. l[i] discards every element other than the ith, and unselected linear values may not be dropped (see Linearity):

import std::df

// error: indexing a list of linear elements is not allowed
pub fn main() -> i64 {
    let l = [df::fromcsv("a.csv"), df::fromcsv("b.csv")];
    df::count(l[0])
}

Selecting all elements through a map or fold closure is fine: every element is consumed by the closure, none is silently dropped.

Runtime errors

std::runtime::error(msg: string) -> unit signals an unrecoverable runtime error: the message is printed to stderr and the program aborts. It never returns, so it is used as the final expression of a -> unit function.

The compiler uses it for its own runtime checks, so a known failure reports a meaningful message instead of a bare trap:

pub fn main() -> i64 {
    [1, 2, 3][5] // aborts: "list index out of bounds"
}

Frames and caches

A data frame is linear: it is an uncomputed pipeline (read CSV, filter, limit, …), and consuming it means executing it exactly once. There is no implicit memoization — reuse of a frame is a type error, so “compute once, use many” must be written as an explicit copy:

import std::df

pub fn main() -> i64 {
    let d = select {a, b, c} from df::fromcsv("test.csv") | df::filter(#{ fn (r) -> r.a > 0 }) | df::cache();
    let a = d | df::to_df() | df::filter(#{ fn (r) -> r.b > 0 }) | df::count();
    let b = d | df::to_df() | df::count();
    a + b
}

• df::cache(frame) consumes the linear frame, executes its pipeline, and materializes the result into a cached table. The result, CachedDataFrame<'a>, is non-linear: it may be named and re-used as often as you like.
• df::to_df(cached) derives a fresh linear frame from the cache. Each derived frame scans the cached table — the original pipeline never re-executes — and is consumed exactly once like any frame.

Caching is the one place a frame’s data is copied; it is deliberate and observable in the source, which is why the rest of the language can stay linear without hidden cost.

Frame sources: select

A frame source is an item that reads a table into a fresh frame — today df::fromcsv (and influxdb::query for query sources). A source call names the row the frame carries with a select form:

// typed row: the columns this frame is expected to carry
select {name: string, total: int} from df::fromcsv("sales.csv")

// shorthand: bare labels leave the field types free (each is `label: _`)
select {name, total} from df::fromcsv("sales.csv")

// mixed: typed and free fields in one row
select {name: string, total, region} from df::fromcsv("sales.csv")

// open row: nothing declared, nothing validated
select ? from df::fromcsv("sales.csv")

The select … from … form is a single expression — the source — and a pipeline continues from it:

pub fn main() -> DataTable<'a> {
    select {a, b} from df::fromcsv("groups.csv")
        | df::filter(\r -> r.a > 1)
        | df::aggregate(\r -> {a: r.a}, \c -> {total: sum(c.b)})
        | df::collect()
}

• The declared row is ? (the open row) or a row literal whose fields are either label: type or a bare label (a free field type, the surface spelling of a type hole). A field type must be concrete: it may not name type or row variables.
• The operand of from must be a single source application — a call whose head is a source item. Anything else (a bare reference, a non-source call, a parenthesized expression, a pipeline) is a compile error: `select` requires a source (e.g. `df::fromcsv`, `influxdb::query`). The rule also rules out the pipeline ambiguity of select {a} from X | df::aggregate(…): there is no reading in which the pipeline is the operand, so the form parses exactly one way.
• The declared row is the frame’s view/schema: the closed set of fields nameable downstream. The underlying table may carry more columns; a field not in the declared row is a type error (reference to non-existent field b on {a: i64}). With the open row (select ?) nothing is declared and nothing is validated.

A source call without a select form is a compile error: the open mode is always visible at the source, never implicit.

Expressions

A quoted expression captures its contents as data. Only static structure is quotable for now: quoting an expression that references a value from the enclosing scope is not yet supported (the general “what is quotable” design is tracked separately).

The tick on a sum injection marks it as a value construction. The label is the final segment of the path; the owner (T::, import::T::) is for the reader, and the anonymous form 'Case(v) is the same value without it. The injected value is type-checked like any other value against the declared type of its position: where type T is expected, the case must exist in T with a compatible payload, and a missing case is a row-mismatch type error, like a bad match arm.

Example

// Filter the frame to rows where b > a, then show it.
import std::df

pub fn main() -> i64 {
    let f = #{ fn (r) -> r.b > r.a };
    select {a, b, c} from df::fromcsv("test.csv") | df::filter(f) | df::show()
}

pub type Tiny = [ A unit, B i64 ]

pub fn classify(t: Tiny) -> i64 {
    match t {
        Tiny::A(_) => 0,
        Tiny::B(n) => n,
    }
}

Contributing to blr

Patches are welcome. Submit them via email to ~nathanielc/blr-devel@lists.sr.ht.

For instructions on configuring git send-email, see here.

Here’s a quick start:

1. Set your global SMTP settings:

   git config --global sendemail.smtpHost smtp.example.com
   git config --global sendemail.smtpUser your@email.com
   git config --global sendemail.smtpPass yourpassword
   

2. Set the mailing list address for this repo:

   git config sendemail.to '~nathanielc/blr-devel@lists.sr.ht'
   

   Both of these settings only need to be done once.

3. Send your patch:

   git send-email HEAD^
   

   git send-email will generate the patch from your latest commit automatically and send it.

   Expect feedback and all communication about the patch in reply to your email.

4. Send updated patch after you have made any changes:

   git send-email -v2 HEAD^
   

Bug Reports & Tickets

Found a bug or have a feature request? Search for existing tickets first:

https://todo.sr.ht/~nathanielc/blr

If you don’t find a matching ticket, submit a new one using the same link.

Announcements

Subscribe to ~nathanielc/blr-announce@lists.sr.ht to follow updates on the project. Send an empty email to ~nathanielc/blr-announce+subscribe@lists.sr.ht to subscribe.