Plan: land ticket 46 with surface row variables (replacing the RowConcat evidence)
1. Branch map
| Branch | Tip | What it is | Disposition |
|---|---|---|---|
master | 4c3d9ee | origin/master | base |
df-aggregate-46-clean | aaf58a7 | the submitted 6-commit series (C1–C6) | kept as reference for the sent series; content re-derived below |
rework-backup | 41a1afb | pre-rework single commit; tree-identical to the clean series | delete (subsumed) |
aggregate-46 | b9f3ece | earlier 2-phase WIP | delete (subsumed) |
df-aggregate-surface | 95e44b9 | WIP with debug instrumentation | delete (subsumed) |
row-monomorph | 703a658 | P0: row monomorph in the mantle, on master | source of S1 |
All feature work hangs off the same master (4c3d9ee). The new integrated series is
built on a fresh branch (e.g. df-aggregate-46-rows).
2. P1 review: what it is, compared to the submitted series
The submitted C3 (4f1b953, 443/−208 across 11 files) introduced the RowConcat
evidence mechanism to express "'gr .. 'b is the concat of rows 'gr and 'b":
Evidence::RowConcat— a new evidence variant whose three type variables stand
for product rows.- A definition-site constraint dropped (desugaring would bind interface type
vars to body-local product rows, collapsing polymorphism). - At every call site,
instantiate/type_check_with_itemsdesugar the
evidence intoTypeEqual×3 +RowCombine(infer.rs ~392, mod.rs ~1474). reify_row_concat_evidence(subst.rs) rewrites bound unifiers back into the
scheme so the equation survives generalization.- Mantle: the evidence lowers to an empty dictionary record (no AST node
references it), with a corresponding empty-record minimum-alignment fix in the
nucleus. - Surface:
..as a type-level operator (ConcatType).
P1 (surface row variables) makes all of that machinery unnecessary, because the
crust already has first-class rows:
Row::Open(RowVar)insideType::Prod,TypeScheme.unbound_rows(already
serialized, already iterated byinstantiate, which mapsOpen → Unifier
with fresh row unifiers and returnswrapper_rowvars).Instantiate::evidencealready turnsEvidence::RowEquationintoConstraint::RowCombineat call sites (inst.rs:38) — the existing evidence
type used by quotes and projections.substitute_wrapperalready re-emitsRowEquationevidence through
generalization (subst.rs:280) — no reification step needed, because rows live
in the scheme'sunbound_rows, not hidden inside type variables.- The solver (
unify_row_comb,partial_row_combs,close_unbound_rows)
already solves row combinations over open rows and keeps interface-reachable
goals open. - The mantle's
lower_ty_schemealready builds akindsvector
(Type/Row per parameter) and foldsType::ty_fun(kind, …)for each —
row parameters becomeTypAbs(Kind::Row). - The call-site
TypApp::Rowemission (mantle Application lowering, thewrapper.rowsfold) already exists — it is dead code today precisely because
the surface can't produce row-polymorphic items. - P0 (S1) makes
TypAbs(Kind::Row)instantiate per applied row in monomorph
and beta-reduce in simplify.
The new surface design (explicit conversion, no promotion):
- Row positions:
DataFrame<…>,CachedDataFrame<…>, and (new)Transpose<…>take a row expression: a row variable'x, a concat'x .. 'y(left-associative), or a row literal{ … }(closed fields
and/or a fragment). - Row variables are row variables in all contexts. A
'xin a row
position is a row variable, full stop; there is no kind pre-pass and no
implicit promotion. Where a type position needs the product of a row
variable, it is written explicitly as a record type with a fragment:{'a}means the product of row'a(lowers toProd(Open(r_a))). - The record row node gains an optional fragment:
{ a: i64 }(closed,
today's syntax),{'a}(fragment only), and{ a: i64, 'b}(fields plus
fragment; the mixed form desugars to a fresh row variable plus aRowEquation, reusing the..mechanism). The same node is read as a
row in row positions and asProd(row)in type positions. ..desugars at lowering to a fresh row variablewplusEvidence::RowEquation { left: 'x, right: 'y, goal: w }in the scheme'sevidence— whichlower_typalready returns and whichitem_source.register_with_visibilityalready puts into the item scheme.
No new evidence variant, no desugaring at call sites (the existing
RowEquation path does it), no reification, no empty-dict record.'xin other type positions (function arrows,Quote<…>,[]…,
sums, …) is a type variable exactly as today — unchanged semantics.- Definition-site enforcement is possible (unlike RowConcat): the equation
references only interface rows, so it does not collapse polymorphism; the
solver keeps the goal open (reachable from the signature) and the equation
re-emerges in the scheme. Checkpoint during S4: confirm the solver handles
an interface-onlyRowCombineat the definition site without over-constraining.
Backward compatibility: DataFrame<'a> / CachedDataFrame<'a> switch from a
type variable to a row variable (the position determines the kind); any use
of that name in a type position (e.g. Quote<'a -> bool>) must be writtenQuote<{'a} -> bool> — an explicit, reviewable migration in S3 (df.blr and
the df type_check tests). Elsewhere, 'x in type positions is untouched.
The expected observable change is snapshot churn
(DataFrame(Var(TypeVar(0))) → DataFrame(Prod(Open(RowVar(0))))); the 174
integration tests are the behavioral oracle.
3. The new series (replaces the sent 6 commits)
Each commit: full suite green + cargo fmt --check + clippy clean. Built ondf-aggregate-46-rows off master.
| # | Commit | Source | Notes |
|---|---|---|---|
| S1 | mantle: instantiate row abstractions in monomorph and simplify | cherry-pick 703a658 (row-monomorph), as-is | P0. Kind-aware substitution, row beta-reduction in simplify, per-row instantiation with row-aware symbol mangling. Row application to non-local items is an ICE here; S6 replaces it with the instance mechanism. |
| S2 | type: Transpose type for the columnar row view | cherry-pick C1 3eeb5a5, as-is | RowConcat-independent (verified: only Transpose unification/injectivity, the Project check arm, bidirectional item application). |
| S3 | air/crust: row variables in DataFrame and CachedDataFrame positions | new (P1a) | Grammar: the DataFrame<…>/CachedDataFrame<…> parameter becomes a row expression (row variable, row literal). The record row node gains an optional fragment: {'a} lowers to Prod(Open(r_a)) (the explicit product conversion); { a: i64 } is unchanged. No kind pre-pass, no promotion. Lowering: DataFrame(row) → Type::DataFrame(Type::Prod(row)); unbound_rows from the existing RowVarSupply. ExtConvert: externs keep Frame/CachedFrame (row positions panic, like C2's Transpose). Migration: df.blr and df type_check tests update Quote<'a -> …> uses of a frame row to Quote<{'a} -> …>. The big compatibility commit: snapshot churn across all df type_check tests; the df e2e suite must pass behaviorally unchanged. |
| S4 | air/crust: row concat 'x .. 'y as RowEquation evidence | new (P1b) | Grammar: 'x .. 'y in row positions (token exists; the old series parsed it at type level, so LALR precedent). Lowering: fresh row var + Evidence::RowEquation into lower_typ's TypeScheme.evidence (plumbing already exists end to end). The mixed row literal { a: i64, 'b} reuses the same equation. Type_check tests: a concat signature's .typ shows the evidence; a type-kind name in a row position is an error. |
| S5 | air: Transpose<row> surface type syntax | C2 6e824a8 adapted | Transpose < RowExpr > (row position, so Transpose<'a> gets a row variable directly instead of a promoted type). Crust side: TypeExpr::Transpose → Type::Transpose(Prod(row)); the C1 machinery is type-level and unchanged. |
| S6 | monomorph: instantiate items with several type and row parameters | C4 e1213be adapted + row extension | Instance mechanism HashMap<Symbol, Vec<Vec<Type>>> → N-ary applied args of kind Type or Row (e.g. Vec<Vec<InstanceArg>>); mangled symbols join every arg's sexpr (Row sexpr exists). The monomorph Item arm's row ICE from S1 becomes the non-local row instantiation path; the call-site wrapper.rows TypApp::Row fold goes live. Nucleus minimum-alignment for empty records is kept (defensive; it was originally for the RowConcat empty dict). Checkpoint: lower_ty_scheme kinds ordering (the .rev() that C3 removed) with mixed (type, row) parameters — verify call-site TypApp order matches TypAbs order. |
| S7 | df: aggregate host function and the std lib df::aggregate | C5 235f1f4 adapted | Host fn, WIT, registration unchanged (identity-based). df.blr in the new syntax: pub fn aggregate(frame: DataFrame<'a>, group: Quote<{'a} -> {'gr}>, agg: Quote<Transpose<'a> -> {'b}>) -> DataFrame<'gr .. 'b> — row variables in row positions, explicit {'a} products at the Quote positions, .. at the result row. e2e tests (df_aggregate*, int_groups.csv) carry over; type_check ok tests updated for the row syntax (agg_row_concat becomes a row-equation test). |
| S8 | type: compile-time normal-form check for df::aggregate | cherry-pick C6 aaf58a7, as-is | Identity-based (STD_DF_AGGREGATE_OPS); no dependence on the type mechanism. |
What gets deleted (never exists in the new series)
Evidence::RowConcatand every arm around it (crust enum, sexpr, inst, linearity-adjacent handling).- The
TypeEqual×3 +RowCombinedesugaring at both call sites (infer.rsinstantiate,type_check_with_items). reify_row_concat_evidence(subst.rs).- The definition-site-drop behavior (no longer needed — see §2).
- The mantle empty-dictionary special case for RowConcat in
lower_ev_ty/ item lowering. - The type-level
..(ConcatType) surface syntax and its air nodes.
What changes shape
Transpose<'a>: type-variable parameter → row position (S5).- All
DataFrame<'a>/CachedDataFrame<'a>surface signatures: type variable →
row variable (S3); type-position uses of a frame row are written{'a}
(S3 migration). - The record row syntax gains an optional fragment
{'a}/{ a: i64, 'b}(S3/S4). - The instance mechanism: N-ary types → N-ary (type|row) args (S6).
4. Risks / checkpoints
- S3 compatibility — field access inside quotes against
Prod(Open(r))
(vs today's bare unifier that the solver treats as a product). The 174-test
e2e suite + type_check snapshots are the oracle; expect localized solver or
snapshot fixes, not redesigns. - S3 migration surface — find every type-position use of a frame-row name
(df.blr, type_check tests) that must become{'a}; the compile errors are
the checklist. lower_ty_schemeparameter ordering — thekinds/.rev()handling with
mixed (type, row) parameters; the call-siteTypAppapplication order must
match theTypAbsorder in the scheme. Verify in S3 and again in S6.lower_ty_schemeparameter ordering — thekinds/.rev()handling with
mixed (type, row) parameters; the call-siteTypAppapplication order must
match theTypAbsorder in the scheme. Verify in S3 and again in S6.- Definition-site RowEquation (S4) — confirm the solver keeps an
interface-onlyRowCombinepartial and re-emits the evidence unchanged
throughsubstitute_wrapper. - LALR conflicts (S3/S4) —
'identis already aTypeExpr; the row
expression non-terminal and the record-fragment alternative are separate
production contexts, but the{…}row literal shared between row and type
positions and the..in row positions need the generated parser checked. - Entry items (
main(d: DataFrame<'a>)) remain broken for both kinds
(pre-existing nucleus gap, verified empirically); out of scope. - Ticket #47 (duplicate wasm export) is orthogonal; row-aware mangling
makes per-row instances distinct but does not fix the underlying dedup.
5. Process
- Everything stays under ticket 46 (no new ticket): all commits reference
Ticket: #46. - The finished series is one
git send-email master..HEADreplacing the sent
6-commit series (note the replacement in the email body and a ticket 46
comment). - Delete
rework-backup,aggregate-46,df-aggregate-surfaceonce the new
series is sent; keepdf-aggregate-46-cleanas the sent-series reference
until 46 lands. - Ticket 46 stays in_progress (under review) throughout; status changes only by
the user.