lode - v0.1.0

Embedded Schemas

Embedded schemas let you keep structured data inside a parent row instead of spreading it across extra tables. A user has a profile; an order has a shipping address and a list of line items. None of those nested shapes is interesting on its own — you never query “all profiles” the way you query “all users” — so modelling each as its own table, with its own foreign keys and joins, buys ceremony you do not need. An embed stores the whole nested record as a single jsonb column on the parent, and gives you a typed Gleam record to read it back into.

In lode an embed is exactly that: a child Schema whose dumped columns are encoded to JSON text and written into one column of the parent’s table. There is no second table, no join, and — unlike an association — no separate repo write. The embed lives and dies with its parent row.

To make this concrete we will build a User that carries an embedded Profile and an embedded list of Items, following the same arc as Ecto’s Embedded Schemas guide.

Defining the embedded schema

In Elixir’s Ecto an embedded child is declared with the embedded_schema macro — a special form for a schema that has no data source. lode needs no separate construct. A Schema(row) is an ordinary value: inert metadata plus a load/dump codec, and it only touches a database when you hand it to a repo function. So the schema for a record you never store on its own table is built with the very same schema.new you use for a persisted one — the same value plays both roles (the rationale for explicit, value-level schemas is in DESIGN.md in the repository).

schema.new always takes a source string. For a purely-embedded child there is no table behind it, so the source is just a descriptive label that is never used to read or write a table; it exists so error messages and tooling have a name to print.

import lode/schema.{type Schema}
import lode/types/primitive
import lode/value.{VString}
import gleam/dict
import gleam/result

pub type Profile {
  Profile(first_name: String, last_name: String, bio: String)
}

// `source: "profile"` is a label only — no `profiles` table is ever queried.
fn profile_schema() -> Schema(Profile) {
  schema.new(
    source: "profile",
    fields: [
      schema.field(name: "first_name", of: primitive.string()),
      schema.field(name: "last_name", of: primitive.string()),
      schema.field(name: "bio", of: primitive.string()),
    ],
    load: fn(row) {
      use first_name <- result.try(schema.require_string(row, "first_name"))
      use last_name <- result.try(schema.require_string(row, "last_name"))
      use bio <- result.try(schema.require_string(row, "bio"))
      Ok(Profile(first_name:, last_name:, bio:))
    },
    dump: fn(p: Profile) {
      dict.from_list([
        #("first_name", VString(p.first_name)),
        #("last_name", VString(p.last_name)),
        #("bio", VString(p.bio)),
      ])
    },
  )
}

A list embed is no different — it is the same kind of child schema, attached to the parent with embeds_many rather than embeds_one (we will see the attachment next). Here is an Item we will embed as a list:

pub type Item {
  Item(name: String, quantity: Int)
}

fn item_schema() -> Schema(Item) {
  schema.new(
    source: "item",
    fields: [
      schema.field(name: "name", of: primitive.string()),
      schema.field(name: "quantity", of: primitive.integer()),
    ],
    load: fn(row) {
      use name <- result.try(schema.require_string(row, "name"))
      use quantity <- result.try(schema.require_int(row, "quantity"))
      Ok(Item(name:, quantity:))
    },
    dump: fn(i: Item) {
      dict.from_list([#("name", VString(i.name)), #("quantity", VInt(i.quantity))])
    },
  )
}

Attaching embeds to the parent

You attach an embed by putting an embeds_one/embeds_many type on one of the parent’s fields, exactly where you would otherwise put primitive.string() or primitive.id(). Both constructors live in lode/embed and take a thunk returning the child schema (a thunk so that mutually-recursive schemas can refer to each other):

The returned LodeType is the bridge: it dumps the child record (or list) to a single VString(json) value for the parent’s jsonb column, and loads that JSON text back into the typed child record (or list). Because it is just a field type, the parent’s load and dump handle it through the ordinary schema.load_field / schema.dump_field helpers — the embed-aware encoding is entirely inside the LodeType.

import lode/embed
import lode/schema.{type Schema}
import lode/types/primitive
import lode/value.{VInt, VString}
import gleam/dict
import gleam/result

pub type User {
  User(id: Int, email: String, profile: Profile, items: List(Item))
}

fn user_schema() -> Schema(User) {
  schema.new(
    source: "users",
    fields: [
      schema.primary_key(name: "id", of: primitive.id()),
      schema.field(name: "email", of: primitive.string()),
      schema.field(name: "profile", of: embed.embeds_one(profile_schema)),
      schema.field(name: "items", of: embed.embeds_many(item_schema)),
    ],
    load: fn(row) {
      use id <- result.try(schema.require_int(row, "id"))
      use email <- result.try(schema.require_string(row, "email"))
      use profile <- result.try(schema.load_field(
        row: row,
        name: "profile",
        of: embed.embeds_one(profile_schema),
      ))
      use items <- result.try(schema.load_field(
        row: row,
        name: "items",
        of: embed.embeds_many(item_schema),
      ))
      Ok(User(id:, email:, profile:, items:))
    },
    dump: fn(u: User) {
      dict.from_list([
        #("id", VInt(u.id)),
        #("email", VString(u.email)),
        #("profile", schema.dump_field(
          of: embed.embeds_one(profile_schema),
          value: u.profile,
        )),
        #("items", schema.dump_field(
          of: embed.embeds_many(item_schema),
          value: u.items,
        )),
      ])
    },
  )
}

That is the whole storage story: profile and items are two jsonb columns on users. Inserting a User writes the JSON inline; reading one back parses it into the Profile record and List(Item). Round-tripping is structural, so a Profile("Ada", "Lovelace", "…") comes back equal to what you wrote.

The migration: one jsonb column per embed

Because an embed is a column, not a table, the migration is unremarkable — each embed is a single jsonb column on the parent table. The jsonb type is a first-class DDL column type, so no raw SQL is needed:

import lode/migration
import lode/migration/ddl

pub fn add_users() -> migration.Migration {
  migration.new(20_260_614_120_000)
  |> migration.create_table("users", [
    ddl.column("id", ddl.Serial) |> ddl.primary_key,
    ddl.column("email", ddl.Text) |> ddl.not_null,
    ddl.column("profile", ddl.Jsonb),
    ddl.column("items", ddl.Jsonb),
  ])
}

(See Constraints and Upserts for running migrations and the raw-SQL migration.execute escape hatch.)

Casting external params into embeds

Inserting a fully-built User value is fine when the data is already trusted, but the usual job is casting external params — a form, an API body — and validating them. lode/embed gives you cast_embed_one and cast_embed_many, which mirror changeset.cast_assoc: they take the raw params for the embed and a with: function that turns those params into a child changeset.

This is where the absence of macros shows through cleanly. Ecto’s cast_embed/3 infers the child changeset function by reflection (or you pass with:); lode always asks for the with: function explicitly. It is just a fn(child, params) -> Changeset(child) — typically a changeset.cast followed by the child’s own validations. Nothing is hidden:

import lode/changeset.{type Changeset}
import lode/value.{type Value}
import gleam/dict.{type Dict}
import gleam/option.{Some}

fn profile_changeset(p: Profile, params: Dict(String, Value)) -> Changeset(Profile) {
  changeset.cast(p, profile_schema(), params, field.fields(["first_name", "last_name", "bio"]))
  |> changeset.validate_required(field.fields(["first_name", "last_name"]))
}

fn item_changeset(i: Item, params: Dict(String, Value)) -> Changeset(Item) {
  changeset.cast(i, item_schema(), params, field.fields(["name", "quantity"]))
  |> changeset.validate_required(field.fields(["name", "quantity"]))
  |> changeset.validate_length(field.field("name"), min: Some(1), max: Some(120))
}

Now cast the parent. cast_embed_one takes the embed’s name, a blank data record to cast onto, the embed params (a Dict), and the with: function; cast_embed_many is identical but takes a List(Dict) of params, one per element:

import lode/changeset.{type Changeset}
import lode/embed
import lode/value.{type Value, VString}
import gleam/dict.{type Dict}

fn user_changeset(
  user: User,
  params: Dict(String, Value),
  profile_params: Dict(String, Value),
  item_params: List(Dict(String, Value)),
) -> Changeset(User) {
  changeset.cast(user, user_schema(), params, field.fields(["email"]))
  |> changeset.validate_required(field.fields(["email"]))
  |> embed.cast_embed_one(
    name: "profile",
    data: Profile("", "", ""),
    params: profile_params,
    with: profile_changeset,
  )
  |> embed.cast_embed_many(
    name: "items",
    data: Item("", 0),
    params: item_params,
    with: item_changeset,
  )
}

The crucial difference from associations: an embed has no separate repo write. Where put_assoc/cast_assoc stage a child that the repo later inserts into another table (and reconciles join rows for), cast_embed_* encodes the validated child straight to JSON and stores it in the parent’s changes. When you repo.insert the parent, the jsonb column simply carries that JSON — there is nothing else to write.

import lode/repo.{type Repo}
import lode/error.{type LodeError}

pub fn create_user(
  r: Repo,
  params: Dict(String, Value),
  profile_params: Dict(String, Value),
  item_params: List(Dict(String, Value)),
) -> Result(User, LodeError) {
  let cs =
    user_changeset(
      User(id: 0, email: "", profile: Profile("", "", ""), items: []),
      params,
      profile_params,
      item_params,
    )
  repo.insert(r, user_schema(), cs)
}

If a child changeset is invalid, the parent is marked invalid too: cast_embed_* adds an "is invalid" error on the embed’s field (tagged validation: "cast"), and the subsequent repo.insert fails with ChangesetInvalid rather than writing a half-validated row. So a bad quantity in one item rejects the whole user insert — exactly the all-or-nothing behaviour you want for inline data. (For the changeset/validation toolkit these examples lean on, see Data mapping and validation.)

Reading into the embedded JSON

Because an embed is a real jsonb column, you can reach inside it from a query. The lode/query/expr module provides the Postgres jsonb operators:

To find every user whose embedded profile has a given last name, pull the last_name member out as text and compare it. The profile column is the first (and only) source in the query, so it lives at binding 0 — expr.Col(0, "profile") is the raw column reference. These jsonb helpers return Expr fragments, so the comparison itself is spelled with expr.fragment, splicing the extracted member as its first argument and the literal as a parameter:

import lode/query
import lode/query/expr
import lode/repo.{type Repo}
import lode/error.{type LodeError}
import lode/value.{VString}
import gleam/list

pub fn users_named(r: Repo, last_name: String) -> Result(List(User), LodeError) {
  let q =
    query.from("users", "u")
    |> query.where(expr.fragment("? = ?", [
      expr.json_get_text(expr.Col(0, "profile"), key: "last_name"),
      expr.Lit(VString(last_name)),
    ]))
  repo.all(repo: r, schema: user_schema(), query: q)
}

Containment is handy for the list embed: expr.json_contains with a small JSON document asks Postgres whether items includes an element with given members, e.g. expr.json_contains(expr.Col(0, "items"), "[{\"name\": \"Widget\"}]"). As with all fragments, these are SQL-only — the in-memory adapter cannot evaluate them.

Divergences

Two differences from Ecto are pure design re-expression, and need no follow-up:

Identity and on_replace are now supported:

Embed identity & on_replace. Plain cast_embed_* replaces the stored embed value wholesale (the simple case). For identity-based editing, embed.cast_embed_many_by_key(name:, existing:, blank:, params:, key_param:, child_key:, on_replace:, with:) matches each param to an existing embed by an id field — a match updates that embed in place (keeping its id), a non-match inserts. embed.put_new_id(params, "id") stamps a generated v4 UUID on new embeds (Ecto’s autogenerated binary_id); call it in the with cast. Embeds dropped from the params follow on_replace: EmbedDelete removes them (the inline default), EmbedRaise invalidates the parent. Ecto’s :update / :mark_as_invalid map onto building the matched embed from its existing record and onto EmbedRaise respectively. TASK:embed-identity

For the complete catalogue of differences from Elixir’s Ecto, see Divergences from Ecto.

✨ Search Document