lode - v0.1.0

Overview

lode is a toolkit for data mapping and integrated query for Gleam — a port of Elixir’s Ecto. Like Ecto, it is split into 4 main components:

In summary:

Because Gleam has no macros, no runtime reflection, and no behaviours, this library is a functional re-expression of Ecto rather than a literal translation: repositories are plain values you pass around, schemas are explicit Schema(row) values that carry their own codecs, query expressions are first-class data, and every fallible operation returns a Result instead of raising (there are no ! variants). These re-expressions are called out inline below; the full rationale lives in DESIGN.md in the repository.

In the following sections, we will provide an overview of those components and how they interact with each other. If you want to see a sample application built step by step, please check the Getting Started guide.

Mirroring Elixir, the project ships as two packages: lode (the database-agnostic core) and lode_sql (the SQL renderer, PostgreSQL adapter, and migrations). Modules in both import as lode/....

Repositories

lode/repo is the heart of the library. In Elixir, a repository is a module you define with use Ecto.Repo, configured through your application environment and started in a supervision tree. lode has none of that machinery: a repository is a value — a lode/repo.Repo wrapping an adapter — that you construct once and pass to every repo function. This is a design re-expression (no use macro, no process-dictionary repo, no OTP app config; see DESIGN.md in the repository).

First add the packages to your gleam.toml:

[dependencies]
lode = { path = "../lode/lode" }
lode_sql = { path = "../lode/lode_sql" }
pog = ">= 4.1.0 and < 5.0.0"

(The packages are not on Hex yet — the names are taken by Elixir’s Ecto and a rename is pending — so they are path dependencies for now.)

Where Elixir’s Ecto reads credentials from config/config.exs, here the database connection is an ordinary pog pool that your application configures and starts — typically under your supervision tree, exactly where Elixir puts the repo process:

import lode/adapters/postgres
import lode/repo
import gleam/erlang/process
import gleam/option.{Some}
import pog

pub fn connect() -> pog.Connection {
  let config =
    pog.default_config(process.new_name("my_app_db"))
    |> pog.host("localhost")
    |> pog.database("lode_simple")
    |> pog.user("postgres")
    |> pog.password(Some("postgres"))
    |> pog.rows_as_map(True)
  let assert Ok(started) = pog.start(config)
  started.data
}

pub fn main() {
  let r = repo.new(postgres.new(connect()))
  // pass `r` to every repo call
}

For tests, swap the adapter and nothing else changes:

import lode/adapters/memory

let r = repo.new(memory.new())

There’s also a SQLite adapter (lode/adapters/sqlite, on sqlight) — the same one-line swap, repo.new(sqlite.new(conn)). SQLite is embedded, so it needs no server (sqlight.open(":memory:")), which makes it a good middle ground for tests that want real SQL semantics without Postgres. The query/DDL renderer is dialect-parameterized, so everything above the adapter — queries, changesets, migrations — is identical across Postgres and SQLite.

sqlite.new also fixes SQLite’s legacy per-connection defaults: foreign-key enforcement on, a 5s busy timeout, and WAL journaling with synchronous = NORMAL (on file databases). Migrations create STRICT tables, so a wrong-typed insert errors instead of being silently coerced — the safe-modern SQLite setup, out of the box.

Because the repo is just a value, “which database am I talking to?” is always explicit at the call site — there is no global, implicit repository.

Schema

Schemas map external data into Gleam records. Gleam has no reflection and no defstruct, so a schema is two explicit pieces: a plain record type you define, and a Schema(row) value describing how that record maps to a table — its fields, its primary key, and load/dump codecs between the record and a column-keyed map of database Values.

Here is the classic weather example:

import lode/schema.{type Schema}
import lode/types/primitive
import lode/value.{VFloat, VInt, VString}
import gleam/dict
import gleam/result

pub type Weather {
  Weather(id: Int, city: String, temp_lo: Int, temp_hi: Int, prcp: Float)
}

pub fn weather_schema() -> Schema(Weather) {
  schema.new(
    source: "weather",
    fields: [
      schema.primary_key("id", primitive.id()),
      schema.field("city", primitive.string()),
      schema.field("temp_lo", primitive.integer()),
      schema.field("temp_hi", primitive.integer()),
      schema.field("prcp", primitive.float_type()),
    ],
    load: fn(row) {
      use id <- result.try(schema.require_int(row, "id"))
      use city <- result.try(schema.require_string(row, "city"))
      use temp_lo <- result.try(schema.require_int(row, "temp_lo"))
      use temp_hi <- result.try(schema.require_int(row, "temp_hi"))
      use prcp <- result.try(schema.load_field(row, "prcp", primitive.float_type()))
      Ok(Weather(id: id, city: city, temp_lo: temp_lo, temp_hi: temp_hi, prcp: prcp))
    },
    dump: fn(w: Weather) {
      dict.from_list([
        #("id", VInt(w.id)),
        #("city", VString(w.city)),
        #("temp_lo", VInt(w.temp_lo)),
        #("temp_hi", VInt(w.temp_hi)),
        #("prcp", VFloat(w.prcp)),
      ])
    },
  )
}

Where Elixir’s schema "weather" do ... end block injects an :id primary key and a struct for you, here the schema.primary_key("id", primitive.id()) line declares it explicitly: it is autogenerated, so it is omitted from insert payloads and the database assigns it. Ecto’s per-field default: option has no equivalent — a Gleam record always carries a value, so whatever record you construct is the default (prcp: 0.0 above plays that role).

If writing the codecs by hand looks tedious: it is meant to be generated. The lode/codegen module (in lode_sql) emits record types, schemas, and typed field accessors from a declarative schema spec — see §14 of DESIGN.md in the repository and the examples/codegen_demo project.

With the schema defined, we can interact with the repository:

import lode/changeset

let r = repo.new(postgres.new(connect()))
let s = weather_schema()

// Insert — the database assigns the id, and the stored row is loaded back.
let weather = Weather(id: 0, city: "Kraków", temp_lo: 0, temp_hi: 23, prcp: 0.0)
let assert Ok(stored) = repo.insert(repo: r, schema: s, changeset: changeset.change(weather, s))

// Fetch by primary key. `Option` replaces Ecto's `nil`:
// `Ok(Some(row))` if found, `Ok(None)` if not.
let assert Ok(Some(found)) = repo.get(repo: r, schema: s, id: VInt(stored.id))

// Delete by primary key.
let assert Ok(Nil) = repo.delete(repo: r, schema: s, row: found)

Notice every operation returns a Result — repo.insert either gives you the stored record or a typed lode/error.LodeError; there are no exceptions and no insert!-style bang variants (a design re-expression; see DESIGN.md).

By defining a schema, the rest of the library knows how to read and write your data, while your application keeps working with ordinary records — the data representation stays decoupled from its storage.

Changesets

Although we can insert a record directly via changeset.change, most of the time we use changesets to filter, cast, and validate external data before writing it. Changesets let you track and check changes as a unit:

import lode/changeset.{type Changeset}
import lode/value.{type Value}
import gleam/dict.{type Dict}
import gleam/regexp

pub type User {
  User(id: Int, name: String, email: String, age: Int)
}

fn email_regexp() -> regexp.Regexp {
  let assert Ok(re) = regexp.from_string("@")
  re
}

pub fn user_changeset(
  user: User,
  params: Dict(String, Value),
) -> Changeset(User) {
  changeset.cast(
    data: user,
    schema: user_schema(),
    params: params,
    permitted: field.fields(["name", "email", "age"]),
  )
  |> changeset.validate_required(field.fields(["name", "email"]))
  |> changeset.validate_format(field.field("email"), with: email_regexp())
  |> changeset.validate_number(field.field("age"), checks: [
    changeset.GreaterThanOrEqual(18.0),
    changeset.LessThanOrEqual(100.0),
  ])
}

cast takes the original data, the schema, the external parameters (a map of field names to Values — say, decoded from a form or JSON payload), and the list of permitted fields: anything not permitted is simply ignored, which is the same safety property Ecto’s cast/4 gives you. The pipeline then layers validations on top. Where Ecto writes validate_inclusion(:age, 18..100), Gleam has no range type, so the numeric-bounds re-expression is validate_number with explicit checks (use validate_inclusion for a finite list of allowed Values).

Since changeset.cast records the changes and whether they are valid, the repository knows what to do with the result:

import lode/error

case repo.update(repo: r, schema: user_schema(), changeset: cs) {
  // user updated; the returned record reflects the changes
  Ok(user) -> "updated " <> user.name
  // validation failed: `errors` is a typed list of per-field errors
  Error(error.ChangesetInvalid(errors: _errors)) -> "validation failed"
  // the database itself objected
  Error(_) -> "database error"
}

The error side is a typed LodeError, so the “did validation fail or did the database object?” distinction is a pattern match, not string parsing. Changesets can also convert database constraint violations (unique indexes, foreign keys, checks) into changeset errors via changeset.unique_constraint and friends — see Constraints and Upserts.

Query

The last component is the query layer, used to retrieve information from a repository. In Elixir, queries are written with the from macro and ^ interpolation. Gleam has no macros, so a query is an ordinary value built with pipe-friendly functions, and conditions are typed expression values from lode/query/expr — plain Gleam variables slot in directly, no interpolation operator needed:

import lode/query
import lode/query/expr.{FieldRef}
import lode/types/primitive

// Typed field accessors — hand-written here, normally generated by codegen.
// The type parameter makes comparisons compile-checked:
// expr.gt(field: user_age(), to: "old") will not compile.
fn user_age() -> expr.FieldRef(User, Int) {
  FieldRef(binding: 0, name: "age", lode_type: primitive.integer())
}

fn user_email() -> expr.FieldRef(User, String) {
  FieldRef(binding: 0, name: "email", lode_type: primitive.string())
}

let adults =
  query.from(source: "users", alias: "u")
  |> query.where(expr.or_(
    left: expr.gt(field: user_age(), to: 18),
    right: expr.is_nil(user_email()),
  ))

let assert Ok(users) = repo.all(repo: r, schema: user_schema(), query: adults)

Queries are rendered to parameterized SQL (age > $1), so they are protected against SQL injection by construction. And because a Query is just a value, composing or branching on conditions is ordinary Gleam — Ecto’s dynamic/2 is unnecessary here (see Dynamic Queries).

You can also query a table directly, without a full schema load, by selecting specific columns; rows then come back keyed by column name. See Schemaless Queries:

import lode/query/sql

let names =
  query.from(source: "users", alias: "u")
  |> query.where(expr.gt(field: user_age(), to: 18))
  |> query.select([expr.col(user_email()), expr.col(user_age())])

let #(text, params) = sql.to_sql(names)
let assert Ok(rows) = repo.query_raw(repo: r, sql: text, params: params)
// each row is a Dict(String, Value); read columns with schema.load_field

The query builder supports distinct, where, or_where, having, join, select, select_merge, order_by, group_by, limit, offset, and preload. With the exception of or_where and select_merge (which Ecto spells differently), these mirror the keywords in Ecto’s from.

query.lock(q, "FOR UPDATE") adds a row lock (Ecto’s lock:), rendered verbatim at the end of the SELECT; run it inside a repo.transaction.

To retrieve results, pass the query to a repo function: repo.all returns every match, repo.one returns Ok(Some(row))/Ok(None) and errors with MultipleResults if more than one row matches, and repo.get/repo.get_by cover the fetch-by-key cases. There are no one!/all! variants — the Result is the API.

Other topics

Associations

lode supports defining associations on schemas. Since there is no reflection, an association is declared by name on the schema value, and you supply the three things Ecto would otherwise derive: how to read the join keys on each side, and how to attach the loaded children to the parent record:

import lode/association
import lode/value.{VInt}

pub type Post {
  Post(id: Int, title: String, comments: List(Comment))
}

pub type Comment {
  Comment(id: Int, post_id: Int, body: String)
}

fn post_schema() -> Schema(Post) {
  post_base()  // the schema.new(...) for posts' stored fields
  |> association.has_many(
    name: "comments",
    related: comment_schema,
    foreign_key: "post_id",
    owner_key: fn(p: Post) { VInt(p.id) },
    child_key: fn(c: Comment) { VInt(c.post_id) },
    put: fn(p, cs) { Post(..p, comments: cs) },
    opts: association.options(),
  )
}

belongs_to, has_one, has_many_through, and many_to_many follow the same pattern.

Like Ecto, lode does not provide lazy loading: associations are loaded explicitly, each in one batched query (no N+1). Where Ecto marks an unloaded association with Ecto.Association.NotLoaded, here the field is just part of your record — it holds whatever you put there (an empty list, say) until you preload; model it as Option(List(Comment)) if you need to distinguish “not loaded” from “none”.

import lode/preload

// preload onto already-loaded rows (Repo.preload):
let assert Ok(posts) =
  repo.preload(repo: r, schema: post_schema(), parents: posts, preloads: [
    preload.one("comments"),
  ])

// or attach the preload to the query itself:
let q =
  query.from(source: "posts", alias: "p")
  |> query.preload([preload.nest("comments", [preload.one("author")])])

Ecto’s Ecto.build_assoc/3 and Ecto.assoc/2 helpers dissolve into ordinary Gleam — with explicit records and first-class queries there is nothing to derive:

// Ecto.build_assoc(post, :comments, text: "cool") — construct the record
// with the foreign key set:
let comment = Comment(id: 0, post_id: post.id, body: "cool")

// Ecto.assoc(post, :comments) — a query on the foreign key, which composes
// like any other (comment_post_id() is a FieldRef accessor, as before):
let comments_q =
  query.from(source: "comments", alias: "c")
  |> query.where(expr.eq(field: comment_post_id(), to: post.id))
let assert Ok(comments) =
  repo.all(repo: r, schema: comment_schema(), query: comments_q)

See Associations for the full tour, including writing associations with changeset.put_assoc/cast_assoc.

Embeds

lode also supports embeds. While associations keep parent and child entries in different tables, embeds store the child along side the parent: lode/embed provides embeds_one and embeds_many field types, stored as jsonb in PostgreSQL, with cast_embed_one/cast_embed_many mirroring cast_assoc. See Embedded Schemas.

Mix tasks and generators

Elixir’s Ecto ships mix tasks (mix ecto.create, mix ecto.migrate, code generators). Gleam has no mix, so the tooling re-expresses as functions you wire into a gleam run entrypoint. The migration runner in lode_sql ships a small argv dispatcher:

import argv
import lode/migrator

pub fn main() {
  let r = repo.new(postgres.new(connect()))
  let assert Ok(_) = migrator.run(r, my_migrations(), argv.load().arguments)
}

gleam run -m migrate migrate      # apply pending migrations
gleam run -m migrate rollback 2   # roll back the most recent two
gleam run -m migrate status       # applied/pending report

Each migration runs in its own transaction and is tracked in a schema_migrations table, just like mix ecto.migrate.

Storage create/drop. storage.create(repo, database:) / storage.drop(repo, database:) create and drop the database (Ecto’s storage_up/storage_down), run against a Repo connected to another database (e.g. postgres). There’s no mix-style CLI — call them from a gleam run entrypoint, or use createdb/psql. TASK:storage-up-down

In place of Ecto’s generators, lode’s codegen is schema-spec driven: author a pure-data spec of your tables once, and codegen.generate_from_spec emits the records, schemas, typed field accessors, and preload constructors — offline, no database needed — while lode/drift verifies a live database still agrees with the spec. The examples/codegen_demo project in the repository shows the whole workflow, and the Getting Started guide walks through it step by step.

✨ Search Document