Stores

Why sibling merge was a footgun

Older toolkit layers always baked Layer.provideMerge(layerDefaultMemory) inside the engine layer. Soft never saw an ambient AppStore, so SQLite AppStores stayed empty while two in-memory journals looked like a working “override” (shared EventJournal).

Now Soft unwrap peeks at ambient Storage at build time:

• No AppStore in context → bake layerDefaultMemory (R fulfilled).

• AppStore fed via Layer.provide / provideMerge into the toolkit layer → engines capture that store (memory + Logs, or SQLite).

Do not sibling-Layer.merge the toolkit layer with AppStore and expect override — Soft never sees Storage, engines stay on the default journal, and the AppStore file stays empty.

Do not Soft-override with a Node-logs-only Store.Service unless that store also registers the engines you run — Soft captures that bridge and toolkit layers die at build (Store.resolveOrDie) when the engine scope is missing. Live-only log bus: Logs.layer (no Storage). Durable journals: one AppStore with Node.logs + Daemon.store / WorkPool.store / ….

One store per Node

• One Store.Service per Node ManagedRuntime: many registrations, one journal/file, one Logs.layer.

• Multi-node demos use N stores / N runtimes — each node its own AppStore.layer*.

• Do not install a second Logs.layer or second Store.Service in the same runtime.

Logs vs layerDefaultMemory

Store.layerDefaultMemory (what soft-default / *Memory toolkit layers use) is engine observability only. It does not install Logs.Relay / durable _logs tails. Durable logs need Store.Service.layer* (bakes Logs.layer) or an explicit Logs.layer.

Node journal + per-service _logs copies of the same live line are intentional — see the logs guide.

Reading back

• Daemon / WorkPool execution rows: toolkit store handles (store.events(), …) or Store.resolveOrDie.

• App-facing queries after override: yield* AppStore / registration helpers.

WorkPool durability — DurableWorkPoolStore (SQL)

Pending + in-flight work that must survive a restart (at-least-once + dedup). Port: hyperlink-ts/DurableWorkPoolStore. SQLite backend: SQLiteDurableWorkPoolStore from hyperlink-ts/storage/sqlite.

Requires a payload / itemSchema on the WorkPool tag so entries can serialize. Compose the backend onto the WorkPool layer — the layer is the switch:

import * as WorkPool from "hyperlink-ts/WorkPool"
import { SQLiteDurableWorkPoolStore } from "hyperlink-ts/storage/sqlite"

const live = WorkPool.layer(Mail, { /* … */ }).pipe(
  Layer.provide(SQLiteDurableWorkPoolStore.layer({ filename: "queue.db" })),
)

This is a different plane from Soft WorkPool.store(Mail) on an AppStore (observability / analytics). You can run durability alone, Soft journals alone, or both.

History backfill — HistoryStore

Keyed append-log for stream history (metrics windows, etc.). Port: hyperlink-ts/HistoryStore (HistoryStore.layerMemory(), or SQLiteHistoryStore from hyperlink-ts/storage/sqlite). Engines read it via serviceOption(HistoryStore) — omit the layer and *History / windowed metrics.query backfill stay empty.

Fleet rate limit — Effect RateLimiterStore (Redis)

Gate / WorkPool / HttpApiClient rate limits take policy on the tag (rateLimit: { limit, window, … }). The store is a Context service: Soft in-memory limiter when absent; for a shared fleet limit provide Effect Redis:

import { RateLimiter } from "effect/unstable/persistence/RateLimiter"
import { NodeRedis } from "@effect/platform-node"

const FleetLive = Layer.mergeAll(EastGate.layer, WestGate.layer).pipe(
  Layer.provide(RateLimiter.layerStoreRedis({ prefix: "fleet:" })),
  Layer.provide(NodeRedis.layer({ host: "127.0.0.1", port: 6379 })),
)

See Gates for the full Redis recipe and live suites.