# Neologo MVP: Unified Executable Symbolic Coordination Prototype

This project is a unified prototype implementing a deterministic coordination system. It demonstrates how to compile private intents into provisional shared realities by resolving the **Federated Threshold Stag Hunt** game.

It is a standalone implementation of the Neologo specification defined in the repository's core files:
1. **[neologo_revised_report_draft.md](file:///Users/kripar/Documents/coding/neologo.com/neologo_revised_report_draft.md)**: The technical report detailing the actor model, fold function semantics, transactional execution, outbox routing rule, and the threshold Stag Hunt game-theoretic correctness proof.
2. **[rant.md](file:///Users/kripar/Documents/coding/neologo.com/rant.md)**: The manifesto explaining the philosophical shift from centralized big-internet architectures to tiny, forkable "personal internets" focused on symbolic coordination.
3. **[index.html](file:///Users/kripar/Documents/coding/neologo.com/index.html)**: The single-page application codebase presenting the conceptual models, language semantics, and interactive game theory calculations.

This prototype connects these core concepts into a running kernel featuring:
- **BSON Event Log**: Event logs are serialized using binary BSON (a lightweight, type-preserving schema) and stored locally on disk (`data/` directory) for 100% database-free, local-first recovery.
- **CTM Attention Scheduler**: A native Elixir port of the Conscious Turing Machine (CTM) model, utilizing a probabilistic tournament tree to coordinate and schedule proposed transaction intents.
- **Neologo Transactional Runtime**: An implementation of the actor/event pipeline, executing actions transactionally, verifying safety invariants, and releasing outbox messages only upon block commit (the Outbox Rule).

---

## 🧭 Codebase Map

All code is inside `neologo-mvp/`:
- `lib/neologo/bson.ex` — BSON binary codec and the file-based append-only `Log` repository.
- `lib/neologo/ctm.ex` — **Experimental**: native Elixir CTM tournament tree (`Node` processes), leaf `Processor` processes, `Broadcast` manager, and discrete tick `Clock`. Drives the CLI walkthrough only; nothing in the transactional kernel depends on it, and its intensity/mood parameters are illustrative.
- `lib/neologo/runtime.ex` — The Neologo kernel. Reconstructs state from BSON logs, runs transactional commands (`apply_command/3` is the pure core), checks safety invariants, and dispatches outbox messages on commit.
- `lib/neologo/actors.ex` — Concurrent actor instances: one supervised GenServer per `(dir, actor_id)` with warm folded state, command serialization per actor, parallelism across actors, and single-writer BSON logs.
- `lib/neologo/game.ex` — The `ThresholdHunt` coordinator and `Agent` actor definitions, specifying event models, transitions, and outbox payouts.
- `lib/neologo/cli.ex` — Interactive walkthrough scenario runner and colorized visualizer.
- `lib/neologo/dsl.ex`, `lib/neologo/dsl/` — The `.neologo` DSL compiler: lexer, parser, static analyzer, evaluator, and boot-time schema registry. Compiles declarative machine definitions into `%Neologo.Machine.Schema{}` structs (Scale-Up Roadmap §3).
- `schemas/` — Declarative `.neologo` machine definitions (AssuranceContract, ThresholdHunt).
- `lib/neologo/demo.ex` — Many-core chaos load demo with a self-auditing exactly-once/conservation check.
- `test/` — Unit and integration tests validating BSON encoding, CTM tournaments, runtime invariants, game outcomes, DSL compilation, concurrency, and crash recovery (49 tests).

---

## 🚀 Getting Started

Ensure you have **Elixir ~> 1.19** installed.

### 1. Run the Automated Tests

Verify the correctness of all modules by running the test suite:

```bash
cd neologo-mvp
mix test
```

All tests (BSON, CTM tournaments, runtime execution, DSL compilation, and full integration) should pass successfully.

### The `.neologo` DSL

Games no longer need to be hand-written as Elixir modules. `Neologo.DSL` compiles declarative `.neologo` files into runnable schemas:

```elixir
schema = Neologo.DSL.compile_file!("schemas/assurance_contract.neologo")
Neologo.Runtime.execute("data", schema, "ac-1", %{"type" => "create", "target" => 100.0})
```

A machine declares its initial `state`, `command` handlers with `guard`/`emit`/`send` statements, `on <Event>` fold blocks, and `invariant` safety conditions:

```
machine AssuranceContract {
  state { status: "Funding", pledges: {}, target: 0, collected: 0.0 }

  command pledge(agent, amount) {
    guard state.status == "Funding" else :not_funding
    guard amount > 0.0 else :invalid_amount
    emit Pledged { agent: agent, amount: amount }
  }

  on Pledged {
    pledges = put(state.pledges, event.agent, get(state.pledges, event.agent, 0.0) + event.amount)
    collected = state.collected + event.amount
  }

  invariant "collected total does not match pledge sum" {
    abs(sum(values(state.pledges)) - state.collected) <= 0.0001
  }
}
```

Outbox routing (`send Agent(a) settle { ... }`) resolves actor names via `compile/2` options: `Neologo.DSL.compile_file!(path, actors: %{"Agent" => Neologo.Game.Agent})`. The full grammar is documented in `Neologo.DSL.Parser`; `schemas/threshold_hunt.neologo` ports the complete Stag Hunt — Ed25519 signature guards, conditional payoff settlement and all.

Every compile runs `Neologo.DSL.Analyzer` first. Unbound variables, unknown builtins, wrong arities, and duplicate command/fold definitions are compile errors; guards placed after emits and events that are emitted-but-never-folded (or vice versa) are warnings. `Neologo.DSL.analyze/1` exposes the same checks without building a schema.

At boot, `Neologo.DSL.Registry` (supervised by the application) compiles everything in `schemas/` and serves it by machine name — a broken schema fails the boot on purpose:

```elixir
schema = Neologo.DSL.Registry.get!("ThresholdHunt")
Neologo.Runtime.execute("data", schema, "hunt-99", command)
```

Adding a new coordination game is now a matter of dropping a `.neologo` file into `schemas/` — zero Elixir required. The directory is configurable via `config :neologo_mvp, :schema_dir`.

### Concurrent Actor Instances

`Neologo.Runtime.execute/4` is one-shot: it re-reads the log per call and offers no serialization, so concurrent callers on the same actor can race. `Neologo.Actors` runs each actor as its own supervised process instead:

```elixir
schema = Neologo.DSL.Registry.get!("AssuranceContract")
{:ok, state, events} = Neologo.Actors.execute("data", schema, "ac-1", command)
```

Same call shape and results as the runtime, but commands against one actor are serialized through its mailbox (concurrent duplicate joins resolve deterministically first-wins), distinct actors run in parallel across all cores, each log file has exactly one writer, state stays warm in memory instead of being re-folded from disk per command, and a killed process recovers by replaying its BSON log on next use. Outbox messages still dispatch only after commit, routed through the target's actor process.

The layer is deliberately node-local; cross-machine federation (Roadmap §1) can later swap the process registry for a distributed one without touching the runtime core.

### Reliability Guarantees

The actor layer closes the remaining rows of the teaser's failure table (index.html §10):

- **At-most-once execution** — commands may carry a `"command_id"`. Committed events are stamped with it; retries are rejected with `{:error, :duplicate_command}` without re-executing, even after an actor restart (the seen-set rebuilds from the log).
- **Atomic commit blocks** — an event block and its outbox entries are written in a single file write (`Log.append_many`, with outbox entries embedded as an `_OutboxPending` record). A crash cannot commit events while losing their settlements, or tear a multi-event block (Lemma 2).
- **Durable outbox, exactly-once effect** — the owning actor dispatches outbox entries asynchronously after commit and acks each delivery in `<actor_id>.outbox.bson`. On respawn, unacked entries are redelivered; each entry's command carries the entry id as its `command_id`, so duplicates are rejected by the target. Settlements are eventually delivered, exactly once in effect.
- **Deadlines** — the DSL has a `now()` builtin, both ThresholdHunt implementations reject `:deadline_passed` joins (deadline `0` = none), and `Actors.schedule_command/5` fires a deadline `close` on the actor's own process, giving liveness (Lemmas 6/7): every round terminates.

### Load Demo (many cores, one machine)

```bash
mix run -e "Neologo.Demo.run()"                  # 1,000 hunts, chaos on
mix run -e "Neologo.Demo.run(hunts: 5000)"       # scale it up
mix run -e "Neologo.Demo.run(chaos: false)"      # clean timing run
```

Each hunt is 6 actors (game + 5 agents with real Ed25519 keypairs); a random subset joins, above-threshold hunts close explicitly, the rest abort via scheduled deadline close. A chaos process SIGKILLs random actors every 50ms throughout. The final audit verifies every hunt reached exactly one terminal state, every agent settled exactly once with the teaser's exact payoff (6.1 / 2.1 / 2.0), and total payout matches prediction — conservation, under fire, across all cores.

### 2. Run the Visual Walkthrough Scenario

Run the step-by-step game scenario on your terminal:

```bash
cd neologo-mvp
mix run -e "Neologo.CLI.run()"
```

You will see a colorized printout showing:
1. The creation of the `ThresholdHunt` actor in `Pending` state.
2. The CTM clock advancing tick-by-tick. High-priority `Join` intents win the tournament, are broadcast down-tree, and are transactionally committed.
3. Multiple duplicate wins (representing network retries) being caught and rejected safely by the Neologo runtime.
4. The successful close of the round once 4/4 joins are committed, triggering outbox messages that settle payoffs for all participants.
5. Final actor balances reconstructed directly from BSON files on disk (joiners receive $6.1$, skippers receive $2.0$).

---

## ⚠️ Known Limitations

Stated plainly so nobody discovers them in production:

- **Durability is page-cache durability by default.** Appends use buffered writes; an OS crash (not just a BEAM crash) can lose the tail of a log. Set `config :neologo_mvp, :fsync, true` to fsync every append — slower, but power-loss safe. Crash recovery, redelivery, and exactly-once tests all assume the file survived.
- **Logs never compact.** Rebuild cost grows linearly with history; there are no snapshots yet. Fine for rounds with bounded lifetimes (the demo), wrong for long-lived actors.
- **No network API.** The runtime speaks Elixir function calls only. "Agents in different administrative domains" cannot reach it until an HTTP/gRPC surface or the federation layer exists.
- **Identity is a raw Ed25519 key.** No rotation, no revocation, no registry.
- **The DSL is untyped.** State access is stringly (`state.collected`); type/enum/fail declarations from the spec are not implemented.
- **Payoffs are runtime-internal numbers.** The runtime guarantees commitment bookkeeping, not external settlement — see report §7.4. The domain where this limitation vanishes is software-agent coordination, which is the intended near-term use.

## 💡 The Coordination Principle

In a simultaneous Stag Hunt game, playing `Stag` risks receiving a payoff of `0.0` if other participants defect. Neologo transforms this risk into a conditional commitment: **"I commit to play Stag if at least 4 others do, otherwise fallback to Hare."**

The mechanism itself is a **dominant assurance contract** (Tabarrok 1998, building on Bagnoli–Lipman 1989): the threshold-conditional refund makes joining safe, and the commit credit `r` upgrades weak to strict dominance. Neologo's contribution is not the mechanism but making it a compilable, replayable, invariant-checked schema rather than a bespoke platform — see report §7.3–§7.4 for lineage and the scope of the payoff assumption.

The experimental CTM acts as an attention scheduler proposing joins in the walkthrough, while the Neologo runtime acts as the **State Transition and Safety Validator**. The Outbox Rule guarantees that settlement messages only leave the hunt coordinator once the event block is committed to the BSON log, preventing divergence between actors.