j17 - Event sourcing for the rest of us

POST events to /{aggregate_type}/{aggregate_id}/{event_type}. That's the fundamental operation in j17. Everything else -- batching, implications, projections -- builds on this.

URL structure

POST /{aggregate_type}/{aggregate_id}/{event_type}

Component	Description	Example
`aggregate_type`	Defined in your spec	`order`, `user`, `task`
`aggregate_id`	Any j17 identifier — UUID v4/v5, humane code, singleton, or tagged UUID	`550e8400-e29b-41d4-a716-446655440000`
`event_type`	Event name from spec	`was_placed`, `had_email_updated`

Request body

{
  "data": {
    "items": [
      { "sku": "WIDGET-1", "quantity": 2, "price": 29.99 }
    ],
    "shipping_address": {
      "street": "123 Main St",
      "city": "Austin",
      "zip": "78701"
    }
  },
  "metadata": {
    "actor": {
      "type": "user",
      "id": "550e8400-e29b-41d4-a716-446655440001"
    },
    "previous_length": 0
  }
}

data (required)

The event payload. Validated against the JSON Schema in your spec.

{
  "data": {
    "email": "alice@example.com",
    "name": "Alice Smith"
  }
}

Validation failures return 422 with details:

{
  "ok": false,
  "error": "Event data failed schema validation",
  "path": "data.email"
}

metadata (required)

actor (required)

Who performed the action.

{
  "actor": {
    "type": "user",
    "id": "550e8400-e29b-41d4-a716-446655440001"
  }
}

The type must match an entry in your spec's agent_types. Common types: user, admin, webhook, integration.

The id must be a valid j17 identifier — a v4/v5 UUID, a humane code, a declared singleton, or a tagged UUID. See Identifiers for the full rule. Prior to 0.7.11 the actor id rule was UUID-only.

For named non-user actors (a webhook source, an external integration), declare a singleton and use it as the actor id:

{
  "actor": { "type": "webhook", "id": "twilio_webhook" }
}

This requires "singletons": ["twilio_webhook"] in your spec.

target (optional)

What was affected, if different from the aggregate.

{
  "target": {
    "type": "order",
    "id": "550e8400-e29b-41d4-a716-446655440002"
  }
}

The id follows the same rule as actor and aggregate keys — see Identifiers. Useful for cross-aggregate queries: "show me all events targeting this order."

previous_length (optional)

For optimistic concurrency control. The expected number of events in the aggregate before this write.

{
  "previous_length": 5
}

If the aggregate has a different length (concurrent write), j17 returns 409:

{
  "ok": false,
  "error": "Concurrent write detected. Stream has 6 events, expected 5."
}

Retry with the new length or handle the conflict.

skip_occ (optional)

For append-only patterns where conflicts don't matter. Requires allow_skip_occ: true in the event's spec definition.

{
  "skip_occ": true
}

timestamp (optional, non-production only)

Override the event's recorded timestamp. By default the worker stamps each event with the current Unix time in seconds; supplying metadata.timestamp replaces that stamp.

{
  "actor": { "type": "user", "id": "..." },
  "timestamp": 1700000000
}

Rules:

Non-production only — accepted in staging, test, and any other environment that is not prod. In prod the request is rejected with 422 and error: "metadata.timestamp is only accepted in non-production environments". This prevents accidental clock drift between staging and prod when test fixtures leak into production code paths.
Format — must be a non-negative integer (Unix seconds). Non-integers, negatives, floats, and other shapes return 422 with error: "metadata.timestamp must be a non-negative integer (Unix seconds)".
Scope — applies to single-event writes (POST /:type/:id/:event_type) and batch writes (POST /:type/:id). In a batch, the timestamp from the request's top-level metadata is applied to every event in the batch.
For per-event timestamps or backdated bulk loads, use the admin import endpoints instead (POST .../import, POST .../import_jsonl, POST .../cold_start).

This is the same field the engine has always recorded under metadata.timestamp on stored events — it was just not previously settable from the write endpoint.

Idempotency

Use the X-Idempotency-Key request header to prevent duplicate writes. This is a header, not a metadata field. Idempotency is supported on both single-event writes (POST /:type/:id/:event_type) and batch writes (POST /:type/:id).

curl -X POST https://myapp.j17.dev/order/abc123/was_placed \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Idempotency-Key: order-abc123-was_placed-20240115" \
  -d '{"data": {...}, "metadata": {"actor": {...}}}'

How it works

On success (2xx), j17 caches the response body, status code, a small set of headers (x-request-id, x-error-id), and a SHA-256 hash of your request body. The cache lasts 24 hours (rolling, per key).

Retry with same key + same body: Returns the cached response — same status code, same body, with X-Idempotency-Replayed: true set. No duplicate event is written and no handlers re-run.
Retry with same key + different body: Returns 422 with error: "Idempotency key reused with different request body". This catches accidental key reuse bugs.
Failed requests (4xx/5xx): Not cached. The same key can be retried freely until a 2xx response is received.

The body hash uses deterministic JSON encoding (keys sorted alphabetically before hashing), so reordering fields in your JSON object does not register as a different body. Equivalent JSON shapes (e.g. {"a":1,"b":2} vs. {"b":2,"a":1}) hash the same.

Scoping

Idempotency keys are scoped to instance + environment. The same key can be reused independently in prod, staging, and test without colliding. Keys from one instance never collide with keys from another instance.

Generating keys

Any string from 1 to 255 printable ASCII characters (0x20–0x7E). What matters is that keys are unique per distinct operation.

Good patterns: - {operation}-{entity_id}-{timestamp} -- order-placement-abc123-1705312800 - {user_id}-{action}-{nonce} -- user-789-checkout-a1b2c3d4 - UUID v4 -- 550e8400-e29b-41d4-a716-446655440000

The idempotency key represents the intent, not the data. If a user clicks "Submit Order" twice, both clicks should use the same key. But if they add an item and click again, that's a new operation with a new key.

Validation errors (422)

Condition	Error
Empty string	`Idempotency key cannot be empty`
Longer than 255 bytes	`Idempotency key exceeds 255 character limit`
Non-printable / non-ASCII characters	`Idempotency key contains invalid characters (must be printable ASCII)`
More than one `X-Idempotency-Key` header on a request	`Multiple X-Idempotency-Key headers not allowed`

Failure semantics

If the idempotency store is unreachable when checking a key (Redis error on read), the request fails open — it proceeds as if the key had not been seen, and the response is not cached for replay. This means a retry during an outage may write a duplicate event; clients that need strict at-most-once delivery should pair idempotency keys with OCC (previous_length).

When to use idempotency keys

Always use them for: - Payment processing - Order placement - Any operation with real-world side effects (sending emails, charging cards) - Batch operations - Any client that retries on network errors

Not needed for: - GET requests (inherently idempotent) - Operations already protected by your own deduplication logic

Response

Success returns 201 Created:

{
  "ok": true,
  "stream_id": "1234567890123-0",
  "implied_count": 2
}

Field	Description
`stream_id`	Redis stream ID of the written event
`implied_count`	Number of implied events triggered (omitted if 0)

Example: Complete flow

Creating an order:

# 1. Place the order
curl -X POST https://myapp.j17.dev/order/ord_123/was_placed \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -H "X-Idempotency-Key: order-ord_123-was_placed-20240115" \
  -d '{
    "data": {
      "customer_id": "550e8400-e29b-41d4-a716-446655440003",
      "items": [
        { "sku": "WIDGET-1", "name": "Widget", "price": 29.99, "quantity": 2 }
      ]
    },
    "metadata": {
      "actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440003" }
    }
  }'

# Response: {"ok": true, "stream_id": "...", "implied_count": 0}

# 2. Add an item (with OCC)
curl -X POST https://myapp.j17.dev/order/ord_123/had_item_added \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "sku": "GADGET-2",
      "name": "Gadget",
      "price": 19.99,
      "quantity": 1
    },
    "metadata": {
      "actor": { "type": "user", "id": "550e8400-e29b-41d4-a716-446655440003" },
      "previous_length": 1
    }
  }'

# Response: {"ok": true, "stream_id": "..."}

# 3. Mark as paid
curl -X POST https://myapp.j17.dev/order/ord_123/was_paid \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "data": {
      "payment_id": "pay_789",
      "amount": 79.97
    },
    "metadata": {
      "actor": { "type": "system", "id": "550e8400-e29b-41d4-a716-446655440004" },
      "previous_length": 2
    }
  }'

# Response: {"ok": true, "stream_id": "..."}

Aggregate IDs

Three formats accepted:

UUIDv4 (recommended)

Standard UUIDs:

550e8400-e29b-41d4-a716-446655440000

Use for most entities. Generate with any UUID library.

Humane codes

Human-readable identifiers:

ABC123XYZ

9 characters, Crockford base32. Good for: - Booking references - Promo codes - Support tickets - Anything customers type

global

Singleton aggregates:

curl https://myapp.j17.dev/config/global

One per aggregate type. Use for app-wide settings, feature flags.

Error handling

400 Bad Request

Validation error. Schema mismatch, missing actor, invalid aggregate ID.

404 Not Found

{
  "ok": false,
  "error": "Event type 'was_placed' not found in spec for aggregate 'order'"
}

Your spec doesn't define this event type. Check for typos or update your spec.

409 Conflict

OCC check failed. Retry with the current length.

413 Request Entity Too Large

Event data exceeds size limit.

422 Unprocessable Entity

Idempotency key reused with a different request body.

429 Rate Limited

You've exceeded your instance's burst rate limit or your plan's event allocation. Check X-RateLimit-Scope (instance or ip) and X-RateLimit-Tier. See rate limits for details.

Retry strategies

For 5xx errors or network failures:

async function writeEventWithRetry(path, data, options = {}) {
  const idempotencyKey = options.idempotencyKey || crypto.randomUUID();
  const maxRetries = options.maxRetries || 3;

  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await fetch(`https://myapp.j17.dev${path}`, {
        method: 'POST',
        headers: {
          'Authorization': `Bearer ${apiKey}`,
          'Content-Type': 'application/json',
          'X-Idempotency-Key': idempotencyKey,
        },
        body: JSON.stringify(data),
      });

      if (response.ok) return response.json();
      if (response.status === 422) throw new Error('Idempotency key mismatch');
      if (response.status < 500) throw new Error(`Client error: ${response.status}`);
      // 5xx: retry
    } catch (e) {
      if (attempt === maxRetries - 1) throw e;
      await sleep(Math.pow(2, attempt) * 100);
    }
  }
}

Always use idempotency keys when retrying to avoid duplicates.

Writing Events