# Pico Engine — full text bundle --- # Pico overview # Picos: what they are, what they enable, and what they are for **Status:** draft (2026-09-24) **Audience:** anyone arriving at this repo — contributors, students, integrators — who needs the model in their head before reading the other design docs. **Sources:** the design docs in this folder and `MEMORY.md`; the `manifold-api`, `sensor-network`, and `MCPforEXP` repos; the Picolabs wiki; and the picos-tagged posts on windley.com (2013–2026). Links at the end. --- ## 1. What picos are A **pico** (persistent compute object) is a small, general-purpose, always-online computer that stands in for one entity: a person, a place, a device, an organization, a group, a concept, even a pothole. It has a unique identity, keeps its own state, runs its own programs, and talks to other picos by exchanging events and answering queries. Networks of picos model the world — one addressable actor per thing we care about. Picos are **actors** in the [actor-model](https://en.wikipedia.org/wiki/Actor_model) sense. In response to a message a pico may send messages to other picos, create other picos, and change its internal state, which affects how it handles the next message. Picos share nothing except messages; one pico cannot see or touch another's state. Three kinds of persistence distinguish picos from most other programming models: | Property | Meaning | |----------|---------| | **Persistent identity** | A pico exists with one stable identity from creation until it is explicitly deleted. Since 1.6 that identity is a portable `did:webvh` that survives moving between engines. | | **Persistent state** | A pico's state lives inside it, isolated from every other pico, and is available to any rule that runs there. No external database. | | **Persistent availability** | Once created, a pico is always on and ready for events and queries. Restarting the engine changes nothing about what picos exist or know. | ### Anatomy of a pico | Part | What it is | |------|------------| | **Rulesets and event bus** | Behavior is installed as KRL rulesets, loaded by URL and compiled to JavaScript. A rule is event-condition-action: `select when` an event expression matches, evaluate a condition, act, then a postlude that updates state and raises further events. Every incoming event is raised on the pico's internal bus; a salience graph routes it to listening rules, so installing a ruleset subscribes its rules automatically. | | **Event-query API** | Not RESTful. Events change state; queries call shared functions. It is a *meta-API*: a pico's concrete API is whatever its installed rulesets provide, and it can change over the pico's life. Sky is the current [Event Exchange Protocol](/exp/) version (`/sky/event`, `/sky/query`). | | **State: entity variables and PDS** | `ent:` variables are scoped to each ruleset (a closure over its data). The **Pico Data Store** (default since 1.4) is the shared data plane on every pico — profile, namespaced general data, per-ruleset settings — written by `pds` events, read by shared functions. | | **Channels (ECIs)** | As many channels as a pico needs, each with an Event Channel Identifier and its own event/query **policy**. Channels are non-correlatable addresses and the local authorization gate. A `discovery` channel on every pico lets integrators learn what it can do. | | **Relationships** | Pairwise, bidirectional, policy-gated connections between picos (*subscriptions* before 1.6.2). They form the heterarchical graph applications run on, independent of the parent/child tree and of which engine hosts each pico. | | **Parent/child tree + Wrangler** | Every pico except a root has a parent, which installs and initializes it; control of a root implies control of its subtree. **Wrangler**, the pico OS on every pico, exposes lifecycle, channels, and relationships as KRL functions and events. | | **Identity** | Three planes: a human unlocks their **root pico** with a passkey (the root is the person's actor on the mesh); picos identify themselves to each other with **DIDs** (`did:webvh` per pico, `did:peer` per relationship, DIDComm across meshes); outside software reaches a mesh with **OAuth** tokens on `/sky/*`. | ### Where picos run Picos live in a **pico engine**, an open-source Node.js runtime built on `pico-framework` (lifecycle, channels, LevelDB persistence) and `select-when` (event-expression matching) with a KRL compiler on top. One engine hosts many picos and, since 1.5, many independent **meshes** (a root pico and its descendants) belonging to different owners. Picos on different engines form relationships as easily as picos on the same engine; the engines need no arrangement between them. The engine runs on a Raspberry Pi and on Fargate. The stack above the engine is strictly layered, and the layering is a design rule: engine and wrangler stay domain-agnostic; Manifold and community rulesets supply the vocabulary (owner, thing, community, borrower). ``` engine ← primitives: picos, channels, policy, persistence, DID keys, passkeys, OAuth wrangler ← the OS: children, channels, relationships, discovery manifold ← a framework for pico meshes: owner, things, communities, registries, notifications mesh apps ← sensor-network, SafeAndMine, Fuse-style apps ``` ### Lineage Picos grew out of the Kynetx rules engine (2008) and its "personal cloud" era: CloudOS, SquareTag, Fuse (a production connected-car platform, 2013–2015, hundreds of customers and over a thousand devices), and a guard-tour system. The Node rewrite (2017) made the engine small and fast; 1.0 (2021) made it modular; 1.4 added PDS by default; 1.5 (July 2026) brought passkeys and OAuth into the engine; 1.6 (August 2026) shipped DID-based identity and relationships. The motivation has not changed: an Internet of Things that is decentralized, heterarchical, and interoperable — an alternative to the "CompuServe of Things," where people rent connected devices from vendors in the middle. --- ## 2. What pico features enable The value is the combination: every entity gets a **closure over identity, data, and behavior**. | Feature | What it enables | | -------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Persistent identity** (`did:webvh`, channels) | Relationships that outlive restarts and moves; reputation and history; credentials with a durable subject; revocable delegation; a lifecycle that follows the thing across owners. Why SPIFFE was set aside: it names replaceable processes; picos are actors that last. | | **Persistent state and availability** (entity vars, PDS) | Databaseless programming; the running system *is* the design, visible live in the developer UI. Data control at the ruleset grain: delete the trip service before transferring a vehicle, keep the maintenance records. Twins keep working while the owner is offline. | | **Rules on an event bus** | Each rule is a microservice for one entity. Functionality accretes — install a ruleset and its rules start listening. Correlation ids, scatter-gather, guard rules, scheduled retries, and routed error events make failure-tolerant asynchrony the default. | | **Event-query meta-API + discovery** | A pico's API is discoverable at runtime. Home Assistant turns discovery bindings into devices and services; the MCP interface turns them into LLM tools, gated to the rulesets actually installed. Details: [Event Exchange Protocol](/exp/). | | **Channels with policy** | Capability-style security: an ECI is permission to do what its policy allows and nothing more. Separate channels prevent correlation, scope webhooks to one endpoint, and let discovery filter bindings per caller. | | **Typed, pairwise relationships** | Heterarchical graphs across the tree and across engines. Rights are *relationships*, not attributes: a fleet can have two owners, a truck can be lent with attenuated rights, a community can accept members from other meshes. The **introduction pattern** — an existing relationship brokers a new one — generalizes to delegation, transfer, and ReBAC. | | **Tree + Wrangler** | Programmatic lifecycle: picos create, initialize, and delete picos; prototypes stamp out configured children. Admin rights follow the tree; everything else is conferred by policy. | | **Actor concurrency** | Lock-free, isolated, location-transparent. Applications shard naturally — one pico per section, sensor, vehicle — and scale by adding engines. A laptop replayed 44,504 registration events into 8,000+ section picos at 21 events/sec. | | **Three-plane identity** | Self-sovereign meshes: no central identity provider, no password database; strangers on different engines trust each other by resolving DIDs, with no federation. Integrators get scoped, revocable OAuth access without becoming the pico. | | **Substitutable, small hosting** | A Pi, a community server, or a cloud, with picos movable between them. Compute moves toward the data, and a vendor's cloud shutting down does not brick the thing. | --- ## 3. Five use cases | Use case | Picos | Relationships | Status | |----------|-------|---------------|--------| | LoRaWAN sensor network | owner, Manifold hub, sensor communities, sensor things | community ↔ thing; hub ↔ community; Home Assistant via OAuth | Running at the cabin | | Connected vehicles, transferable twins | owner, fleet, vehicle | FleetOwner per owner; vehicle ↔ mechanic, insurer, borrower | Fuse shipped 2014; DID-era transfer designed | | Personal inventory and lost-and-found | owner, things, communities, tag and skills registries | thing ↔ community; tag → thing via registry | Manifold on 1.5+; tag-scan path in progress | | AI that works for the owner | root or household pico and children as memory and event sources | pico ↔ model via webhook or callback; later a local model | Webhook pattern now; `llm:*` designed | | High-fan-out institutional systems | institution, index picos, tours, locations, reports; or one pico per course section | personal cloud ↔ institution; collection → members | Guard tour delivered 2014; registration proof 2017 | ### 3.1 A LoRaWAN sensor network Battery-powered Dragino sensors in a pump house, a crawl space, a deck, and a beehive report over the Helium network. Each sensor is a **thing pico** in a **sensor community pico** under the owner's Manifold hub. Helium posts each heartbeat to a channel on the sensor's pico; a router ruleset decodes the payload and raises `sensor:new_readings`; a thresholds ruleset raises `sensor:threshold_violation`; the community raises `manifold:add_notification`, and Manifold fans the alert out to whatever the owner enabled (inbox, SMS, Prowl, Home Assistant). Home Assistant connects over OAuth and renders the mesh as devices — history, dashboards, automations — with no custom UI. Picos contribute one actor per sensor with its own decoding, thresholds, and history; communities that own bootstrap and aggregation; a notification platform no domain ruleset reimplements; and a way to admit a machine (the Helium webhook) into an OAuth-locked mesh by exempting one narrow channel. Adding a sensor type is one router ruleset. ### 3.2 Connected vehicles and transferable digital twins Fuse gave every vehicle a pico fed by an OBD-II device — trips, fuel, maintenance, diagnostic codes, alerts, each a set of rules in the vehicle pico — with an owner pico and a fleet pico above. Ownership was a **relationship** (`FleetOwner`), so two owners were two relationships, and a new owner was added through the introduction pattern. The design docs carry this forward: a vehicle's relationships with manufacturer, mechanic, insurer, borrower, and DMV are one mechanism with different roles and Rx policies. When Phil sells his truck to Dave on another engine, the truck's `did:webvh` stays the same, parent and community links move to Dave's mesh, and the battery-health service relationship survives via DID rotation. Trip data can be deleted before transfer; maintenance records stay with the vehicle. The twin's identity is bound to the vehicle, not to any owner. ### 3.3 Personal inventory, tags, and lost-and-found SquareTag, and now Manifold with SafeAndMine, gives a person a mesh of **thing picos** for the objects they care about — a backpack, a bike, a furnace — grouped into communities. Each thing carries skills: SafeAndMine registers QR/NFC tags in the tag registry and shows owner contact information to a finder who scans one; Journal records notes such as filter changes. Things are created, tagged, and annotated through Home Assistant or through a conversational interface: an MCP server exposes Manifold operations as tools, and skill-gating shows the model only the tools whose rulesets are installed on the pico at hand, offering to install what is missing. Every thing has its own API, skills are installed per thing at runtime, a thing can belong to several communities, and the data stays in the owner's mesh rather than a vendor's account. ### 3.4 AI that works for the owner Picos already have what an AI integration needs: persistent identity, owned state, and an event-driven interface. The progression starts shallow — a pico fires an event, an external routine reasons and posts a response event back — then moves to KRL actions that call a model with a callback event, then to the model using pico queries as tools so that the **pico is the memory**, and finally to the model raising events into the pico graph to make things happen. A household pico gives a model a cross-domain view over its children (car, sensors, calendar) while the hierarchy encodes the privacy boundary. The local-model design adds `llm:complete`, `llm:extract`, `llm:classify`, and `llm:summarize` as KRL primitives bounded by channel policy, with the rule that the model *proposes* and KRL *commits*. Fuse's bare alerts are the motivating example: a diagnostic code enriched with context and a recommendation is what drivers needed. ### 3.5 High-fan-out institutional systems The **guard tour** system modeled an institution, tours, locations, and reports as separate picos, with index picos mapping identifiers to channels; guards and managers used their own personal clouds with a guard-tour ruleset installed — no accounts in the tour system — plus temporary channels for the duration of a tour. **BYU priority registration** modeled each of 5,393 course sections as a pico keeping its own roster; replaying two days of add/drop requests showed that an actor per entity absorbs a surge without contention. The pattern: many entities with independent state and bursty activity get one pico each, plus collection picos for grouping, and people participate through their own picos rather than an account the institution owns. --- ## Open work Moving picos between engines ([pico-move.md](./pico-move.md)), pico-level schemas ([pico-schemas.md](./pico-schemas.md)), the `peer` relationship type and ReBAC in community rulesets ([pico-relationships.md](./pico-relationships.md)), delegated admin ([delegated-admin-relationship.md](./delegated-admin-relationship.md)), local LLM primitives ([pico-llm-capabilities.md](./pico-llm-capabilities.md)), and the channel-policy ceiling and pico mirrors (`MEMORY.md` roadmap). ## Sources **Actor model:** [Actor model](https://en.wikipedia.org/wiki/Actor_model) — general overview; Hewitt, Bishop and Steiger, *A Universal Modular ACTOR Formalism for Artificial Intelligence* (IJCAI 1973) — the origin; [Agha, *Actors: A Model of Concurrent Computation in Distributed Systems*](https://dspace.mit.edu/handle/1721.1/6952) (MIT AITR-844, 1985) — the canonical treatment. **Repos and docs:** this folder and `MEMORY.md`; [manifold-api](https://github.com/Picolab/manifold-api) (`README.md`, `docs/Manifold.md`, `docs/Managing_PDS.md`); [sensor-network](https://github.com/windley/sensor-network); [MCPforEXP](https://github.com/picolab/MCPforEXP). **Docs in this repo** — start with the [Programming lessons](/lessons/) (Quickstart through Relationships). Engine and Wrangler: [Operator guide](/pico-engine) · [Wrangler](/wrangler/) · [Channels](/wrangler/channels) · [Event Exchange Protocol](/exp/). KRL: [KRL Manual](/krl/) · [Developing with Picos](/developing/) (workflow, tips, misconceptions). Identity and relationships: [Relationships](/wrangler/relationships) · design notes in this folder. Platform: [Manifold](https://github.com/Picolab/manifold-api) (sibling repo). Older Confluence pages that have not been moved yet remain at the [Pico Labs wiki](https://picolabs.atlassian.net/wiki/spaces/docs/overview). **Blog (windley.com, tag: picos):** [Fundamental Features of Persistent Compute Objects](https://www.windley.com/archives/2013/10/fundamental_features_of_persistent_compute_objects.shtml) · [Pico APIs: Events and Queries](https://www.windley.com/archives/2013/12/pico_apis_events_and_queries.shtml) · [Complex Pico Structures: Guard Tours](https://www.windley.com/archives/2014/01/complex_pico_structures_guard_tours.shtml) · [Fuse as a Microservice Architecture](https://www.windley.com/archives/2014/07/fuse_as_a_microservice_architecture.shtml) · [Fuse with Two Owners](https://www.windley.com/archives/2014/10/fuse_with_two_owners.shtml) · [Reactive Programming with Picos](https://www.windley.com/archives/2015/11/reactive_programming_with_picos.shtml) · [Reactive Programming Patterns: Examples from Fuse](https://www.windley.com/archives/2016/04/reactive_programming_patterns_examples_from_fuse.shtml) · [Using Picos for BYU's Priority Registration](https://www.windley.com/archives/2017/01/using_picos_for_byus_priority_registration.shtml) · [Announcing Pico Engine 1.0](https://www.windley.com/archives/2021/02/announcing_pico_engine_10.shtml) · [Persistence, Programming, and Picos](https://www.windley.com/archives/2021/02/persistence_programming_and_picos.shtml) · [Ten Reasons to Use Picos](https://www.windley.com/archives/2021/07/ten_reasons_to_use_picos_for_your_next_decentralized_programming_project.shtml) · [Alternatives to the CompuServe of Things](https://www.windley.com/archives/2021/07/alternatives_to_the_compuserve_of_things.shtml) · [Monitoring Temperatures in a Remote Pump House](https://www.windley.com/archives/2023/03/monitoring_temperatures_in_a_remote_pump_house_using_lorawan.shtml) · [Building a Conversational Interface for Manifold](https://www.windley.com/archives/2026/04/building_a_conversational_interface_for_manifold_with_mcp_and_picos.shtml) · [AI Integration in Picos Starts with Events](https://www.windley.com/archives/2026/06/ai_integration_in_picos_starts_with_events.shtml) · [Manifold API and Sensor Network](https://www.windley.com/archives/2026/06/manifold_api_and_sensor_network_two_new_repos.shtml) · [Identity for the Pico Engine](https://www.windley.com/archives/2026/07/identity_for_the_pico_engine.shtml) · [Why Not SPIFFE for Pico Identity?](https://www.windley.com/archives/2026/08/why_not_spiffe_for_pico_identity.shtml) · [Moving My Sensor Network onto Manifold](https://www.windley.com/archives/2026/09/moving_my_sensor_network_onto_manifold.shtml). --- # Developing with Picos # Developing with Picos Practical notes for people writing KRL and running picos on the pico engine: how a ruleset gets from your editor into a running pico, what the engine does when an event arrives, and the mistakes that cost the most time. If you're new, start with the [Programming lessons](/lessons/), which build a working system step by step. Come back here when you want the "why did that happen?" and "how do I set things up?" answers. ## In this section 1. [Programming Workflow](/developing/workflow): develop, build, deploy, test, and release a ruleset. 2. [Event Loop Introduction](/developing/event-loop): the five phases a pico goes through for every event, and why rule order can surprise you. 3. [Tips](/developing/tips): engine setup, hosting and flushing rulesets, `krl-check`, and short technique tips (data, HTTP, picos working together). 4. [Event and Query URLs with Postman](/developing/postman): building Sky event and query requests in Postman (or `curl`). 5. [KRL misconceptions and common errors](/developing/misconceptions): where habits from other languages mislead, and the typos that are hard to see. 6. [Webhook signatures](/developing/webhook-signatures): check a Square-style signature over the raw body. 7. [AI tools over MCP](/developing/mcp-ai-tools): connect Claude, or another remote MCP client, to a channel or a mesh. ## Elsewhere in the docs - [Programming lessons](/lessons/): hands-on lessons from the Quickstart to scatter-gather. - [KRL manual](/krl/): the language and its libraries. - [Event Exchange Protocol](/exp/): the Sky event and query APIs. - [Cookbook](/cookbook/): runnable demos from the **Try It!** tab. The engine listens on port 3000 by default, so the examples here use `http://localhost:3000`. Change that if you've configured another port or public URL (see the [operator guide](/pico-engine)). --- Section intro for pages ported from the Pico Labs wiki (Programming Workflow, Event Loop Introduction, Developer Tips, Event and Query URLs with Postman, KRL misconceptions, and the KRL Cookbook tips), updated for this engine. --- # Developing: Programming Workflow # Programming Workflow A suggested workflow for new pico developers. Programming picos uses a small tool chain, used more or less in sequence. This page is organized by its stages: The pico tool chain: Develop (editor, Git), Build (Pico Labs, Git), Deploy and Test (Pico Labs), Release (AWS, GitHub) The loop you'll spend most of your time in is Develop → Build → Deploy (flush) → Test. Release only matters when someone other than you depends on the ruleset. ## Develop Use Git (or any version control) and your favorite editor. A KRL ruleset is a plain text file, by convention named after its ruleset id with a `.krl` extension. ```sh mkdir myproject cd myproject git init ``` Create a ruleset file in the repository, for example `io.picolabs.hello_world.krl`. Editors don't need special support, but KRL is close enough to JavaScript that syntax highlighting for JavaScript is a reasonable fallback. Write `/** … */` doc comments on your rules and shared functions, using the `@attr`, `@param`, and `@example` tags. The compiler turns them into the ruleset's *schema*, which the **Testing** tab uses to build forms. They don't change what the ruleset does. The [Quickstart](/lessons/quickstart) shows an example. ## Build "Building" a ruleset means making sure it parses and compiles. Use `krl-check`, which ships with `krl-compiler`: ```sh npm install -g krl-compiler krl-check my.ruleset.krl ``` `krl-check` parses and compiles the file, and also lints doc comments and views. It exits with status 1 when there is an error, and prints warnings to stderr. With no file argument it reads standard input. `--json` prints a machine-readable result. > **Note:** `krl-check` replaces the old parse-only hook scripts (`krl-compiler --verify`, and the hooks built on the old parser). If you still have one of those in a repository, replace it. Run it often, from your editor or a file watcher. To make sure nothing that fails to compile is ever committed, add a Git pre-commit hook. [Tips](/developing/tips) has the hook, and it checks the *staged* version of each file, which is what you want. Then add and commit as usual: ```sh git add io.picolabs.hello_world.krl git commit -m "initial commit" ``` With the hook in place, a ruleset that commits is one that compiles. (It doesn't promise the rules do what you want. That's the Test stage.) ## Deploy Deploying means installing the ruleset in each pico that needs it. If it's already installed, you only need to flush it. ### Install Open the developer UI at (port 3000 is the default), click the pico, and use the **Rulesets** tab's **Install Ruleset by URL** form. You need the ruleset's URL: - **Local file.** While developing on the same machine as the engine, use `file://` followed by the full path. The path starts with a slash, so you end up with three slashes: `file:///Users/you/myproject/io.picolabs.hello_world.krl`. The **Browse…** button stages a *copy* of the file, which is convenient for a first install but is not reread by a later flush; type the `file:///` URL yourself if you want flush to see your edits. - **GitHub.** Use the *raw* URL, `https://raw.githubusercontent.com////.krl`, not the page GitHub shows. GitHub caches raw files for a few minutes, so a change you just pushed may not be visible right away. - **The engine's own rulesets and demos.** `http://localhost:3000/krl/.krl` and `/krl/demos/…`. When the engine can't compile the ruleset, the install fails and the UI shows the error with its line and column. Rulesets tab: Install Ruleset by URL form with a file URL entered ### Flush The engine compiles a ruleset once and keeps the result. After you change the source, **flush** it so the engine fetches the URL and compiles it again. On the **Rulesets** tab, click **flush** next to the ruleset. Flushing is per URL: every pico on this engine that has the ruleset installed from that URL gets the new version. The ruleset's hash changes with each new version of the source. Installed Rulesets list with flush and uninstall buttons next to each ruleset You can also flush from KRL, which is handy in scripts: ```krl raise wrangler event "rulesets_need_flushing" ``` That re-fetches and recompiles every ruleset installed in the pico, and then raises `wrangler:rulesets_flushed`. Flushing doesn't raise `wrangler:ruleset_installed`, so a rule that initializes the ruleset on install doesn't run again. If you changed what that rule sets up, run it yourself (or reinstall in a fresh pico). If a ruleset doesn't compile, the flush fails with the same kind of error you see on install. ## Test The **Testing** tab sends events and queries to the pico without you building URLs. Choose a channel (ECI) and a ruleset. The forms come from the ruleset's schema, so the doc comments you wrote in the Develop stage become the form fields and presets. The tab keeps a history so you can send the same thing again. See [Events and Queries](/lessons/events-and-queries) for a walk-through. The **Logging** tab is where you find out why a rule did or didn't run. It lists each event and query the pico handled, newest first, with the channel and policy decision, the attributes, which rules were selected and which fired, any `klog` output, the directives returned, and the events the rules raised. Add `klog` to a binding to see a value: ```krl my_name = event:attr("name").klog("saw name: ") ``` For repeatable tests, or to try an event from a script, use `curl` or [Postman](/developing/postman) against the [Event and Query APIs](/exp/). If you run the engine under a different host name or port than `localhost:3000`, remember the browser's passkey is tied to that address (see the operator guide's `baseUrl` and `webauthn` settings). Otherwise you can ignore passkeys in the workflow: you register once, on the first visit. ## Release For local development there is nothing to do beyond Deploy. When others use your ruleset, they need a *stable* version, even while you keep working. Any HTTP server will do: - **GitHub.** Make a Git tag or a GitHub release for each version, and give users the raw URL at that tag (for example `https://raw.githubusercontent.com///v1.2.0/.krl`). Keep `master` for work in progress. - **Object storage such as S3.** Use a directory per release. Make sure the engine can read the URL. Users install from that URL. A flush rereads the same URL, so a tagged URL stays put until they choose a new one. --- Ported from the [Pico Labs wiki](https://picolabs.atlassian.net/wiki/spaces/docs/pages/32754595) (last edited there 2021-01-07), and updated for this engine. --- # Developing: Event Loop Introduction # Event Loop Introduction Each pico presents an event loop that handles the events sent to it, according to the rulesets installed in it. The event is the input that makes the cycle happen. When the event has been handled, the pico waits for the next one. Pico event evaluation cycle: wait, decode event, schedule rules, eval rules, assemble response The five phases, in order: **Wait**, **Decode**, **Schedule Rules**, **Rule Evaluation**, **Assembling the Response**. Then the pico waits again. ## Wait A pico spends nearly all its time waiting. An idle pico does no work. Events the pico receives are added to its event queue and handled in the order received. The pico leaves the wait phase when the queue has an event to process. **Picos are single-threaded.** A pico never processes two events at the same time, so rules in one pico can't race each other over its entity variables. Other events that arrive meanwhile queue up. ## Decode The decode phase unpacks the event from whatever carried it and puts it in a common form, the *request* that the rest of the cycle uses whenever it needs information about the event: its domain, type, attributes, and event id. Most events arrive over HTTP, through the [Sky event API](/exp/event) (`/sky/event////`). The channel in the URL picks the pico and its policy decides whether this domain and type is allowed in. Events sent from another pico with `event:send` arrive the same way, and events raised by the pico itself skip the transport altogether. Other transports could be supported by adding a decoder. ## Schedule Rules Building the schedule determines what happens for the rest of the cycle. The engine keeps a *salience graph* for each pico. For any event domain and type, it tells which rules are *salient*: which have an event expression that mentions that domain and type. The incoming event has one domain and one type, and the engine looks up the rules that are listening for that pair and puts them on the **schedule**. The graph is computed from the rulesets installed in the pico, so it's recalculated whenever the pico's rulesets change. It acts as an event bus: when you install a ruleset, its rules are subscribed to the events they select on, with no other wiring. Order matters in some places and not in others: - **Within a ruleset**, rules appear in the schedule in the order they are written. - **Across rulesets**, there is no defined order. Don't count on one ruleset's rules running before or after another's. If you need an order, make it explicit with events (see below). "Salient" doesn't mean "will run". A rule on the schedule has only shown that it listens for this domain and type. ## Rule Evaluation The engine goes down the schedule one rule at a time. For each rule it evaluates the event expression to see whether the rule is *selected*, and if so runs it: the prelude, the action when its condition is true, and then the postlude. A rule can be on the schedule and still not be selected, such as when a compound event expression is waiting for another event, or a `where` clause is false. Most of what a rule does doesn't affect the loop. Two things do: - **`raise` appends to the schedule.** A `raise` in a postlude doesn't call anything. The engine looks up the rules salient for the raised event and **appends** them to the end of the schedule, and the current rule's evaluation continues from where it left off. Everything scheduled by the original event runs before anything scheduled by a raised event. This is why `raise` is not a function call, and why a rule that raises an event and then expects the effect right away is surprised. See [KRL misconceptions](/developing/misconceptions#order-of-rule-evaluation). - **`last`** in a postlude stops the remaining rules of that ruleset on the schedule. Rules that raise events are still within the same pico and the same cycle. The whole cascade runs, and the schedule empties, before the cycle ends. Two other kinds of effects are outside the schedule: - **HTTP calls.** A synchronous call to an external API from the [http library](/krl/libraries/http) makes the rule wait until the resource answers, and the pico handles nothing else meanwhile, since it's single-threaded. - **Sending an event to another pico.** `event:send` is **asynchronous**. It starts an independent cycle in the receiving pico and doesn't change the schedule you're running. If both picos are on the same engine they run concurrently, though not necessarily in parallel. You can't assume anything about their relative order unless you make one, usually by having the receiver send an event back when it's done. The [Scatter-Gather](/lessons/scatter-gather) lesson does exactly this, with a correlation id so the gatherer can tell which reply is which. If a pico sends an event to its *own* channel, that event goes to the back of its queue and is handled in a later cycle, since the pico is busy. ## Assembling the Response When the schedule is empty, the engine assembles the response from the output of all the rules that fired: the directives (`send_directive`), the outcome of each rule, and the events raised during the cycle. Over HTTP this is the JSON body described on the [Event API](/exp/event#response) page. An event having a response is unusual, and mostly a historical convenience. For an event sent synchronously over HTTP the response confirms that it arrived and was processed, and it can carry useful results. Events sent asynchronously, including `event:send` from a rule, never have a response the sender can use. Responses are optional and depend on which rules ran. ## Waiting again Once the response has gone, the pico takes the next event in its queue, or waits if the queue is empty. Queries are separate. They call a shared function and return its value, and they don't run rules or change the pico (see [Query API](/exp/query)). ## See also - [Events and Queries](/lessons/events-and-queries): events and queries over HTTP, with the Logging tab showing each cycle - [Scatter-Gather](/lessons/scatter-gather): ordering concurrent picos explicitly - [KRL misconceptions and common errors](/developing/misconceptions) - [Event expressions](/krl/event-expressions/) and [raising explicit events](/krl/rulesets/rules/postlude/raising-explicit-events-in-the-postlude) in the KRL manual --- Ported from the [Pico Labs wiki](https://picolabs.atlassian.net/wiki/spaces/docs/pages/1189951) (by Phil Windley; last edited there 2021-02-22), and updated for this engine. --- # Developing: Tips # Tips Practical notes for running the engine and writing KRL. Engine setup is below; technique pages (from the old Patterns and tips / KRL Cookbook) are indexed further down. The [operator guide](/pico-engine) is the full configuration reference. ## Managing your engine ### Install, update, and check the version ```sh npm install -g pico-engine # install, or update to the latest npm install -g pico-engine@ # a specific version pico-engine # run it (port 3000 by default) pico-engine --version # which version is this? npm ls -g --depth 0 pico-engine # what npm has installed ``` Back up your data directory (`~/.pico-engine` by default) before you update, and especially before you *downgrade*. A different engine version may not read what a newer one wrote. The version also appears at the bottom of the developer UI's root page. The developer UI footer: Powered by pico-engine, with the version number To remove the engine code (not your picos, which live in the data directory): ```sh npm rm -g pico-engine ``` ### Start over The engine keeps everything in its data directory, `PICO_ENGINE_HOME`, which defaults to `~/.pico-engine`: | Folder | What's in it | |--------|--------------| | `db/` | Picos, channels, users, identity | | `rulesets-db/` | Which URL each ruleset came from (rebuildable) | | `rulesets/` | Compiled rulesets (rebuildable) | To wipe all picos and begin with a fresh engine, stop the engine and delete the data directory. You then register a new passkey on your next visit, and get a new root pico: ```sh rm -r ~/.pico-engine ``` > **There is no undo.** Deleting `db/` deletes your picos. Deleting only `rulesets-db/` and `rulesets/` is harmless: they're rebuilt when rulesets are installed again. Adjust the path if you set `PICO_ENGINE_HOME` to somewhere else. ### Keep separate projects apart The data directory is an *image* of the engine's state: all its picos and rulesets. Point `PICO_ENGINE_HOME` at different directories to keep projects separate, and return to where you left off in each: ```sh PICO_ENGINE_HOME=~/.game_image pico-engine PICO_ENGINE_HOME=~/.iot_image pico-engine ``` ### Run more than one engine To run several engines on one machine at once, give each its own port, its own data directory, and the matching public URL: ```sh PORT=3001 PICO_ENGINE_HOME=~/.pico-engine-1 pico-engine ``` Never point two running engines at the same data directory. Engines can also be configured with a config file per process (`pico-engine -c file.json`). See [Multiple engines on one host](/pico-engine#multiple-engines-on-one-host). ### The base URL The engine knows its own address, `http://localhost:3000` by default, and gives it to your KRL as `meta:host` (and uses it to build links). To use another address, such as a machine's name or a public domain, set `PICO_ENGINE_BASE_URL`: ```sh PICO_ENGINE_BASE_URL=http://my-machine.local:3000 pico-engine ``` This is also what passkeys are bound to: the browser's passkey only works at the address the engine thinks it has. If you open the engine from a different host name than the one you registered with, sign-in fails; see `webauthn` in the operator guide. The engine reads the address once, at startup. TLS isn't terminated by the engine, so for HTTPS put a reverse proxy in front of it and set the base URL to the public HTTPS address. ### Docker There's an official image, `picolabs/pico-engine`. Mount a volume for the data directory and set the same variables you'd set for the command line: ```sh docker run -p 3001:3000 \ -v ~/images/pico-image:/var/pico-image \ -e PICO_ENGINE_HOME=/var/pico-image \ -e PICO_ENGINE_BASE_URL=http://localhost:3001 \ picolabs/pico-engine: ``` You can run the image several times with different ports and volumes to get several independent engines. Don't share one volume between running containers. The package README covers Docker Compose and AWS Fargate with EFS. ### Windows WSL 2 (a Linux environment on Windows) is a good way to run the engine on Windows. If a browser on Windows can't reach an engine running in WSL 2, the usual culprit is bridging the port between Windows and the virtual machine. Also check that the base URL matches the address the browser uses. ## Hosting, flushing, and checking rulesets ### Host the source where the engine can reach it The engine installs a ruleset from a URL and remembers it: - **Your own files**, while you work: `file:///path/to/my.ruleset.krl` (three slashes). The engine must be on the same machine. - **GitHub:** use the *raw* URL, `https://raw.githubusercontent.com////.krl`, not the page that displays the file. GitHub caches raw files for a few minutes. For a private repository the engine needs a URL that includes an access token. - **The engine's own:** `http:///krl/.krl` for rulesets that ship with it, and `/krl/demos/...` for demos. - **S3 or any static host:** make sure the URL is readable by the engine. ### Flush after every change The engine compiles each ruleset once and caches it. After you change the source, flush it so the engine fetches and compiles it again: - **From the UI.** On the pico's **Rulesets** tab, click **flush** next to the ruleset. - **From a rule.** `raise wrangler event "rulesets_need_flushing"` re-fetches every ruleset installed in the pico and then raises `wrangler:rulesets_flushed`. The Rulesets tab with hello_world installed, and its flush and uninstall buttons If the changed source doesn't compile, the flush fails and the error appears next to the ruleset: The Rulesets tab showing a ParseError next to hello_world after a flush Flushing is by URL. The engine keeps one compiled version per URL, so every pico with that ruleset gets the update. Flushing does **not** raise `wrangler:ruleset_installed` again, so rules that run on install don't run. If you change what such a rule sets up (for example a channel's policy), run that rule again yourself. The [Bootstrap and channels](/cookbook/bootstrap/) demo shows how. The UI uses `/api/flush?url=`, which needs the signed-in browser session, so it doesn't work from a plain `curl`; use the UI or the wrangler event instead. ### Check your KRL before the engine sees it ```sh npm install -g krl-compiler krl-check my.ruleset.krl ``` `krl-check` parses and compiles, and lints doc comments and views. It exits with status 1 on an error and prints warnings to stderr. With no file, it reads standard input. Keep `krl-compiler` at the same version as your engine. Editor with krl-check validation output before flushing a ruleset A Git pre-commit hook that checks the *staged* version of each `.krl` file: ```sh #!/bin/sh status=0 for f in $(git diff --cached --name-only --diff-filter=ACM -- '*.krl'); do if ! git show ":$f" | krl-check >/dev/null; then echo "krl-check failed: $f" status=1 fi done exit $status ``` Save it as `.git/hooks/pre-commit` and make it executable. `git show ":$f"` is the staged copy, so a fix you haven't staged yet doesn't hide an error. See also [Programming Workflow](/developing/workflow). ## Writing and debugging ### Use the Testing and Logging tabs - The **Testing** tab builds a form for each event and query a ruleset offers, using the ruleset's doc comments (`@attr`, `@param`, `@example`). Document your rules and shared functions and the forms write themselves. - The **Logging** tab shows what the pico did for each event or query: the channel and its policy decision, the attributes, which rules were selected and which fired, directives, raised events, and `klog` output. Use the filters to narrow it to a ruleset, an event, or errors. It keeps recent transactions per pico in memory (200 by default, `PICO_ENGINE_LOG_RING`), so a restart clears it. The log file on disk keeps the full record; see [Reading the engine's log files](/developing/engine-log-files). - `.klog("label: ")` returns the value it's applied to, so you can add it in the middle of an expression without changing the code. ### "Not allowed by channel policy" This is the most common surprise when calling a pico from outside. The channel you used doesn't permit that event or query. Try another channel on the **Channels** tab, or give the channel a policy that allows what you need. See [Managing channels](/wrangler/channels). ### Initialize a ruleset when it's installed Rulesets often need setup: default values for entity variables, a scheduled event, a channel. Wrangler raises `wrangler:ruleset_installed` when a ruleset is installed in a pico, and a rule in your ruleset can select on that event. Always check that the ruleset that was installed is yours, or the rule runs every time *any* ruleset is installed: ```krl rule initialize_ruleset { select when wrangler ruleset_installed where event:attr("rids") >< meta:rid if ent:heartbeat_period.isnull() then noop() fired { ent:heartbeat_period := event:attr("heartbeat_period") || 60; ent:emitter_state := "running" } } ``` Two cautions: - The event isn't raised again on a flush, so a change to this rule's setup doesn't take effect on picos that already have the ruleset. - Guard each initialization with `isnull()` so you never overwrite state that has accumulated. More: [Initializing entity variables](/developing/initializing-entity-variables) and [Bootstrap and channels](/cookbook/bootstrap/). ### Wrangler, not `ctx`, in application rulesets For children, channels, ruleset installation, and queries between picos, use wrangler's events and functions (`wrangler:new_child_request`, `wrangler:picoQuery`, and so on). The engine's lower-level `ctx:*` actions exist for wrangler and the platform to use. See [Wrangler](/wrangler/). ## Technique tips Short techniques for writing KRL and working with picos, checked against this engine. For runnable examples, see the [Cookbook](/cookbook/) and the **Try It!** tab. Event and query URLs live in the [Event Exchange Protocol](/exp/). ### Working with data | Page | What it covers | |------|----------------| | [Initializing entity variables](/developing/initializing-entity-variables) | `defaultsTo`, a guard rule, or a rule that runs on install | | [Arrays and maps](/developing/arrays-and-maps) | Loops with `reduce`, flattening, and sorting arrays of maps by one key or two | | [Decoding form-encoded data](/developing/decoding-form-data) | Turning `a=1&b=2` into a map, keeping repeated names, and decoding `+` and `%xx` | | [Importing tables and CSV](/developing/tabular-data) | Tab- and comma-separated exports to arrays and maps, fetched with `http:get` | | [Converting time formats](/developing/converting-time-formats) | RSS (RFC 2822) dates to the RFC 3339 form the time library uses | | [What's in a name?](/developing/names-and-scope) | KRL's separate kinds of names, and how a `pre` declaration shadows a global | ### HTTP and the outside world | Page | What it covers | |------|----------------| | [Cookies and redirects](/developing/cookies-and-redirects) | The `_cookie` and `_redirect` directives, reading cookies, and a URL shortener | | [Events from email](/developing/events-from-email) | A mail webhook to an event, routing by address, and turning message lines into events | | [Reading the engine's log files](/developing/engine-log-files) | Where the logs are, how they rotate, and one pico's history with `grep` and `jq` | | [Event and Query URLs with Postman](/developing/postman) | Building Sky requests in Postman or `curl` | ### Picos working together | Page | What it covers | |------|----------------| | [Querying another pico](/developing/querying-another-pico) | `wrangler:picoQuery`, which channel to use, why a pico can't query itself, and `http:get` | | [Accepting relationship requests](/developing/accepting-relationship-requests) | An approving rule, wrangler's auto-accept configuration, and a ruleset that accepts later and removes itself | | [One pico per resource](/developing/singleton-picos) | Giving a device or account its own pico, so requests can't collide | ### Patterns that became demos | Pattern | Demo | |---------|------| | Idempotency and guard rules; authorize, then use | [Guard rules](/cookbook/guard_rules/) | | Splitting rules for clarity and extensibility; layering functionality with rulesets; semantic translation | [Layered rulesets](/cookbook/layered_rulesets/) | | Collection picos | [Collection](/cookbook/collection/) | | Access tokens and other secrets; keeping keys safe | [Configuration and secrets](/cookbook/secrets/) | | Installing a multi-ruleset application; creating a channel for a ruleset | [Bootstrap and channels](/cookbook/bootstrap/) | | An echo server | [Echo](/cookbook/echo/) | | A score tracker | [Score tracker](/cookbook/score_tracker/) | | A child pico for an asynchronous task | [Child pico tasks](/cookbook/child_tasks/) | | Automatic replies to relationship requests; querying over a relationship | [Two picos talking](/cookbook/two_picos/) | | An event hierarchy across rulesets | [KBlog](/cookbook/kblog/) | ### In the KRL manual - [Persistent variables](/krl/persistent-variables/) - [User-defined actions](/krl/krl-expression-language/user-defined-actions) ### Left out of the old cookbook - **Sending secure messages** used `engine:signChannelMessage` and its relatives, which are gone. Relationships can use DIDComm; see [Relationships](/wrangler/relationships). - **Shorter URL for a query or event** used `io.picolabs.rewrite` on the root pico, removed in engine 1.0. A ruleset can do the same with `_redirect`: see [Cookies and redirects](/developing/cookies-and-redirects). - **Account management using the root pico** described owner picos from before engine 1.0. - **Generate an image with KRL**, **Using Foursquare data**, **Using Wovyn sensors**, **Using Git**, and **Using a Google sheet** are about particular services rather than KRL. - **Proposals, experiments, and proofs of concept** were experiments on earlier engines. --- Merges the [Developer Tips](https://picolabs.atlassian.net/wiki/spaces/docs/pages/31544297) wiki page (last edited there 2021-04-12) with the Patterns and tips / KRL Cookbook pages that still apply, updated for this engine. --- # Developing: Event and Query URLs with Postman # Event and Query URLs with Postman [Postman](https://www.postman.com/) is a desktop tool for building and sending HTTP requests and examining the responses. It's handy for trying events and queries on a pico: you can save requests, change attributes quickly, and keep them in a collection. Everything here is plain HTTP, so `curl` works equally well. Each Postman step has a `curl` equivalent. For the full API, see the [Event API](/exp/event) and [Query API](/exp/query). For a gentler introduction, see [Events and Queries](/lessons/events-and-queries). The examples use `http://localhost:3000`, the engine's default address. Use your own host and port if you've changed them. ## Before you start: pick a channel Both events and queries are sent to a pico through one of its **channels**, and a channel is named by its ECI (event channel identifier). The channel's *policy* decides which events and queries it allows. 1. Open the developer UI, click the pico, and open its **Channels** tab. 2. Choose a channel whose policies allow what you want to send. The pico's own `self` channel (its ECI is the pico's id) allows everything, which is fine for experiments. For anything you'll leave around, make a channel that allows only what's needed. See [Managing channels](/wrangler/channels). 3. Copy the ECI. Channels tab showing a channel and its ECI to copy If a request is refused, you'll get HTTP 200 and `{"error":"Error: Not allowed by channel policy"}`. Use a channel with a more permissive policy. If the mesh has [OAuth](/wrangler/oauth) turned on, requests also need an `Authorization: Bearer ` header, which in Postman is the **Authorization** tab. ## Raising an event The event URL looks like this: ``` http://localhost:3000/sky/event//// ``` - ``: the channel - ``: an event id you choose, for your own tracing (`none` is common) - `` and ``: the event's domain and type, which rules select on This example asks wrangler to create a child pico: the domain is `wrangler` and the type is `new_child_request`. > **Older docs and rulesets** call this event `child_creation`. The current wrangler event is `wrangler:new_child_request`. Likewise, in URLs `/sky/query/` replaces `/sky/cloud/`. ### In Postman 1. Create a new request tab. Postman: create a new request tab 2. Set the method to **POST**. (GET works for events, but POST is preferred because an event may change the pico.) Postman request method set to POST 3. Enter the URL, with your ECI: ``` http://localhost:3000/sky/event//child_test/wrangler/new_child_request ``` Postman address bar with the Sky event URL 4. Open the **Body** tab, choose **raw**, and set the type to **JSON**. Put the event's attributes in the body as a JSON object: Postman Body tab with raw and JSON selected ```json { "name": "test child", "backgroundColor": "#87cefa" } ``` Postman raw JSON body with event attributes 5. Click **Send**. Postman Send button to submit the request *These screenshots are from an earlier release: they show port 8080 and the old `child_creation` event name. Use the URL and event above.* A JSON body is the best way to send attributes: it keeps numbers, booleans, and nested maps as they are, and avoids URL-encoding. You can also send attributes as a query string (`?name=test%20child`) or as `application/x-www-form-urlencoded`. When a name appears in both body and query string, the body wins. ### The response The response is JSON. It carries the transaction id, any directives from rules that ran, which rules fired, and the events raised during the cycle: Postman response JSON with directives and rule outcomes after Send ```json { "eid": "…", "directives": [ … ], "ruleOutcomes": [ { "rid": "io.picolabs.wrangler", "rule": "createChild", "status": "fired" } ], "events": [ { "domain": "wrangler", "name": "new_child_created", "attrs": { … } } ] } ``` For a wrangler event like this one there may be no directive: the `events` list and the new child in the developer UI are your confirmation. A rule that calls `send_directive` shows up under `directives`. The [Event API](/exp/event#response) page describes every field. ### With curl ```sh curl -X POST "http://localhost:3000/sky/event//child_test/wrangler/new_child_request" \ -H 'content-type: application/json' \ -d '{"name": "test child", "backgroundColor": "#87cefa"}' ``` ## Querying a pico A query calls a function that a ruleset lists in its `shares`, and returns the function's value. It can't change the pico. The URL looks like this: ``` http://localhost:3000/sky/query///? ``` - ``: the ruleset id that shares the function - ``: the function's name - arguments: the function's parameters, by name (order doesn't matter) ### In Postman 1. Create a new request tab and set the method to **GET**. Postman: create a new request tab for a query Postman request method set to GET 2. Choose the pico, the ruleset, and a function that ruleset shares. The pico's **Testing** tab lists the queries a ruleset shares, and the **Channels** tab has the ECI. This example uses wrangler's `children` query: Developer UI Testing tab: pico, ruleset, and shared function Channels tab ECI for the query URL ``` http://localhost:3000/sky/query//io.picolabs.wrangler/children ``` 3. For a function with parameters, add them on the **Params** tab (Postman builds the query string), such as `name = Ada` for `…/io.picolabs.hello_world/hello?name=Ada`. The names must match the function's parameter names. Postman Params tab and query URL with parameters 4. Click **Send**. The body is the function's return value, as JSON. Postman response body showing the query result as JSON If the function isn't in the ruleset's `shares` list, or the ruleset isn't installed in that pico, the response has an `error` field instead of a value. A GET is the natural method, though POST with a form-encoded or JSON body also works and is useful for long arguments. ### With curl ```sh curl "http://localhost:3000/sky/query//io.picolabs.wrangler/children" curl "http://localhost:3000/sky/query//io.picolabs.hello_world/hello?name=Ada" ``` ## Tips - Use Postman **variables** (such as `{{eci}}` and `{{base}}`) for the base URL and ECI, so a saved request works on another pico or engine by changing one value. - Add `.txt` or `.html` to a function name (`…/page.html`) to get the function's string result as plain text or a web page. See the [Query API](/exp/query#text-and-html). - If a rule doesn't seem to run, open the pico's **Logging** tab and look at the transaction your request created. - To exercise a whole system, not one pico, the [Programming lessons](/lessons/) and the [EXP](/exp/) pages show how events and queries fit together. --- Ported from the [Pico Labs wiki](https://picolabs.atlassian.net/wiki/spaces/docs/pages/1177157666) (last edited there 2020-06-26), and updated for this engine. --- # Developing: KRL misconceptions and common errors # KRL misconceptions and common errors KRL is the programming language for picos. It's a notation for how a pico reacts to events and how it answers queries. This page collects the places where habits from other languages mislead, and the small mistakes that are hard to see. Skim the headings first: most readers recognize their own bug. ## KRL is not JavaScript KRL's syntax looks like JavaScript's: braces, arithmetic and Boolean expressions, things that look like assignment, function definitions and calls, `if`. There is no `for` or `while`, and there are constructs JavaScript doesn't have, such as `select when`, `foreach`, and `defaction`. It has a different execution model and a different memory model. KRL compiles to JavaScript, and the engine is a Node.js program, but a ruleset isn't a program that runs. It's a set of rules that a pico evaluates in reaction to events. The unit of compilation is the ruleset. Rule names must be unique within it, and the *order* the rules are written in matters. Its `global` block can define functions, which evaluate in the context of the ruleset and the pico they're installed in. ## Common misconceptions ### Request/response vs. event/action and query/response If you know HTTP programming, you may think of a pico as a resource. With REST, a GET shouldn't change the resource and a POST or PUT often does. With picos: - A **query** can never change the pico. It calls a function the ruleset lists in `shares` and returns the value, given the current state. Functions can't change state, and an HTTP POST isn't allowed in one. (KRL does let a function make a synchronous HTTP GET to use outside information, and a poorly designed resource may change in response to a GET, but that is the outside world's problem.) - Only an **event** can change a pico. Rules react by (conditionally) taking actions on the outside world and by changing entity variables in the postlude, and they can raise more events to the same pico. Think of a pico as a state machine: queries tell you what state it's in; events (maybe) change it. An event's rules can change things in two places: **outside** the pico, through actions, and **inside** it, through the postlude. A pico with changes outside (action) above it and changes inside (postlude) within it Queries are `GET`s to `/sky/query///?…` and events are `POST`s (or `GET`s) to `/sky/event////`. The first path component is the giveaway. The event URL names the pico, the domain, and the type, but **not a ruleset**: every installed ruleset with a salient rule gets a chance. A query names the ruleset and function, and only that one answers. See the [Event API](/exp/event), the [Query API](/exp/query), and [Event Loop Introduction](/developing/event-loop). The response to an event isn't produced until the whole cycle, including the rules that events raised along the way scheduled, has finished. Actions on the outside world happen as the rules evaluate, not at the end. ### Name binding is not variable assignment Most languages have an assignment that replaces the value at an address. KRL has two different things, and they're written differently. **Declarations bind a name to a value.** In a `global` or `pre` block, `x = 3` binds the name `x`. The left side is always a plain name. There is no assignment, and the binding is in force from that point on: through the rest of the `global` block and the rules, or through the rest of the rule. Names are also bound by `setting(...)` in an event expression, by an action, and by `foreach`. Don't declare a name twice. A second declaration doesn't change the first. It shadows it, and functions that closed over the first still see the first. The compiler warns about a duplicate declaration, so take notice of that. ```krl pre { x = 3 y = function(a){ x + a } x = 5 // shadows, doesn't change; y(6) is still 9 } ``` Rather than "refining" a binding, give each step its own name. See [Declaration Semantics](/krl/rulesets/rules/prelude/declarations/declaration-semantics). **Entity variable assignment** changes pico state. It uses `:=`, only in a rule's postlude, so only events can change state: ```krl fired { ent:count := ent:count.defaultsTo(0) + 1; ent:rooms{[room, "temp"]} := temp // path assignment for a nested key } ``` Use the *composite path* form, `ent:m{[a, "b"]} := v`, to set a nested key. Chained braces (`ent:m{a}{"b"} := v`) don't parse as an assignment target, and building a new map with `put` and assigning the whole thing back is more work and slower for large values. Each pico keeps its own state: the entity variables of each ruleset installed in it, which persist. ### Operators produce new values KRL's operators never change a structure in place. They return a new one. A line like this ```krl temp = ent:var temp[1].put("b", temp[1]{"b"} + 1) // computes a new map; changes neither temp nor ent:var ``` computes a new value and then throws it away, because nothing receives it. Even if you used it, the result would be only the changed *part*, not the whole structure. To change state, assign the new whole value, or a path, in a postlude: ```krl fired { ent:counts{[room, "n"]} := ent:counts{[room, "n"]}.defaultsTo(0) + 1 } ``` ### Order of rule evaluation When an event arrives, the engine builds a **schedule** of the rules salient for it. Rules in one ruleset are in the order they're written. Across rulesets the order is undefined. Evaluation then runs down the schedule. A `raise` in a postlude doesn't run anything. The engine finds the rules salient for the raised event and **appends** them to the end of the schedule. They run after *all* the rules already on it. The lexical order applies each time rules are looked up, once for the arriving event and again for each raised one, so it can produce surprising results. Here's a real example. A ruleset records temperature readings to a spreadsheet at `ent:url`. To change the URL each month, someone added a rule *before* the recording rule, so it would run first when `wovyn:new_temperature_reading` arrived: ```krl rule set_up_url { select when probe_temp_recorder new_url url re#^(http.*)# setting(url) fired { ent:url := url; ent:urlInEffectSince := time:now() } } rule check_for_new_month { select when wovyn new_temperature_reading pre { … } if new_month && new_url then noop() fired { raise probe_temp_recorder event "new_url" attributes { "url": new_url } } } rule record_probe_temp_to_sheet { select when wovyn new_temperature_reading where ent:url … http:post(ent:url, qs = data) setting(response) } ``` `set_up_url` is written first, but the schedule for the original event was `check_for_new_month` then `record_probe_temp_to_sheet`. The raised `new_url` event put `set_up_url` at the *end*, so it ran last: the first reading of a new month went to the old URL. The fix was to do the work directly: put the postlude statements of `set_up_url` in `check_for_new_month`'s postlude instead of raising. It's tempting to avoid the repetition by chaining with `raise`, but that doesn't work when the order matters. If one rule's result must be visible to another rule in the same cycle, do the assignment in the first rule's postlude (rules later in the schedule see it) rather than through a raised event. ### Interference between rulesets An event goes to *every* installed ruleset with a salient rule. Usually just one ruleset cares, but make sure your rules react only to what they should. The standard example is `wrangler:ruleset_installed`, raised when a ruleset is installed in a pico. Every installed ruleset sees it, not only the one just installed. The `rids` attribute lists the rulesets that were installed, so guard with it: ```krl select when wrangler ruleset_installed where event:attr("rids") >< meta:rid ``` Another example is creating a child. A rule that asks wrangler for a child with `raise wrangler event "new_child_request"` gets `wrangler:new_child_created` back, but so does any other rule in any ruleset that selects it, whether or not it asked. Correlate with an attribute that passes through, such as `meta:txnId`, which is unique to this schedule: ```krl rule make_child { select when app new_thing name re#(.+)# setting(name) fired { raise wrangler event "new_child_request" attributes { "name": name, "txn_id": meta:txnId } } } rule child_ready { select when wrangler new_child_created where event:attr("txn_id") == meta:txnId … } ``` The second rule only selects when the child it asked for is created, not when someone creates a child by hand in the UI. Wrangler passes the extra attributes of the request along. (Note `meta:txnId` works here because both rules run in the same cycle. If your reply arrives in a later one, which is common with `event:send`, pass a correlation id of your own instead.) ### A rule's name is not its event The name of a rule is for reference (in logs, for example) and must be unique within a ruleset. The event it listens to is in its `select when`. Rule names are usually verbs and event types usually nouns. ```krl rule send_twilio_sms_message { select when twilio new_sms_message … } ``` This rule is **not** triggered by raising `twilio:send_twilio_sms_message`. That's a common mistake: rules aren't procedures that you call by raising an event named after them. Raise the event the rule selects on, `twilio:new_sms_message`. When two rules select the same event, both are scheduled, in the order written. That makes this pattern work: ```krl rule check_for_a_known_number { select when twilio new_sms_message if ent:known_numbers >< event:attr("to") then noop() notfired { last } } rule send_twilio_sms_message { select when twilio new_sms_message … } ``` If the number is unknown, `last` in the postlude stops the rest of *this ruleset's* rules for the event. `send_twilio_sms_message` was selected but isn't evaluated. (`last` doesn't affect other rulesets' rules on the schedule.) ### One action block and one postlude A rule has at most **one** condition, one action block, and one postlude: ```krl rule name { select when domain type pre { x = event:attr("x") } if cond then every { action1(); action2() } fired { ent:things{[id, "name"]} := x; raise domain event "next" attributes { "x": x } } else { error warn <> } } ``` - Several actions go in one `every { … }` block. Don't write two `if … then` blocks, and there's no `else` on the action. - The postlude is `fired`, `notfired`, or `always`. `fired` and `notfired` take an optional `else` (which runs in the opposite case), and an optional `finally`, which runs either way. Don't write `fired { } notfired { }` as two blocks. `finally` can only come after `fired` or `notfired`. Use `always` when you want something to run whatever happened, with nothing else. - `event:send` is an action, so it goes in the action block, not in a postlude. - `if X then noop()` with `fired { … }` runs the postlude when `X` is **true**. Use `notfired` for the other case. ### Strings: double quotes and chevrons A double-quoted string is a constant. It can't contain line breaks, and the compiler turns it into a JavaScript string. A chevron-quoted string (`<< … >>`) can contain line breaks and unescaped double quotes. It can also contain **beestings**, `#{expression}`, which are evaluated at run time, so the value can differ each time. When that happens depends on where it appears: in `global`, when the ruleset is installed; in a function, each time it's called; in a prelude, each time the rule selects; in a postlude, each time it runs. ```krl url = <<#{meta:host}/sky/event/#{eci}/none/byu_hr_login/logout_request>> ``` A beesting can contain a chevron string: ```krl #{is_self => << title="this is you">> | ""} ``` (or, less readable, `#{is_self => " title=\"this is you\"" | ""}`). When a chevron string holds markup, put the closing `>>` on its own line, and the opening `<<` too if the first thing is a tag. `>>>` is ambiguous (the compiler sees `>>`, then a stray `>`): ```krl << " selected" | ""}>#{e} >> ``` KRL chevron string with beestings for selected option markup ### KRL isn't Java: `raise` is not a call A ruleset installed as a module doesn't make an object. The pico pre-existed, and the ruleset's entity variables belong to that pico. Consider a ruleset that offers a "setter" by event: ```krl ruleset holder { meta { provides getter } global { getter = function(){ ent:stuff } } rule stuffer { select when holder setter fired { ent:stuff := event:attrs{"stuff"}; raise holder event "stuffed" attributes event:attrs } } } ``` and a naive caller: ```krl ruleset java_pattern { meta { use module holder } rule checker { select when user checker pre { value = random:word() } fired { ent:value := value; raise holder event "setter" attributes {"stuff": value} // schedules the setter, doesn't call it } } rule tester { select when user checker send_directive("result", {"okay": ent:value == holder:getter()}) // false } } ``` The schedule when `user:checker` arrives is `checker`, then `tester`. `holder:setter` is *appended*, so `tester` runs before `stuffer` does and reports `false`. "Saves it away" should have been read as "schedules it to be saved, later." The right pattern uses the terminal event the module raises, `holder:stuffed`, which it provides for exactly this purpose: ```krl rule tester { select when holder stuffed send_directive("result", {"okay": ent:value == holder:getter()}) } ``` A module that does work in response to events should raise a *terminal* event, one it doesn't react to itself, so callers know when it has finished. ## Common errors ### Regular expression capture groups `setting` binds to a capture group, which needs parentheses in the regular expression: ```krl select when phone incoming number re#\d{10}# setting(caller_id) // caller_id is never bound select when phone incoming number re#(\d{10})# setting(caller_id) // right ``` Without the parentheses `caller_id` stays unbound and evaluates to `null` when used. ### Regular expression capture vs. explicit binding These look equivalent and aren't: ```krl select when console incoming attr_name re#(.+)# setting(local_name) ``` ```krl select when console incoming pre { local_name = event:attr("attr_name") } ``` The first only selects when `attr_name` matches. Regular expressions work on a single line, so with a multi-line value, don't count on getting more than the first line. The second binds the whole attribute value, which is also what you want when the value might be empty or absent. Use `event:attr` in the prelude for a multi-line value. ### Forgetting the parentheses on a function call A function is called with parentheses, even with no arguments: `f()`. Without them, the name evaluates to the function itself, and the errors that follow can be astonishing. ### Dots, colons, and braces - **Dots** introduce operators: `list.length()`. Don't use a dot to get a map key, as in JavaScript. - **Colons** separate a module or library name from what it provides: `time:now()`, `ent:name`. A dot in its place gives an unknown operator error. - **Curly braces** after a map get the value for a key or path: `m{"k"}`, `m{["a","b"]}`. - **Square brackets** index arrays: `a[0]`. ### A name or a string in a map reference Inside braces, a quoted key is the literal string; a bare name is the value the name is bound to. It's easy to get this wrong: ```krl their_key = event:attr("sender_key") se = ent:se{"their_key"} // the key "their_key", almost certainly null se = ent:se{their_key} // the key the name is bound to ``` The first always gives `null`, and the `http:post` that uses it will always fail. ### The empty string isn't null A blank form field arrives as the empty string, not as a missing attribute, so `event:attr("name")` is `""`, not `null`. Then: - `isnull()` is `false`. - `defaultsTo()` only replaces `null` (or a missing value), so `event:attr("name").defaultsTo(ent:name)` stays empty. Use `||` when you also want to replace the empty string (and other false values): ```krl name = event:attr("name") || ent:name ``` Also the string `"null"` is not `null`. `"null".decode()` is `null`, and so is `null.decode()`, so if you're not sure, decode. (`decode` returns a string unchanged if it isn't JSON.) ### Regular expressions and case A regex is case sensitive unless you flag it: ```krl "1E4C".match(re#1e4c#) // false "1E4C".match(re#1e4c#i) // true ``` A regex made from a string, `"1e4c".as("RegExp")`, has no way to carry the `i` flag. Change the case of the string you're matching (or the pattern) with `uc()` or `lc()` instead: ```krl "1E4C".match("1e4c".uc().as("RegExp")) // true ``` ### Operator precedence Infix operators rarely need parentheses, but sometimes you need them, and the bug is subtle. The conditional (`=>`) binds more loosely than `+`. This conditionally-added part didn't do what was intended: ```krl html:header("Tic Tac Toe", css) + <<

…

>> + board + need_detail => << Moves: #{moves.encode()} >> | "" // wrong: the whole sum is the test + js ``` Everything before `=>` was concatenated into the condition. It's a non-empty string, so true, and the result was just the "Moves" part. Parenthesize: ```krl + (need_detail => << Moves: #{moves.encode()} >> | "") ``` A conditional inside a beesting is already enclosed, so it doesn't need them. ### Attributes when you generate an event There are two ways to generate an event from a rule, and each names its attributes differently. **`event:send`** is an *action* (action block only, no `use module` needed). Its first argument is a map. Keys: `"eci"` (the target channel, usually another pico's), `"domain"`, `"type"` (or `"name"`), optionally `"eid"`, and `"attrs"` (a map of the attributes). The event goes out asynchronously, perhaps to another engine. If the ECI is your own pico's, the event is queued and handled in a later cycle. **`raise … event … attributes …`** is a postlude statement. The event always goes to *this* pico, and the rules it selects are scheduled in the same cycle, after the ones already scheduled. It's easy to write `"attributes"` where you mean `"attrs"`, or the other way around. And a key that `event:send` doesn't know is silently ignored, so this sends *no* attributes: ```krl event:send({ "eci": eci, "domain": "wrangler", "type": "ready_for_deletion", "co_id": meta:rid // not a recognized key; ignored }) ``` The attributes belong under `"attrs"`: ```krl event:send({ "eci": eci, "domain": "wrangler", "type": "ready_for_deletion", "attrs": { "co_id": meta:rid } }) ``` ### An empty array is truthy KRL follows JavaScript's truthiness rules: `0`, `""`, `false`, and `null` are false, but an empty array is **true**. A test like ```krl pending => handle_them() | nothing_to_do() // pending is an array of requests ``` is true even when `pending` is `[]`, so `handle_them()` is chosen when there's nothing to handle. Test the length, since zero is false: ```krl pending.length() => handle_them() | nothing_to_do() ``` ### A domain isn't a ruleset id An event domain is often similar to a ruleset id, but a ruleset id like `byu.hr.connect` has dots, which a bare event domain can't. So you can't use `meta:rid` where a domain is expected. Spell out the domain (`byu_hr_connect`), or convert: `"byu.hr.connect".replace(re#[.]#g, "_")`. ### Initializing entity variables on install A rule that gives entity variables initial values when a ruleset is installed needs two guards: ```krl rule initialize { select when wrangler ruleset_installed where event:attr("rids") >< meta:rid if ent:apps.isnull() then noop() fired { ent:apps := built_ins() } } ``` - The `where` clause. Without it the rule selects whenever *any* ruleset is installed in the pico, and resets the variable each time, erasing changes made since. - The `isnull()` test. It protects against the ruleset being initialized again. A pico's state can accumulate for months, and a reinstall that re-initializes it would throw that away. (Wrangler raises `ruleset_installed` only when the ruleset's id is new on the pico, and a flush doesn't raise it, but the guard is cheap and covers any other path to this rule.) See [Initializing entity variables](/developing/initializing-entity-variables). ### Regular expressions and multi-line strings When an event expression captures an attribute value with a regular expression, only a single line of it is matched and kept. For a multi-line attribute, read it with `event:attr` in the prelude instead. ### The ruleset id and the file name By convention the file is named after the ruleset id, with `.krl` on the end, but nothing enforces it. If you don't follow the convention, know where each is used: - The **rid** (what's in `ruleset { … }`, and `meta:rid`) goes in query URLs (`/sky/query///…`) and in the `rid` attribute of wrangler's install request. - The **URL** ends with the whole file name, extension included, and is what the `url` attribute of an install request and the flush use. The engine enforces uniqueness of installed rulesets in a pico by rid, not by URL. ## See also - [Event Loop Introduction](/developing/event-loop): how the schedule works - [KRL manual](/krl/): the language reference - [Guard rules](/cookbook/guard_rules/): demo of the guard patterns here --- Ported from the [Pico Labs wiki](https://picolabs.atlassian.net/wiki/spaces/docs/pages/676397057), and updated for this engine. --- # Tip: initializing entity variables # Initializing entity variables Each ruleset installed in a pico gives the pico its own entity variables, specific to that pico and that ruleset. Until a rule's postlude first assigns one, its value is `null`. There are three ways to deal with that. ## Use `defaultsTo` wherever you read it `defaultsTo` supplies a value when the variable is `null`. Suppose `ent:owners` will be a map but might not exist yet: ```krl nameExists = function(ownername) { ent:owners.defaultsTo({}) >< ownername } ``` The first time the map is stored, start from the same default: ```krl rule owner_token { select when owner token_created pre { name = event:attr("name") new_owner = event:attr("owner") } fired { ent:owners := ent:owners.defaultsTo({}).put(name, new_owner) } } ``` With this technique, every use of `ent:owners` in the ruleset needs `defaultsTo`. A shared function that returns `ent:owners.defaultsTo({})` keeps that in one place. ## Initialize it in a guard rule Put a rule before the others that select the same event, and have it create the variable when it is missing: ```krl rule guard_owners_map { select when owner token_created if ent:owners.isnull() then noop() fired { ent:owners := {} } } ``` Rules in one ruleset that select the same event run in the order they are written, so this one runs first. The rules after it can then use a path assignment, which is also faster for a large map: ```krl rule owner_token { select when owner token_created pre { name = event:attr("name") new_owner = event:attr("owner") } fired { ent:owners{name} := new_owner } } ``` Functions still need `defaultsTo`, in case a query arrives before any event does. ## Initialize it when the ruleset is installed A rule that selects the ruleset's own `wrangler:ruleset_installed` runs before anything else can use the ruleset: ```krl rule initialization { select when wrangler ruleset_installed where event:attr("rids") >< meta:rid if ent:owners.isnull() then noop() fired { ent:owners := {} } } ``` The `where` clause matters. Many rulesets are installed in a pico over its life, and without it this rule would run every time. Checking `isnull()` first is good practice too. Installing again from the same URL, or flushing, does not raise `ruleset_installed` again. A rule like this runs once per install. See also: [Persistent variables](/krl/persistent-variables/) in the KRL manual, and the [Guard rules](/cookbook/guard_rules/) demo. *From the Pico Labs KRL Cookbook page "Initializing entity variables".* --- # Tip: arrays and maps # Arrays and maps KRL has no `for` loop in expressions. Array operators do the same work: `map`, `filter`, `reduce`, `sort`, and the others in [Array operators](/krl/krl-expression-language/operators/array-operators). Here are a few patterns that come up often. ## Loop with `reduce` To repeat a step a fixed number of times, `reduce` over a range and carry the state in the accumulator. This converts a number to an 8-byte array, high byte first, the way this JavaScript does: ```js for (var index = 0; index < byteArray.length; index++) { var byte = long & 0xff; byteArray[index] = byte; long = (long - byte) / 256; } ``` ```krl toByteArray = function(str) { 1.range(8) .reduce(function(a, i) { [a[0].append(a[1] % 256), math:int(a[1] / 256)] }, [[], str.as("Number")]) .head() .reverse() } ``` `1.range(8)` is `[1, 2, ... 8]`. It only counts; the function doesn't use `i`. The accumulator is a pair: the bytes so far, and what's left of the number. Each step appends the low byte and divides by 256. At the end, `head()` takes the bytes and `reverse()` puts the high byte first. `toByteArray("1551127401")` is `[0,0,0,0,92,116,83,105]`. Add `.klog("step")` inside the function to watch the accumulator change on each step. The inverse is one `reduce`. With no initial value, it starts from the first element: ```krl fromByteArray = function(bytes) { bytes.reduce(function(a, b) { a * 256 + b }) } ``` ## Flatten an array of arrays ```krl [[1, 2, 3], [4, 5, 6], [7, 8, 9]].reduce(function(a, b) { a.append(b) }, []) // [1, 2, 3, 4, 5, 6, 7, 8, 9] ``` ## Sort an array of maps `sort` takes a comparison function. A function that returns one, for a given key, reads well: ```krl by = function(key) { function(a, b) { a{key}.encode() cmp b{key}.encode() } } ``` `wrangler:channels().sort(by("id"))` reads "my channels, sorted by id". `cmp` compares strings, and `encode()` turns any value into a string, so `by` works whatever the values are. For numbers, compare them as numbers instead: `people.sort(function(a, b) { a{"born"} <=> b{"born"} })`. For a primary and a secondary key: ```krl by = function(key1, key2) { key2 => function(a, b) { a{key1}.encode() cmp b{key1}.encode() || a{key2}.encode() cmp b{key2}.encode() } | function(a, b) { a{key1}.encode() cmp b{key1}.encode() } } ``` `||` stops at the first comparison that isn't zero, so the second key only breaks ties. `by("last", "first")` sorts by last name, then first name, and `by` still works with one key. A key can also be a path, such as `by(["meta", "flushed"])`, because `a{["meta", "flushed"]}` looks up a nested value. *From the Pico Labs KRL Cookbook pages "Looping using the reduce operator", "Flatten an Array of Arrays", and "Sorting an Array of Maps".* --- # Tip: decoding form-encoded data # Decoding form-encoded data Form-encoded content, such as `q=5&value=%20space%20`, sometimes arrives as a single string: in a webhook body, a stored URL, or a query string you built. A chain of operators turns it into a map. ## Names and values ```krl nv_split = function(x) { x.extract(re#([^=]*)=?(.*)#) } // "n=v" -> ["n", "v"] frst = function(a) { a.head() } dup_1st = function(v) { v[0][1] } decodeContent = function(content) { content // "n0=v0&n1=v1&..." .split(re#&#) // ["n0=v0", "n1=v1", ...] .map(nv_split) // [["n0", "v0"], ["n1", "v1"], ...] .collect(frst) // {"n0": [["n0", "v0"]], "n1": [["n1", "v1"]], ...} .map(dup_1st) // {"n0": "v0", "n1": "v1", ...} } ``` `decodeContent("q=5&value=%20space%20")` is `{"q": "5", "value": "%20space%20"}`. - `split` makes an array of `name=value` strings. - `extract` matches each one against a regular expression and returns its capture groups, the name and the value. - `collect` groups the pairs into a map keyed by name. Each value is an array of every pair with that name. - The map operator `map` keeps the first value for each name. ## Keep every value of a repeated name When a name can repeat, keep all its values: ```krl dup_all = function(v) { v.length() == 1 => v[0][1] | v.map(function(w) { w[1] }) } ``` Use `.map(dup_all)` in place of `.map(dup_1st)`. `a=1&a=2&b=3` becomes `{"a": ["1", "2"], "b": "3"}`. ## Decode `+` and `%xx` `replace` accepts a function as its second argument. It is called with the match and then each capture group, and returns the replacement: ```krl hexDigits = "0123456789abcdef" hex2dec = function(hex) { hex like re#^[0-9A-Fa-f]+$# => hex.lc().split("").reduce(function(a, h) { a * 16 + hexDigits.split("").index(h) }, 0) | null } hex2char = function(encoded, hex) { val = hex2dec(hex) val && val < 128 => val.chr() | encoded } decode_val = function(s) { s.replace(re#[+]#g, " ").replace(re#%([0-9A-Fa-f]{2})#g, hex2char) } ``` Replace `+` first: early encoders used it for a space, and a real plus sign arrives as `%2B`. Then use `function(v) { decode_val(v[0][1]) }` as the last step of the chain. `value=%20space%20&plus=a%2Bb&pct=100%25` becomes `{"value": " space ", "plus": "a+b", "pct": "100%"}`. This handles ASCII characters. Anything above 127 is left encoded. Events and queries that arrive over HTTP are already decoded: the engine gives you their attributes as a map. This is for strings you have in hand. *From the Pico Labs KRL Cookbook page "Decoding WWW-Form Encoded Content or Query Parameters".* --- # Tip: importing tables and CSV # Importing tables and CSV Data exported from a spreadsheet or another system usually arrives as text: tab-separated (TSV) or comma-separated (CSV) lines. If the other system can export JSON, use that instead. `decode()` turns a JSON string into KRL data in one step. ## Lines Exports end lines with `\n` or, from Windows, `\r\n`. One regular expression handles both: ```krl lineBreak = re#\r?\n# lines = content.split(lineBreak).filter(function(s) { s }) ``` `filter(function(s) { s })` drops empty lines: an empty string is falsy, and there is usually one after the last line break. Another way is to let a global regular expression find the lines: `content.extract(re#(.+)#g)`. ## A table to an array of maps When the first row names the columns, make each data row a map: ```krl tsv = function(content) { lines = content.split(re#\r?\n#).filter(function(s) { s }) keys = lines.head().split(re#\t#) lines.tail().map(function(line) { values = line.split(re#\t#) keys.reduce(function(m, k, i) { m.put(k, values[i]) }, {}) }) } ``` Given ``` A B 1 2 3 4 ``` `tsv(content)` is `[{"A": "1", "B": "2"}, {"A": "3", "B": "4"}]`. The `reduce` passes each key's index as its third argument, which picks the matching value. ## CSV with a known shape CSV is harder, because a value that contains a comma is wrapped in double quotes. When you know what each line looks like, match the whole line with a regular expression and let the last group take the rest: ```krl graduands = function(content) { content.extract(re#(.+)#g) .tail() .map(function(line) { line.extract(re#^([\w ]+),(\w+),(\d+),(\d+),(.*)$#) }) } ``` `Ada Lovelace,AL,1,1843,"Notes, on the engine"` becomes `["Ada Lovelace", "AL", "1", "1843", "\"Notes, on the engine\""]`. Strip the quotes from that last field if you need to. `tail()` skips the header line. ## Fetch the file Rather than passing a whole file to a function, give it a URL: ```krl import = function(url) { tsv(http:get(url){"content"}) } ``` A function that calls `http:get` runs on every query, so for a large file, import it once in a rule and store the result in an entity variable. *From the Pico Labs KRL Cookbook pages "Converting a table to an Array of Maps" and "Importing data from other systems".* --- # Tip: converting time formats # Converting time formats The [time library](/krl/libraries/time) works with ISO 8601 / RFC 3339 times such as `2011-09-21T15:02:01-07:00`. Other formats need converting first. RSS feeds, for example, give dates like `Wed, 21 Sep 2011 15:02:01 -0700` (RFC 2822), and `time:new` rejects them: `time:new was given an invalid date string`. This function rearranges the parts: ```krl mon2num = { "Jan": "01", "Feb": "02", "Mar": "03", "Apr": "04", "May": "05", "Jun": "06", "Jul": "07", "Aug": "08", "Sep": "09", "Oct": "10", "Nov": "11", "Dec": "12" } rss2krlTime = function(pubDate) { tarr = pubDate.split(re# #) tzoff = tarr[5].replace(re#(\d\d)$#, ":$1") tarr[3] + "-" + mon2num{tarr[2]} + "-" + tarr[1] + "T" + tarr[4] + tzoff } ``` `split` gives `["Wed,", "21", "Sep", "2011", "15:02:01", "-0700"]`. The offset `-0700` becomes `-07:00`, and `+0530` becomes `+05:30`. Then the time library can format it: ```krl time:strftime(rss2krlTime("Wed, 21 Sep 2011 15:02:01 -0700"), "%b %d %Y") // "Sep 21 2011" ``` This expects a two-digit day, as RSS feeds use. For a feed that writes `1 Sep`, pad the day first, for example with `tarr[1].length() == 1 => "0" + tarr[1] | tarr[1]`. *From the Pico Labs KRL Cookbook page "Converting Time Formats" (originally by Ed Orcutt). The time-zone handling here also covers offsets that aren't whole hours.* --- # Tip: what's in a name # What's in a name? KRL keeps many kinds of names apart: the ruleset id, rule names, event domains and types, attribute names, global and local names, function parameters, and directive names. As an exercise, here is a ruleset where every one of them is `name`: ```krl ruleset name { meta { shares name } global { /** * @param name String */ name = function(name) { name.klog("name") => name | "name" } } rule name { select when name name pre { name = event:attr("name").defaultsTo(name(name), "name") } send_directive("name", {"name": name || "name"}) } rule name_provided { select when name name name re#(.+)# setting(name) send_directive("name", {"name": name}) } } ``` It works. Most of the uses can't collide: an event domain, a rule name, and a directive name live in different places. One line is worth a closer look: ```krl name = event:attr("name").defaultsTo(name(name), "name") ``` The right-hand side is evaluated before the binding. At that moment `name` is still the global function, so `name(name)` calls the function with itself as the argument, and it returns itself, because a function is truthy. After the line, `name` is bound to the result, and the global function is shadowed for the rest of the rule. Raise `name:name` with `name` set to `Ada` and both rules answer `{"name": "Ada"}`. Raise it without the attribute and `defaultsTo` uses the function (logging `"name"`). The directive's `name` is then a function, which doesn't survive conversion to JSON, so the reply shows `{}`. So: a name declared in `pre` hides a global of the same name for the rest of that rule. That is legal, and usually a mistake. And a name in `pre` can't be declared twice. KRL declarations are not assignments. *From the Pico Labs KRL Cookbook page "What's in a name?".* --- # Tip: cookies and redirects # Cookies and redirects Two directive names are special. When a rule sends one in response to an event raised over HTTP, the engine turns it into part of the HTTP response. ## Set a cookie: `_cookie` ```krl rule set_cookie { select when cookie needed pre { eci = wrangler:channels(["wellKnown_Rx", "Tx_Rx"]).head(){"id"} || meta:eci } send_directive("_cookie", {"cookie": <>}) } ``` The engine adds `Set-Cookie: wellKnown_Rx=; Path=/sky` to the response, and the browser sends the cookie back with later requests to the same engine. This one shares the pico's well-known channel, so other picos reached from the same browser could offer it a relationship. The ruleset uses `io.picolabs.wrangler` as a module. ## Read one cookie `_headers` does not include the `Cookie` header. Read one cookie by name: ```krl eci = http:cookie("wellKnown_Rx") ``` That is `"abc123"` when the browser sent `Cookie: wellKnown_Rx=abc123`. A cookie that was not sent is `null`. The engine's own session and ceremony cookies are not available to rulesets. `_cookie` and `_redirect` are unchanged: they still set the response. ## Redirect: `_redirect` ```krl send_directive("_redirect", {"url": "https://picolabs.io/"}) ``` The engine answers the event with HTTP 302 and that URL as the `Location`. ## A URL shortener Together with a channel per link, `_redirect` makes a URL shortener. Each link gets its own channel, whose policy allows only the event that follows it. The channel id is the short URL: ```krl ruleset s { meta { use module io.picolabs.wrangler alias wrangler provides u } global { u = function(eci) { <<#{meta:host}/sky/event/#{eci}/none/s/u>> } } rule recordShortcut { select when s u url re#(.+)# setting(url) pre { tag = event:attr("tag").lc().replace(re#[^a-z0-9_.-]#g, "-") eid = tag => tag | "none" } every { wrangler:createChannel( [eid, "s"], {"allow": [{"domain": "s", "name": "u"}], "deny": []}, {"allow": [], "deny": [{"rid": "*", "name": "*"}]} ) setting(channel) send_directive("shortcut registered", {"tag": tag, "eci": channel{"id"}, "shortcut": u(channel{"id"}), "url": url}) } fired { ent:shortcuts{channel{"id"}} := url } } rule redirectShortcut { select when s u where event:attr("url").isnull() send_directive("_redirect", {"url": ent:shortcuts{meta:eci}}) } } ``` Another ruleset uses it as a module. It registers a link with `raise s event "u" attributes {"url": long_url, "tag": "docs"}`, then finds the channel with `wrangler:channels(["docs", "s"]).head()` and the short form with `s:u(eci)`. Opening the short URL redirects to the long one. The short URL can only raise `s:u`, and queries on it are all denied. Short links are handy in QR codes, which get smaller as the URL does. *From the Pico Labs KRL Cookbook pages "Setting and reading cookies" and "Ruleset that provides a general purpose URL shortener".* --- # Webhook signatures # Webhook signatures A webhook such as Square's is an HTTP event. The sender signs the exact bytes of the body. The parsed attributes are not those bytes: a space or the order of keys would change the signature. The channel has to keep the raw body, and the comparison has to take the same time whether or not it matches. ## Keep the raw body On the Channels tab, check **Keep the raw request body**. From KRL, pass `rawBody: true` when the channel is created: ```krl raise wrangler event "new_channel_request" attributes { "tags": ["square", "webhook"], "eventPolicy": { "allow": [{ "domain": "square", "name": "webhook" }], "deny": [] }, "queryPolicy": { "allow": [], "deny": [] }, "rawBody": true } ``` `event:raw_body()` then returns the body as a string. On a channel without that setting it returns `null`, and so does `event:send` or a scheduled event. The body is not an attribute. Storing `event:attrs` does not store it. A rule that raises another event in the same transaction can still read it. ## The signing key Keep the key in the ruleset's own PDS settings and mark it secret. `settings` then returns `{ "secret": true, "set": true }`. The ruleset reads the key with `pds:config_value("signatureKey")`. See [PDS settings](/wrangler/pds). ## Square Square's `x-square-hmacsha256-signature` header is the base64 HMAC-SHA256 of the notification URL concatenated with the raw body, using the webhook signature key. The URL is the one configured in Square, exactly, not whatever host the request happened to arrive on. Header names on the event are lowercase. ```krl ruleset com.example.square_webhook { meta { use module io.picolabs.pds alias pds shares verdict } global { notification_url = "https://example.com/sky/event//none/square/webhook" verdict = function() { ent:verdict || "none" } } rule verify { select when square webhook pre { key = pds:config_value("signatureKey") body = event:raw_body() given = event:attr("_headers"){"x-square-hmacsha256-signature"}.defaultsTo("") ready = body.typeof() == "String" && key.typeof() == "String" expected = ready => math:hmac("sha256", key, notification_url + body, "base64") | "" ok = ready && math:timing_safe_equal(expected, given) } noop() fired { ent:verdict := ok => "accepted" | "rejected" } } } ``` `math:timing_safe_equal` compares the two strings without stopping at the first different byte. Use it instead of `==` for the signature. `math:hmac` is described in the [math library](/krl/libraries/math). --- # AI tools over MCP # AI tools over MCP An MCP client such as Claude can call a pico the same way it calls any remote MCP server: add a URL, sign in with a passkey, approve, and use the tools. The engine also has a separate local MCP server for coding agents on the same machine. That one is `pico-engine mcp --stdio` and does not use this sign-in. This page is the remote connection. ## Which URL | You want | URL | Tools | |----------|-----|--------| | One channel | `https:///mcp/` | That channel's events and queries, one tool each | | The mesh under a channel | `https:///mcp/mesh/` | `pico_tree`, `pico_channels`, `query`, `raise_event` | No trailing slash. An ECI is a channel id from the Channels tab. `mesh:read` covers schemas, lists, and queries. `mesh:act` covers events. `mesh:admin` is reserved and unlocks nothing. A sign-in that does not name a narrower scope receives both `mesh:read` and `mesh:act`. A channel is not exposed until you allow it. On the Channels tab, check the channel surface for `/mcp/`, or the mesh surface for a channel that should appear in `/mcp/mesh/…`. The channel you name in the mesh URL does not itself need the mesh checkbox. A missing or empty policy denies both surfaces. A pico ceiling from an older engine does not block this. A ceiling that has its own `mcpPolicy` does. On a channel endpoint, that channel's event and query policies decide which calls succeed. The tools are the events and queries those policies allow. A mesh endpoint is not bound to the policies of the channel in the URL. `pico_tree` and `pico_channels` list the subtree. `query` and `raise_event` name some other channel, and that channel's event and query policies are what allow or refuse the call. `pico_channels` includes a channel only when its MCP policy allows the mesh surface. ## The root has to allow OAuth Install `io.picolabs.oauth` on the **root** pico before anyone signs in. The Engine KRL Directory lists it only on the root. On the Rulesets tab the URL is `https:///krl/io.picolabs.oauth.krl`. That ruleset is the switch for issuing sign-in tokens. A connector for one channel and a connector for the whole mesh both require it. The channel URL still limits the token to that channel. Installing the ruleset also makes external `/sky/*` calls on the mesh require a bearer token. Channels tagged `mesh-oauth-exempt`, or `didcomm` and `ingress`, stay open. See [OAuth](/wrangler/oauth). A channel tagged `oauth-webhook` can obtain a client-credentials token for its own MCP URL without this ruleset. That is a machine credential, not the passkey sign-in below. ## Claude cannot use localhost Claude connects from Anthropic's network, including from Claude Desktop. The connector URL must be HTTPS on port 443. `http://localhost:3000` and a URL with another port are refused. Keep the engine on port 3000 and put a tunnel in front of it. A Cloudflare quick tunnel does not need an account: ```sh brew install cloudflared cloudflared tunnel --url http://localhost:3000 ``` The public URL is in the box near the top of the output, `https://….trycloudflare.com`. A line about a missing `config.yml` is normal. Leave that process running. The hostname changes if you stop it and start it again. Anyone with the URL can reach the engine, so stop the tunnel when you are done. Quick tunnels do not carry server-sent events. If sign-in succeeds and the tools then fail to stay connected, use `ngrok http 3000` instead. A named Cloudflare tunnel, with a hostname you choose, needs a free Cloudflare account and a domain on Cloudflare. ## Set the host before you create the account A passkey works only for the host it was created on. A passkey for `localhost` will not sign in on the tunnel host, and a passkey for one quick-tunnel hostname will not sign in on the next one. Channels do not store the host. The account that owns them does. Create the passkey, then the channel, on the host Claude will use. 1. Start the tunnel and copy the hostname. 2. Point the engine at that host and restart it. In the config file (`~/.config/pico-engine/config.json`, or the file in `PICO_ENGINE_CONFIG`): ```json { "baseUrl": "https://example.trycloudflare.com", "allowSelfSignup": true, "webauthn": { "rpId": "example.trycloudflare.com", "origin": "https://example.trycloudflare.com", "rpName": "Pico Engine" } } ``` `baseUrl` and `origin` have no trailing slash and no port. `rpId` is the hostname only. `allowSelfSignup` can go back to `false` after the account exists. The engine reads this file at startup. 3. Open the tunnel URL in a browser and register a passkey. 4. Install `io.picolabs.oauth` on that root. 5. Open the Channels tab, allow the channel MCP surface on the channel the agent should use, and copy its ECI. The connector URL is then `https://example.trycloudflare.com/mcp/`. ## Add the connector in Claude On a Pro or Max plan: **Customize** → **Connectors** → **+ Add** → **Add custom connector**. On a Team or Enterprise plan an owner adds it under **Organization settings** → **Connectors** → **Add** → **Custom** → **Web**. Each person then connects it from **Customize** → **Connectors**. - Name it whatever you like. - Remote MCP server URL: the channel or mesh URL from above, exactly. - Authentication: **Sign in now**. - OAuth client: **Use Claude's published identity**. Leave the client id and secret empty. **Register automatically** is dynamic client registration. This engine does not offer it. **Use your own OAuth client** is an `app_…` client you create on the engine first. The "Detected" label means Claude found an OAuth server, not that a client id is required. If Claude says automatic client registration is not supported, or the connector page says the connection stopped working and never opens the passkey page, remove the connector and add it again with **Use Claude's published identity**. **Reconnect** repeats the registration attempt Claude already saved. The passkey page is the tunnel host, not `localhost`. Sign in, then approve. The approval screen names Claude, its host, the mesh, and the scopes. After that, turn the connector on from the **+** menu in a chat. Ask it to run a query (`mesh:read`) or raise an event (`mesh:act`). A public client's access token lasts one hour. Each refresh returns a new refresh token. Sending the previous refresh token again revokes that grant. ## What another MCP client needs A client that implements the MCP authorization spec can use the same URLs. A request with no bearer token gets `401` and a `WWW-Authenticate` header pointing at the protected-resource document. | Document | URL | |----------|-----| | Protected resource, one channel | `/.well-known/oauth-protected-resource/mcp/` | | Protected resource, mesh | `/.well-known/oauth-protected-resource/mcp/mesh/` | | Authorization server | `/.well-known/oauth-authorization-server` | The same authorization-server document is also served with the MCP path inserted, and at `/.well-known/openid-configuration`, because some clients look there first. PKCE is S256 only. A `client_id` that is an `https` URL is fetched as a Client ID Metadata Document. The token endpoint accepts a public client (`none`) and a confidential client that sends its secret in the body (`client_secret_post`). The token's `resource` is the MCP URL it was issued for. A token for one channel is rejected on another channel and on `/sky/*`. --- # Tip: events from email # Events from email Any service that can call a webhook can raise events on a pico. Email is a good example: services such as [CloudMailin](https://www.cloudmailin.com/) give you an address and POST each message that arrives to a URL you choose. Point that URL at a pico: ``` https:///sky/event//mail/mail/received ``` The engine has to be reachable from the internet. For an engine on your own machine, a tunnel such as [ngrok](https://ngrok.com/) gives it a public address. Use a channel made for this purpose, whose policy allows only `mail:received`, so the webhook URL can't do anything else. ## Parse the message The service's POST body becomes the event's attributes. With CloudMailin's JSON format, the headers and the plain-text body are in `headers` and `plain`: ```krl rule mail_incoming { select when mail received pre { headers = event:attr("headers").defaultsTo({}) } always { raise mail event "parsed" attributes { "date": headers{"Date"}, "to": headers{"To"}, "from": headers{"From"}, "subject": headers{"Subject"}, "text": event:attr("plain") } } } ``` ## Route it to another pico One mail pico can serve many. Put the target pico's channel id in the address, as in `me+@cloudmailin.net`, and forward on it: ```krl rule mail_router { select when mail parsed to re#[+]([^@]*)@# setting(eci) event:send({"eci": eci, "domain": "mail", "type": "events", "attrs": event:attrs}) } ``` The person writing the email needs that address, but not where the target pico lives. ## Turn lines into events The target pico can treat each line of the body as an event to raise. Here, a line such as `aurora/on` raises `aurora:on`, and every other line is ignored: ```krl rule mail_events { select when mail events foreach event:attr("text").defaultsTo("").split(re#\r?\n#) setting(request) pre { parts = request.split(re#/#) wanted = parts.length() == 2 && parts[0] == "aurora" } if wanted then noop() fired { raise aurora event parts[1] } } ``` A body of ``` Turn on the lights, then show the second theme after this one. aurora/on aurora/next aurora/next ``` raises `aurora:on` and `aurora:next` twice, which the pico's existing rulesets handle. The new ruleset doesn't change them. Parsing the body is the fragile part. Email clients add things you don't expect, such as invisible characters. Test with messages from the clients your senders use. *From the Pico Labs KRL Cookbook page "Triggering Events from Email Messages".* --- # Tip: reading the engine's log files # Reading the engine's log files The **Logging** tab shows recent events and queries for a pico, from the engine's memory. To look further back, or across every pico, read the log files. ## Where they are The engine writes one JSON object per line to `pico-engine.log` in its home directory (`PICO_ENGINE_HOME`, `~/.pico-engine` by default). When the file reaches 100 MB it rotates: the older entries move to `pico-engine.log.1`, `pico-engine.log.2`, and so on, and the engine keeps up to twelve files. `PICO_ENGINE_LOG_FILE` writes somewhere else instead (`stdout` to write to standard output). See [the operator guide](/pico-engine) for the settings. Each line has `level`, `time`, and `msg`, and lines about a pico's work have `picoId` and `txnId`. A transaction is one event or query and everything it caused. ```json {"level":50,"time":"2026-11-13T18:46:16.317Z","picoId":"cmwfbv2y1001e4jysd0cn9w05","txnId":"cmwfbvcvx00244jysd16w7grz","txnKind":"query","error":"This is a family channel and only the owner can use it.","msg":"..."} ``` ## One pico's history Find the pico's id on its **About** tab, then: ```sh cd ~/.pico-engine grep -h '"picoId":""' pico-engine.log* | jq -s -c 'sort_by(.time)[]' > my_pico.log ``` `grep` collects the lines from every file, and [`jq`](https://jqlang.org/) sorts them by time, since rotated files aren't in order relative to each other. To follow one event through everything it caused, filter on its `txnId` instead. ## What to look for - `"msg":"txnQueued"` lines show each event or query as it arrived, with its attributes. - `"error"` fields show what failed: a channel policy refusal, a KRL runtime error, a family channel used from outside. - `klog` output from your own rules is in the log, with the message you gave it. The logs hold every event's attributes. Anything secret that arrives in an event is in them. See the [Configuration and secrets](/cookbook/secrets/) demo. *From the Pico Labs KRL Cookbook page "Examining pico-engine log files". The log format and rotation have changed since that page was written.* --- # Tip: querying another pico # Querying another pico To call a function another pico's ruleset shares, use `wrangler:picoQuery`. Two things are needed: a channel to the other pico whose query policy allows that function, and `io.picolabs.wrangler` as a module. ```krl meta { use module io.picolabs.wrangler alias wrangler } ``` ```krl rule ask_the_other_pico { select when example question_asked pre { answer = wrangler:picoQuery(other_eci, "my.ruleset.id", "myFunction", {"arg1": "a", "arg2": "b"}) } if answer{"error"}.isnull() then noop() fired { ent:last_answer := answer } } ``` On the same engine, `picoQuery` calls the other pico directly. On another engine, or for a `did:` identifier, it goes over HTTP or DIDComm. Either way it returns the function's value, or a map with an `error` key if something went wrong. ## Which channel - **Your children.** `wrangler:children()` gives each child's family channel (`eci`). `picoQuery` can use it, because the parent owns it. - **Picos you have a relationship with.** Use the relationship: `subscription:queryOnSub(Id, rid, name, args)` (see the [Two picos talking](/cookbook/two_picos/) demo), or `picoQuery` on the relationship's `Tx` channel. - **Anyone else.** A channel they give you, made for the purpose, with a policy that allows the functions you need. ## Don't query yourself A pico handles one thing at a time. A query from a pico to itself would wait behind the work that's making it. `picoQuery` refuses: it returns `{"status_code": 400, ...}` with the error "Pico should not query itself". To use a function in another ruleset on the same pico, use that ruleset as a module instead. ## With `http:get` A pico has a URL, so `http:get` works too: ```krl response = http:get(meta:host + "/sky/cloud/" + other_eci + "/my.ruleset.id/myFunction", {"arg1": "a"}) answer = response{"content"}.decode() ``` It's slower than `picoQuery` on the same engine, and it can't use family channels: the answer is an error, "This is a family channel and only the owner can use it". The HTTP status is still 200 in that case, so check the content for `error`, not just `status_code`. *From the Pico Labs KRL Cookbook page "Accessing a Function Shared by Another Pico".* --- # Tip: accepting relationship requests # Accepting relationship requests When another pico asks for a relationship, the request waits on the receiving pico (its **Relationships** tab lists it) until something approves it. Here are three ways to approve requests automatically. The examples share one setup: a list pico asks participant picos to join it, in the role `participant list` for itself and `participant` for them. ## A rule that approves On each participant: ```krl rule accept_list { select when wrangler inbound_pending_subscription_added where event:attr("Rx_role") == "participant" && event:attr("Tx_role") == "participant list" always { raise wrangler event "pending_subscription_approval" attributes event:attrs } } ``` Passing the event's attributes along tells wrangler which request to approve. Roles arrive from the receiving pico's point of view: `Rx_role` is this pico's, and `Tx_role` is the other pico's. Don't approve every request. A rule without a `where` accepts a relationship with any pico that has your well-known channel. Accept only what you expect: the right roles, a request from your parent, a request that carries a shared secret. To refuse, raise `wrangler:inbound_rejection` with the same attributes. The [Collection](/cookbook/collection/) demo does both, and remembers whom it turned away. ## Wrangler's auto-accept configuration Without writing a rule, give the participant a pattern to accept: ```krl raise wrangler event "autoAcceptConfigUpdate" attributes { "config": {"configName": "lists", "entries": {"Tx_role": ["^participant list$"]}} } ``` Each entry maps an attribute of the request, as the participant sees it, to a list of regular expressions. A request that matches any of them is approved. Send the same event with `"delete": true` and the `configName` to remove it. `autoAcceptConfig()`, shared by `io.picolabs.subscription`, shows what is set. You can also send this event from the pico's **Testing** tab. ## Accept later, by installing a ruleset If a request arrived before there was anything to approve it, install a ruleset that approves it and then removes itself: ```krl ruleset acceptance { meta { use module io.picolabs.subscription alias subs } rule initialize { select when wrangler ruleset_installed where event:attr("rids") >< meta:rid pre { s = subs:inbound("Tx_role", "participant list") .filter(function(i) { i{"Rx_role"} == "participant" }) .head() } if s then noop() fired { raise wrangler event "pending_subscription_approval" attributes s } finally { raise wrangler event "uninstall_ruleset_request" attributes {"rid": meta:rid} } } } ``` It looks for a pending request from a participant list, approves it, and uninstalls itself either way. This is handy when a parent asked many children to join before they could accept: install it on each child. *From the Pico Labs KRL Cookbook pages "Automatic Replies to Subscription Requests" and "Accepting a subscription offer by installing a ruleset".* --- # Tip: one pico per resource # One pico per resource Some things should be used by only one party at a time: a motor, a GPIO pin, a printer, an account with an outside service. Give each one its own pico. Everything else asks that pico, by event, to use the resource. Because a pico handles one event at a time, its rulesets are the only code that touches the resource, and requests can't collide. A Pico Labs rover project did this with its wheels. Each of four wheel picos kept the index of the one motor it drove, and only its rules called the motor driver: ```krl rule set_motor_index { select when rover motor_index_assigned always { ent:motor_index := event:attr("motor_index") } } ``` A controller pico had a channel to each wheel. To move, it sent each wheel the direction that wheel should turn: ```krl to_wheels = defaction(type, fr, fl, br, bl) { every { event:send({"eci": ent:wheels{"FR"}, "domain": "rover", "type": type, "attrs": fr}) event:send({"eci": ent:wheels{"FL"}, "domain": "rover", "type": type, "attrs": fl}) event:send({"eci": ent:wheels{"BR"}, "domain": "rover", "type": type, "attrs": br}) event:send({"eci": ent:wheels{"BL"}, "domain": "rover", "type": type, "attrs": bl}) } } forward = defaction() { to_wheels("run", {"direction": "fwd"}, {"direction": "back"}, {"direction": "back"}, {"direction": "fwd"}) } ``` The controller could have driven the motors itself. But then every new pico that needed the wheels would need the motor code too, and two of them could drive the same motor at once. With a pico per wheel, any number of controllers can share the wheels safely. Keep the channels to the resource picos in relationships (or, for children, in their family channels), not in ECIs written into the code. The original rover hard-coded them, and noted that relationships would be much better. *From the Pico Labs KRL Cookbook page "Singleton Pattern".* --- # Pico engine operator guide # pico-engine — operator guide This document is the **canonical operator reference** for running and configuring the pico engine. It lives in the repo's top-level `docs/`, is published at , and is indexed in [`llms.txt`](llms.txt). **New to picos?** Read [pico-overview.md](pico-overview.md) — what picos are, why they exist, and the event-query model (`docs/design/pico-overview.md` in the repo). ## Documentation source of truth | Location | Role | |----------|------| | **This file** (`docs/pico-engine.md`) | Authoritative for install, config, security flags, and deployment. Versioned with the engine; bundled into the npm package for the dev MCP server, so it's what agents read. | | **README** in the same package | Quick start, CLI overview, OAuth, containers — links here for config details. | | **`MEMORY.md`** (engine repo) | Maintainer log: rationale, gotchas, env registry — not end-user docs. | | **Confluence / wiki** | Optional mirror for Picolab internal use. **Do not treat Confluence as primary:** edit this file in git first, then update Confluence to match (or replace Confluence pages with a short blurb + link to [`docs/pico-engine.md`](https://github.com/Picolab/pico-engine/blob/master/docs/pico-engine.md) on `master` or the release tag). | If Confluence and this guide disagree, **trust the repo at the release you run** (`pico-engine --version`). --- ## Quick start ```sh npm install -g pico-engine pico-engine ``` Open `http://localhost:3000`. First visit registers a passkey and creates the root pico. Copy the example config and adjust paths: ```sh mkdir -p ~/.config/pico-engine cp "$(npm root -g)/pico-engine/config.example.json" ~/.config/pico-engine/config.json # or from a git checkout: packages/pico-engine/config.example.json pico-engine ``` Show which config file the process will use: ```sh pico-engine config path ``` Alternate config file (one file **per engine process**): ```sh pico-engine -c /path/to/engine-a.json # or PICO_ENGINE_CONFIG=/path/to/engine-a.json pico-engine ``` --- ## Config file vs data directory | | Config | Data (`home`) | |---|--------|----------------| | **Default path** | `$XDG_CONFIG_HOME/pico-engine/config.json` or `~/.config/pico-engine/config.json` | `~/.pico-engine/` | | **Override** | `PICO_ENGINE_CONFIG`, `pico-engine -c` | config JSON `home`, or `PICO_ENGINE_HOME` | | **Contains** | Port, public URL, WebAuthn, security flags, policy ceiling | LevelDB mesh (`db/`), ruleset cache, logs | | **Delete to reset** | No — keeps operator settings | Yes — **fresh mesh** (picos, passkeys in DB gone) | **Precedence:** defaults → **environment variables** → **config file** → programmatic overrides (tests). If both env and file set `port`, **the file wins**. In **`NODE_ENV=test`**, the config file is ignored unless `PICO_ENGINE_CONFIG` is set (isolated tests). --- ## Config JSON fields Example templates: [`config.example.json`](../../config.example.json), [`config.policy-ceiling.example.json`](../../config.policy-ceiling.example.json), [`config.instances.example/`](../../config.instances.example/). | Field | Type | Description | |-------|------|-------------| | `home` | string | Data directory (supports `~/…`). Default `~/.pico-engine`. | | `port` | number | HTTP listen port. Default `3000`. | | `baseUrl` | string | Public origin (no trailing slash): WebAuthn, did:webvh, links. Default `http://localhost:3000`. | | `allowSelfSignup` | boolean | Open registration after bootstrap. Default `false` (invite-only). | | `allowLocalhostChannelBypass` | boolean | Allow `/c/*`, `/api/dev/local`, and `/api/try-it` from a **direct loopback** socket without a passkey or the internal token. Default `false`. A request with `X-Forwarded-For`, `X-Real-IP`, or `Forwarded` is never treated as local. The dev MCP uses `$PICO_ENGINE_HOME/internal-token` instead. | | `wildcardSkyRequireBearer` | boolean | Require OAuth bearer on `/sky/*` for channels with unrestricted `*.*` allow rules. Default `false` (1.8 opt-in hardening). | | `logToStdout` | boolean | Log to stdout instead of a file. Use on Fargate (set `logToStdout: true` in config). | | `logFile` | string | Absolute log path when `logToStdout` is false. Values `stdout`, `-`, or `0` force stdout. | | `logMirror` | boolean | When **`logToStdout`** is true: also write structured JSON lines to **`$PICO_ENGINE_HOME/pico-engine.log`** for operators. Default **`true`**. Set **`false`** on EFS + Fargate to avoid mirror-file churn (then use CloudWatch from stdout, or set **`logFile`** to a writable path). The Developer UI **Logging** tab reads an in-memory ring and does not need this file. **Env:** `PICO_ENGINE_LOG_MIRROR` (`true`/`1` / `false`/`0`). | | `show_tryit_menu` | string | Where the Developer UI shows the **Try It!** tab: **`off`** (nowhere, and `/api/try-it` lists no demos), **`root_only`** (the root pico only), or **`all`** (every pico). Default **`root_only`**. **Env:** `PICO_ENGINE_SHOW_TRYIT_MENU`. | | `defaultRootPolicyCeiling` | object | Max channel policy for new root picos (`eventPolicy` / `queryPolicy`). See policy example file. | | `globalEventDeny` | array | Extra deny rules applied on **every** ECI before channel policy (e.g. block `engine:secret_ping`). | | `globalQueryDeny` | array | Same for queries. | | `webauthn.rpId` | string | WebAuthn RP ID (usually hostname without port). | | `webauthn.origin` | string | Must match browser URL (scheme + host + port). | | `webauthn.rpName` | string | Display name in passkey UI. | Legacy top-level `rpID`, `rpName`, `origin` are still accepted if `webauthn` is omitted. --- ## Environment variables These env overrides cover config path, data dir, port, public URL, and a few logging knobs; everything else (WebAuthn, signup, localhost bypass, policy ceiling, …) belongs in the **config file**. Maintainer registry of all env keys: engine repo `MEMORY.md` (Engine environment variables). | Variable | Purpose | |----------|---------| | `PICO_ENGINE_CONFIG` | Path to config JSON | | `PICO_ENGINE_HOME` | Data directory | | `PORT` | Listen port (if not in file) | | `PICO_ENGINE_BASE_URL` | Public URL | | `PICO_ENGINE_LOG_FILE` | Log file path, or `stdout` / `-` / `0` / empty for stdout-only | | `PICO_ENGINE_LOG_MIRROR` | When stdout-only: `false` / `0` disables writing `$PICO_ENGINE_HOME/pico-engine.log` (config: **`logMirror`**). The Logging tab does not read that file | | `PICO_ENGINE_LOG_RING` | How many recent transactions per pico the Logging tab keeps in memory. Default **200**. A restart clears the ring | | `PICO_ENGINE_SHOW_TRYIT_MENU` | `off`, `root_only` (default), or `all`: where the Developer UI shows the **Try It!** tab (config: **`show_tryit_menu`**) | | `PICO_ENGINE_DEMOS_DIR` | A directory to serve as `demos/` (at `/krl/demos/`) instead of the copy shipped with the engine, such as the `demos/` folder of a pico-engine checkout you're editing | --- ## Multiple engines on one host Give each process its own config with distinct **`home`**, **`port`**, **`baseUrl`**, and matching **`webauthn.origin`**. ```sh pico-engine -c ~/.config/pico-engine/engine-a.json # terminal 1 pico-engine -c ~/.config/pico-engine/engine-b.json # terminal 2 ``` Cursor MCP: separate `mcpServers` entries, each with its own `PICO_ENGINE_CONFIG` pointing at the engine you want. Docker: mount **one config file per service** and **one data volume per service** — never share a `home` volume between two running tasks. --- ## Docker and AWS (summary) Mesh state lives under `home`: | Subdir | Purpose | |--------|---------| | `db/` | Picos, channels, auth, identity | | `rulesets-db/` | Cached ruleset metadata (rebuildable) | | `rulesets/` | Compiled ruleset JS | - One writer per data directory (LevelDB). - **Fargate:** mount EFS at e.g. `/var/pico-image`, set `home` accordingly, `baseUrl` to public **HTTPS**, `logToStdout: true` in config. Set **`logMirror: false`** if you do not want the engine writing `pico-engine.log` on EFS (stdout → CloudWatch only). - `allowLocalhostChannelBypass` defaults to **false**. Leave it false on public deployments. The dev MCP reads `internal-token` from the engine data directory (on Docker, the host path of the `home` volume). Detailed checklists: package **README** § Containers / Fargate. --- ## Wrangler and bundled rulesets **Wrangler** (`io.picolabs.wrangler`) is the pico **OS** — the **preferred layer** for operating on picos. It wraps engine primitives (persistence, channels, ruleset registry, policy) in stable **`wrangler:*` events and shared queries**: children, channels, ruleset install/flush, relationships, discovery, and schema (`picoSchema`, `rulesetSchema`, `channelSchema`). It is installed on every pico with **PDS**, **subscription**, and **pico-engine-ui**. **Use wrangler, not engine escape hatches.** Application rulesets, integrators, and harnesses should raise **`wrangler` events** and call **`wrangler:*` functions** (or the UI / `/sky/*` / mesh MCP surfaces that send those operations). The engine also exposes lower-level **`ctx:*`** actions and owner-only tooling; those exist for wrangler and the platform rulesets to call, not as the everyday API. If something you need **cannot be done through wrangler**: 1. **Red flag — you may be out of bounds.** Lifecycle and mesh structure belong in wrangler (or in a domain framework such as Manifold built *on* wrangler), not in ad hoc `ctx:` calls or custom engine hooks from app KRL. 2. **Possible bug or missing OS feature.** If the gap is real OS work (a new child type, channel operation, install path, …), [open an issue](https://github.com/Picolab/pico-engine/issues) on **pico-engine** and describe the wrangler-shaped event or query you expected — not a request to bless more direct `ctx` use. **How to interact:** developer UI (Rulesets, Channels, About), **`wrangler:*` on `/sky/event`** where channel policy allows, **`wrangler:picoQuery`** on family channels, and the generated [wrangler API doc](rulesets/io.picolabs.wrangler.md). **Optional** rulesets ship in the same `krl/` directory but are not auto-installed on every pico: | Ruleset | When to install | |---------|-----------------| | [io.picolabs.oauth](rulesets/io.picolabs.oauth.md) | **Root pico only** — require OAuth bearer on most `/sky/*` traffic (mesh lock) | | [io.picolabs.portable_subtree](rulesets/io.picolabs.portable_subtree.md) | Portable subtree export/import (e.g. identity-aware move) | Install optional rulesets from the **Rulesets** tab (URL like `https://your-engine/krl/io.picolabs.oauth.krl`) or send `wrangler:install_ruleset_request`. **Generated API docs** (events, queries, `@example` from KRL annotations) live under **[rulesets/](rulesets/README.md)** — notably [wrangler](rulesets/io.picolabs.wrangler.md). Refresh in a git checkout: `npm run check:krl-schema-samples` then **`npm run generate:llms`** (copies samples into `docs/rulesets/`). --- ## Manifold (mesh framework) Above wrangler sits **[Manifold](https://github.com/Picolab/manifold-api)** — KRL rulesets that define a **personal mesh**: owner, Manifold hub, **things** (devices, vehicles, objects), **communities** (groups of things), tag and skills registries, and centralized notifications. Manifold uses **wrangler** for all lifecycle (`manifold:create_thing` delegates to `wrangler:new_child_request`, and so on); app code should call **`manifold:*` and domain events**, not `ctx:`. Manifold is **not** one product. It is the shared platform for many meshes, for example: - **Fuse-like connected car** networks (owners, vehicles, shared access) - **Household / personal IoT** with things and communities - **SafeAndMine**, **Home Assistant**, and other integrators shipped from manifold-api - **Sensor or industrial monitoring** (separate repos install bootstrap rulesets on the Manifold pico) Install **`io.picolabs.manifold_bootstrap`** on the engine root to stand up the graph; domain repos add their own rulesets on the Manifold child. **Documentation (separate repo):** - Overview: [docs/Manifold.md](https://github.com/Picolab/manifold-api/blob/main/docs/Manifold.md) - Agent index: [docs/llms.txt](https://github.com/Picolab/manifold-api/blob/main/docs/llms.txt) - Generated ruleset APIs: [docs/rulesets/](https://github.com/Picolab/manifold-api/tree/main/docs/rulesets) (e.g. [manifold_pico](https://github.com/Picolab/manifold-api/blob/main/docs/rulesets/io.picolabs.manifold_pico.md)) --- ## Security and policy (1.8) - **Deny overrides permit** on channel policies (pico-framework 0.8.2+). - **Policy ceiling:** parent can cap child channel policies; optional `defaultRootPolicyCeiling` on the root. - **Wildcard channels:** opt-in `wildcardSkyRequireBearer` so wide `/sky/*` access requires a validated bearer. - **Channel policy audit:** [channel-policy-audit-2026-10.md](https://github.com/Picolab/pico-engine/blob/master/docs/design/channel-policy-audit-2026-10.md) (repo; summary in [1.8.0 release notes](https://github.com/Picolab/pico-engine/blob/master/docs/release/1.8.0.md)). Owner UI uses the **UI channel** (passkey). Automation uses **OAuth** on `/sky/*` and narrow channels — see README § OAuth. --- ## HTTP surfaces | Path | Auth | Use | |------|------|-----| | `/` | Browser | Developer UI | | `/c/*` | Passkey session, or `X-Pico-Internal` (dev MCP) | Developer UI (browser). **Discouraged** for automation and third-party apps — use `/sky/*` or mesh MCP instead. | | `/sky/*` | Channel policy ± OAuth bearer | Events, queries, integrators | | `/mcp/:eci` | OAuth bearer | Mesh MCP (Streamable HTTP). A missing token is `401` with `WWW-Authenticate: Bearer resource_metadata="…"`. | | `/.well-known/oauth-authorization-server` | Public | OAuth authorization-server metadata (RFC 8414) for MCP clients | | `/.well-known/oauth-protected-resource/mcp/:eci` | Public | Protected-resource metadata (RFC 9728) for that channel's MCP URL | | `/auth/*` | WebAuthn flows | Registration, login | Local dev MCP: `pico-engine mcp --stdio` — not exposed on hosted load balancers; see [1.8 plan § MCP](https://github.com/Picolab/pico-engine/blob/master/docs/release/1.8.md) (deployment table). --- ## Agents and KRL authoring - **`krl-check`** on ruleset files before install. - **Dev MCP** (stdio): validate, install, flush, raise, query — same mesh as the UI when `PICO_ENGINE_HOME` / config match. It runs the Try It! demos too (`list_try_it_demos`, `run_try_it_demo`, `try_it_status`), and serves the cookbook and Developing with Picos pages as `pico://docs/cookbook//README`, `…/instructions`, and `pico://docs/developing/`. - **Docs for LLMs:** [`llms.txt`](llms.txt), [`llms-full.txt`](llms-full.txt), [`openapi.yaml`](openapi.yaml). Regenerate after doc edits: `npm run generate:llms` at repo root. --- ## Troubleshooting | Symptom | Check | |---------|--------| | Passkey fails | `baseUrl` / `webauthn.origin` match browser URL; HTTPS in production | | “Not allowed by channel policy” | Channel event/query policy; ceiling; global deny lists | | Install/flush errors in UI | Event-wait returns `events` with `ruleset_install_failed` / `ruleset_flush_failed` and line/col in `errors` | | LevelDB LOCK on startup | Unclean shutdown; stop all tasks using the volume; engine may remove stale LOCK | | Empty mesh after deploy | Fargate EBS vs EFS — use persistent volume for `home` | Logs: default `$home/pico-engine.log`, or stdout when configured. Engine logs **`txnError`** at error level for ruleset compile failures (includes line/col when available). --- ## Related docs - [Pico overview / philosophy](pico-overview.md) · [Pico views](pico-views.md) · [Bundled rulesets (wrangler, PDS, …)](rulesets/README.md) · [Manifold mesh framework](https://github.com/Picolab/manifold-api/blob/main/docs/llms.txt) · [KRL manual](krl/README.md) - Package [README](../../README.md) — OAuth, authentication, full container checklist - Release [1.8.0 notes](https://github.com/Picolab/pico-engine/blob/master/docs/release/1.8.0.md) — what changed in this line --- # Pico views # Pico views A **view** is a shared function that returns [XMLUI](https://xmlui.org) markup. Install the ruleset, open it on a channel that may call that function, and the engine serves the page. The engine ships the XMLUI runtime (`/xmlui/0.14.28.js`) and the optional ruleset `io.picolabs.xmlui`. That ruleset builds the app shell, navigation, and the markup helpers. It is not installed on every pico. Install it from the engine: `/krl/io.picolabs.xmlui.krl` Channel policy applies to views the same way it applies to any other query. A channel that cannot call the function does not get that page. Live data still comes from `/sky/query` and `/sky/event` on that same channel. Examples live in `demos/xmlui/` in the pico-engine repository; the [cookbook](/cookbook/xmlui/) shows how to install and open each one. The design record is `docs/design/pico-views-xmlui.md`. ## Write a view Tag the function with `@view` and return one ``: ```krl ruleset com.example.hello { meta { name "Hello" shares hello } global { /** * A first view * @view "Hello" */ hello = function() { <<

Hello

This view is a shared function in #{meta:rid}.
>> } } } ``` `@view "Title"` sets the navigation label. The shell turns the function name into the component tag by capitalizing the first letter (`hello` → ``). The `` in the markup has to be that same name. Give the function a path when you want a specific URL: ```krl @view "Settings" /settings ``` With one view and no path, the page is `/`. With several views and no path, each page is `/` plus the function name. A path that contains `:` is a detail page (`@view "Contact" /contact/:id`). It is left out of the navigation. The component reads `{$routeParams.id}`. Other views link to it with ``. Do not name a view after a built-in XMLUI tag (`List`, `Table`, `Form`, `Text`, …). A function named `list` becomes ``, which is the framework list, not your page. Name it `contactsPage` and, if you want the URL to be `/list`, set the path. `#{}` runs in KRL while the page is generated. Use it for constants: `meta:rid`, labels, and the helpers below. Do not put entity variables or query results in `#{}`. Those values belong in a `DataSource`, which XMLUI renders as text. `krl check` warns when a `@view` function interpolates `ent:` and when the heredoc is not well-formed XML. ## Open it | URL | What it serves | |-----|----------------| | `/ui/{eci}` | A list of rulesets on this channel that have a view the channel may call | | `/ui/{eci}/{rid}/` | That ruleset's app. Routes are hash URLs under this path | The shell (`index.html`), `Main.xmlui`, and `config.json` come from `io.picolabs.xmlui:file`. `components/.xmlui` is the view function itself, queried on the channel in the URL. In the developer UI, a ruleset with at least one `@view` function has a **View** button on the Rulesets tab. Pick the channel, then open the app. The channel must allow: - `io.picolabs.xmlui` / `file`, or the shell does not load - the view function, or that component is not found - every query and event the page calls A policy denial on these routes is HTTP 403. Anything else the route cannot serve is HTTP 404. These `/ui/` files are the ruleset's markup plus the ECI and RID already in the URL. They are served without a bearer token. Pico data is not in them. When the engine requires a bearer on `/sky/*`, the shell sends `sessionStorage.picoViewBearer` as `appGlobals.auth` on each `DataSource` and `APICall`. See [OAuth-protected meshes](#oauth-protected-meshes). ## Read and write `use module io.picolabs.xmlui alias ui`. The helpers return markup, so they go in the heredoc with `#{}`. | Call | Result | |------|--------| | `ui:query(id, name, opts)` | A `DataSource`. Set `opts.rid` to `meta:rid`: without it the `DataSource` calls `io.picolabs.xmlui`, because inside the module `meta:rid` is the module's own. `opts.args` is a map of query parameters (values may be XMLUI bindings such as `"{$routeParams.id}"`). `opts.select` sets `resultSelector`. | | `ui:event(id, domain, type, opts)` | An `APICall` that posts the form body. `opts.refresh` is a list of `DataSource` ids to refetch. `opts.confirm` sets a confirmation message. | | `ui:query_url(name, rid, eci)` | `/sky/query/…` for this channel, ruleset, and query. | | `ui:event_url(domain, type, eci)` | `/sky/event/{eci}/xmlui/{domain}/{type}`. | | `ui:app_url(tags, app_rid)` | `/ui/{eci}/{rid}/` for the newest channel that has every tag in `tags`, or `null`. Extra tags on the channel still match. | | `ui:escape(s)` | Escape `&`, `<`, `>`, and `"` for text placed in markup. | A guestbook view: ```krl meta { name "Guestbook" use module io.picolabs.xmlui alias ui shares entries, book } global { /** Guestbook entries, newest first. @returns Array */ entries = function() { ent:entries.defaultsTo([]).reverse() } /** * Sign and read the guestbook * @view "Guestbook" */ book = function() { << #{ui:query("entries", "entries", {"rid": meta:rid})} #{ui:event("sign", "guestbook", "signed", {"refresh": ["entries"]})}
>> } } ``` The matching rule selects `guestbook signed`. After it fires, `entries.refetch()` reloads the table. The full ruleset is `demos/xmlui/com.example.guestbook.krl`. ## App channel `ui:create_app_channel(tags, domains, queries, app_rid)` creates a channel whose events are limited to `domains` (each name is `*`) and whose queries are `queries` on `app_rid`. When `queries` is `null`, every shared function of `app_rid` is allowed. The query policy always includes `io.picolabs.xmlui` / `file`. ```krl rule create_app_channel { select when wrangler ruleset_installed where event:attr("rids") >< meta:rid ui:create_app_channel(["guestbook", "app"], ["guestbook"], null, meta:rid) } ``` `ui:rotate_app_channel(tags)` creates a new channel with the same tags and policy as the newest channel that already has those tags. ## Components A shared function tagged `@component` returns one `` that other views use as a tag. It is not a page: it has no `Page` and no navigation entry. The component name is the function name with its first letter capitalized (`readingHistory` → ``). Pass data in as attributes and read them with `$props`. Use `` for children, `method.open="…"` for a method, and `emitEvent('saved', data)` for an event. ```krl /** * Readings as a table. Props: rows. * @component */ readingHistory = function() { <<
>> } ``` A view then writes ``. Share the function like a view. Do not name a component after a built-in XMLUI tag (`Table`, `Form`, `List`, …): `krl check` warns, and the engine will not serve it. The engine serves `components/.xmlui` from the app ruleset first, then from `io.picolabs.xmlui`, then from one other installed ruleset. A component is a query, so the channel must allow it. `ui:create_app_channel` allows every `@component` of the app ruleset and of `io.picolabs.xmlui`. When a schema changes (install, uninstall, or a flush that changes it), `io.picolabs.xmlui` adds missing component queries to every app channel, for the rulesets that channel already allows. `io.picolabs.xmlui` provides four components every view can use: | Component | Use | |-----------|-----| | `` | Wraps the page in the theme stored in this tab (`picoViewTheme`), with a background that keeps rows readable in dark themes | | `` | A settings dialog. Put fields inside it. Save stores each field in `sessionStorage` as `picoView` plus the field name capitalized, then reloads. Open it with `settings.open()` | | `` | The theme picker, for `PicoSettings` | | `` | The access token for an [OAuth-protected mesh](#oauth-protected-meshes), for `PicoSettings`. Leave it out on an open mesh | ```xml