Tag

#memory

292 results found

SAIHM — Sovereign AI Horizontal Memory
@SAIHM-Admin

What this is A sovereign, encrypted, sharable, persistent memory protocol for AI agents. A Model Context Protocol server that exposes eight tools any MCP-capable AI agent (Claude Code, Claude Desktop, custom agents) can call to gain a persistent, encrypted memory layer the user owns: saihm_remember — store an encrypted memory cell saihm_recall — retrieve and decrypt your memories saihm_forget — true cryptographic erasure (GDPR Art. 17) saihm_status — your protocol-runtime stats and storage tier dashboard saihm_share / saihm_revoke_share — selectively share a memory with another agent or user saihm_governance_propose / saihm_governance_vote — protocol governance via gSAIHM Each tool forwards to a SAIHM operator endpoint that runs the full protocol stack on COTI V2 mainnet. The server itself holds no crypto, no storage, and no protocol runtime — those live behind the operator endpoint. Tool reference Tool Title Behavior saihm_remember Remember writes a new memory cell saihm_recall Recall read-only; safe to repeat saihm_forget Forget (GDPR erasure) destructive — irreversible erasure saihm_status Status read-only saihm_share Share writes a sharing contract saihm_revoke_share Revoke share withdraws a grant saihm_governance_propose Propose (governance) opens a proposal saihm_governance_vote Vote (governance) casts a vote Each tool carries MCP annotations (readOnlyHint, destructiveHint, idempotentHint, openWorldHint) and a human-readable title, so MCP hosts can gate confirmations and agents can select the right tool at reasoning time. Companion package This package speaks MCP. For production client-side cryptography — post-quantum sealing, authenticated sharing, and provable erasure performed on your own machine so the operator stays blind — pair it with @saihm/client-pro. See it run Runnable, one-command demos ground a memory you own in every major model — Claude, GPT, DeepSeek, Qwen, Kimi, GLM — then prove you can erase it, alongside drop-in adapters for LangChain, LlamaIndex, CrewAI, AutoGen, and LangGraph. Each runs offline in about a minute; no account needed. Live demos: https://citw2.github.io/saihm-demos/ demo-claude-code wires this server into Claude Code and Cursor as an MCP server. Measured — up to ~86% fewer context tokens. Most agents re-send their entire transcript every turn, so context spend grows ~O(N²) over a session; recalling a bounded set of memory cells instead cut input tokens by 62.8%–85.9% across a realistic multi-session coding task. The benchmark is open, offline, and deterministic — reproduce the number rather than trust it: git clone https://github.com/citw2/saihm-token-benchmark cd saihm-token-benchmark && npm install && node benchmark.mjs Install npm install @saihm/mcp-server # or run directly without install: npx @saihm/mcp-server Configure The server needs two env vars: SAIHM_ENDPOINT_URL=https://operator.example.com/mcp SAIHM_AUTH_HEADER=Bearer <token-issued-by-your-operator> SAIHM_ENDPOINT_URL — the SAIHM operator endpoint. Operators publish their endpoint URLs at https://saihm.coti.global. SAIHM_AUTH_HEADER — the Authorization header value the operator expects (typically a Bearer <token> issued to you after key-bound enrolment). The server is authentication-agnostic and never transmits raw private keys; the operator's enrolment flow keeps your signing key on your machine. Place these in a .env file alongside the server (the .gitignore excludes all .env* files from any future repo). Wire into Claude Code { "mcpServers": { "saihm": { "command": "npx", "args": ["@saihm/mcp-server"], "env": { "SAIHM_ENDPOINT_URL": "https://operator.example.com/mcp", "SAIHM_AUTH_HEADER": "Bearer <token>" } } } } What gets persisted, where The server itself persists nothing. The operator endpoint runs the full protocol stack: cells are encrypted under a per-cell DEK, sealed by a per-agent KEK, persisted to the operator's configured durable storage, and audited on COTI V2 mainnet. See the operator's documentation for tier details, and Storage is the operator's responsibility (by design) below. Storage is the operator's responsibility (by design) For operators — read this first. SAIHM does not hard-wire your durable storage to any single provider, and it does not silently provision storage for you. Choosing and configuring where cells are persisted is your job, on purpose. This is a deliberate design choice for operator convenience and data sovereignty — not a missing feature. If memory writes fail with a storage error, it almost always means the backend has not been configured yet. Why it works this way: Provider sovereignty. You decide where your tenants' encrypted cells live. The protocol never locks you to one vendor or one network. Local-first, then deep-archive. A typical operator routes writes to a local IPFS (Kubo) node first — fast, authoritative, and under your own control — and then asynchronously to a Filecoin deep-archive provider such as Pinata, Synapse, or Lighthouse. The same content addressing spans both tiers. Your memory and your tenants' take the same path. Whatever backend you configure serves both the operator's own memory and every tenant's — there is no separate hidden sink hard-coded to one provider. What you configure (your operator deployment guide lists the exact settings): a reachable IPFS / Kubo endpoint (a local node is recommended) for the authoritative low-latency tier, and credentials for at least one Filecoin / IPFS pinning provider for durable deep-archive. If neither is configured, the endpoint has nowhere durable to put cells and will reject writes rather than lose data. That refusal is intentional. Prefer not to run storage yourself? Join SAIHM. You have two paths, and either is fine: Run your own operator endpoint and configure the storage backend as described above — full sovereignty, your infrastructure. Join the hosted SAIHM operator and let it provide durable storage for you. It runs blind / non-custodial: paired with client-side sealing (see @saihm/client-pro and @saihm/mcp-server-pro), it only ever stores ciphertext and never holds your keys — so you get managed storage without giving up custody. Enrol via Join SAIHM at https://saihm.coti.global (a paid hosted service). Reporting engine A reporting library is bundled as a sub-export, so operators can compose the eight MCP calls into bespoke reports with their own tooling (no extra dependency, no extra service): import { validateBespokeTemplate, registerTemplate, generateRegistryAttestation, StubPublicRegistry, InMemoryReportingRuntime, GDPR_ART15_FIELDS, REGISTRY_ATTESTATION_FIELDS, type BespokeReportTemplate, } from "@saihm/mcp-server/reporting"; What it covers Field universe (FIELD_UNIVERSE) — 280 fields (262 framework + 18 ledger). Templates that project a field outside this set are rejected at validation. Bespoke template schema — zod validator + universe-membership check + scope/cap enforcement. Authorization path validators — 4 paths: public / self / operator-self / operator-for-downstream. Receipt emission — 6 sub-kinds (report_generated / report_rejected / template_registered / template_superseded / erasure_chain_broken / rate_limit_exceeded) under a stable, domain-separated receipt namespace. Framework smoke — registry-attestation (public auth) for end-to-end plumbing verification. Constraints Every fieldProjections[] entry MUST be in FIELD_UNIVERSE. scope.customerIdHashes 64-hex; max 10,000 per template. scope.timeRange window ≤ 366 days. fieldProjections length 1–200. framework ∈ {gdpr-art-15, gdpr-art-17, soc2-t1, soc2-t2, iso27001, aml, audit-export, billing-history, registry-attestation}. format ∈ {pdfa3, json, csv}. Worked example const template: BespokeReportTemplate = { templateId: "acme-q1-summary", templateVersion: 1, operatorIdHash: "ab".repeat(32), scope: { customerIdHashes: ["cd".repeat(32)], timeRange: { from: "2026-01-01T00:00:00Z", to: "2026-04-01T00:00:00Z" }, }, framework: "gdpr-art-15", fieldProjections: [GDPR_ART15_FIELDS[0], GDPR_ART15_FIELDS[1]], format: "pdfa3", }; const v = validateBespokeTemplate(template); if (!v.valid) throw new Error(v.errors.join(", ")); const runtime = new InMemoryReportingRuntime(); // replace with your audit-ledger runtime const reg = await registerTemplate(template, runtime); if (reg.ok) console.log("registered:", reg.templateHash); In production, replace InMemoryReportingRuntime with a runtime that persists audit payloads to your operator's audit ledger. Operators who inject signature verifiers should use pure-crypto libraries (@noble/curves for EIP-712, @noble/post-quantum for FIPS 204 ML-DSA) — the package itself bundles no EVM tooling. Security The server enforces a small set of defaults so misconfiguration cannot leak the Authorization header in transit: HTTPS-only endpoints. SAIHM_ENDPOINT_URL must use https://. Plain http:// is rejected at construction time, except for 127.0.0.1 and localhost (so a local operator endpoint works during development). Per-call abort window. Each request runs under an AbortController that aborts after 30s, preventing a hung endpoint from starving the MCP server. Response-size cap. Responses whose Content-Length exceeds 16 MB are rejected before deserialisation. No header echo. Authorization is never included in thrown error messages or stdout. No filesystem reads. The package never reads from disk; configuration flows entirely through env vars. Zero EVM tooling. No ethers, no eth_*, no Solidity. If operators inject signature verifiers via AuthVerifiers, they should use pure-crypto libraries (@noble/curves, @noble/post-quantum). Trust model: this client trusts whatever endpoint the operator configures. Cell IDs, audit anchors, and report receipts returned from that endpoint are surfaced to the agent verbatim — operators are the authority for content shown via saihm_recall. Verifying receipts against COTI V2 mainnet anchors is out of scope for this server; consume the cellId and auditCellId fields and verify against your own SAIHM mainnet read path. For distribution integrity, each release carries the npm registry signature; verify with npm audit signatures (and inspect npm view @saihm/mcp-server --json | jq .dist). Dependencies The published npm package has a minimal runtime surface: Dependency License Role Node.js (≥ 20.x) MIT Runtime @modelcontextprotocol/sdk MIT MCP SDK; binds the eight-tool surface TypeScript Apache-2.0 Build-time only tsx MIT TypeScript runner for tests + CLI No copyleft, no proprietary dependencies. Cryptographic primitives at the operator-endpoint layer (ML-DSA-65 / Ed25519 / key derivation) are not bundled into this MCP server; operators implementing the protocol stack are recommended to use @noble/post-quantum and @noble/curves (MIT) rather than rolling custom code. Achievements OpenSSF Best Practices Passing badge — project 12898, 100% Passing criteria (2026-05-19). https://www.bestpractices.dev/projects/12898 IETF Independent Submission Stream — draft-saihm-memory-protocol-01 (2026-05-27) is In ISE Review in the Independent Submission Stream. It is not an Internet Standard, is not endorsed by the IETF, and has no formal standing in the IETF standards process. https://datatracker.ietf.org/doc/draft-saihm-memory-protocol/ npm registry — @saihm/mcp-server@0.3.4 published (2026-06-22) adds a conspicuous "Storage is the operator's responsibility (by design)" section — documenting BYO storage and the Join-SAIHM hosted, non-custodial option. 0.3.3 (2026-06-22) was a documentation release that states the Independent-Submission status precisely (no implied IETF endorsement) and cross-references the companion package @saihm/client-pro. 0.3.2 (2026-06-22) corrected the documented operator-endpoint path to /mcp (the canonical SAIHM_ENDPOINT_URL path) across the README and client comments. 0.3.1 (2026-05-28) was a metadata patch that sources the MCP serverInfo.version from package.json (was hardcoded "0.1.0" from 0.1.0 through 0.3.0). 0.3.0 (also 2026-05-28) aligned the saihm_status response shape with draft-saihm-memory-protocol-01 §3.4 (full eight-field schema: prs, bfsi, bfsi_window_start_ts, bfsi_R, bfsi_M, shards, contracts, governance). 0.2.0 (also 2026-05-28) aligned the cell-tuple response shape with §2.1; 0.1.3 was the OpenSSF Best Practices Passing badge release (2026-05-19). MCP Registry / Glama — server listed for discovery (2026-05-16). Roadmap A 12-month roadmap is maintained in the project's AAIF proposal and will be mirrored to https://saihm.coti.global/roadmap with the v0.2.x release. Near-term tracks: 2026-Q2 — Operator-endpoint reference implementation; OpenSSF Silver pursuit (governance, code-of-conduct, DCO, signed releases, coverage tooling, assurance case). 2026-Q3 — First 2–3 external organization deployments; formal AAIF Project Proposal submission when adoption blockers clear. 2026-Q4 — NIST AI RMF crosswalk public review; EU AI Act compliance-checklist generator. OpenSSF Silver award (target). 2027-Q1 — Independent-stream (ISE) RFC publication, subject to RFC-Editor review — not an IETF-consensus standard; v1.0 reference implementation. License Apache-2.0 — see LICENSE. Project Site: https://saihm.coti.global Issue tracker: https://github.com/SAIHM-Admin/saihm-mcp/issues Security: see SECURITY.md for private vulnerability disclosure Contributing: see CONTRIBUTING.md and CODE_OF_CONDUCT.md Governance: see GOVERNANCE.md Changelog: see CHANGELOG.md

5 days ago
Spf Smart Gate rust binary mcp server with built in local tools. preconfigured
@joseph stone

README.md--- license: apache-2.0 language: - en tags: - mcp-server - ai-gateway - security - rust - agent-framework - tool-enforcement - lmdb - rag - transformer - mesh-network - voice - android - termux - self-hosted - ai-safety - memory-system - flint - build-anchor - complexity-formula - agent-memory - p2p - quic - heed - self-learning - harness - ai-memory - persistent-memory - online-learning - agent-tools - tool-gateway - web-automation - browser-automation - social-media - p2p-communication - voice-synthesis - tts - embedded-database - zero-copy - code-search - filesystem - git - database pipeline_tag: text-generation --- ``` _____ _____ ______ _____ __ __ _____ _______ _____ _______ ______ / ____| __ \| ____| / ____| \/ | /\ | __ \__ __| / ____| /\|__ __| ____| | (___ | |__) | |__ | (___ | \ / | / \ | |__) | | | | | __ / \ | | | |__ \___ \| ___/| __| \___ \| |\/| | / /\ \ | _ / | | | | |_ | / /\ \ | | | __| ____) | | | | ____) | | | |/ ____ \| | \ \ | | | |__| |/ ____ \| | | |____ |_____/|_| |_| |_____/|_| |_/_/ \_\_| \_\ |_| \_____/_/ \_\_| |______| ``` # SPF Smart Gateway v3.0.0 **MCP Server Gateway with Multi-Layer Security Enforcement, Agent Memory, FLINT Transformer, Mesh Network, and 81 Gated Tools** > **NOTE: Full system upload still in progress.** Not all files are present yet. Repository is actively being populated — some modules may be missing until upload completes. Copyright (C) 2026 Joseph Stone — All Rights Reserved --- ## Quick Start ```bash # Clone into home folder git clone <repo-url> ~/SPFsmartGATE # Or for clones/SWARMagents: # ~/SWARMagents/1/SPFsmartGATE cd SPFsmartGATE cargo build --release # Copy optimized binary cp ~/SPFsmartGATE/target/release/spf-smart-gate ~/SPFsmartGATE/LIVE/BIN/spf-smart-gate # Configure MCP server filepath nano ~/SPFsmartGATE/LIVE/LMDB5/.mcp.json # Install Claude CLI in project directory # Use included configs, deny native Claude CLI tools # ~/SPFsmartGATE/LIVE/LMDB5/.claude.json # ~/SPFsmartGATE/LIVE/LMDB5/.claude/settings.json # Boot into flat-file agent runtime cd ~/SPFsmartGATE/LIVE/LMDB5 && claude # Boot into LMDB-backed agent runtime cd ~/SPFsmartGATE/LIVE/LMDB5.DB && claude ``` ### Route Other Models Through Claude CLI Adjust `~/SPFsmartGATE/LIVE/LMDB5/.claude/settings.local.json` with your model choice and API key. Uses OpenRouter for API and agent selection. Swap agents without changing sessions or losing project data. ### Build Notes - Cross-compiles on **Android** and **Linux** with minimal installation - Only rebuild on first boot or after system modifications - Binary: `~/SPFsmartGATE/LIVE/BIN/spf-smart-gate/spf-smart-gate` --- ## Overview SPF Smart Gateway is a **Rust-based MCP (Model Context Protocol) server** that acts as a security gateway for AI tool calls. Every file operation, bash command, brain query, and mesh call routes through compiled Rust enforcement logic. **No AI hallucination gets past the gate.** ### Web Agent Feature SPF agents can directly interact with the web and social media platforms through `spf_web_api` — a full HTTP client supporting GET, POST, PUT, DELETE, PATCH with custom headers and JSON body. Tested and working. **What agents can do:** - Post to X/Twitter, Facebook, Instagram, Reddit via their APIs - Reply to comments, send messages, manage accounts - Make authenticated API calls to any platform with stored API keys - Search, fetch, and download web content All web API calls pass through the 6-step gate pipeline with rate limiting (30-120 calls/min), content inspection, and full audit logging. Agents never touch the open web unmonitored. ### Why Heed + LMDB All persistent storage — config, agent state, brain vectors, session logs, gate training data — runs through **[heed](https://github.com/meilisearch/heed)**, a safe Rust wrapper over LMDB. This is what makes SPF extremely fast with a low memory footprint: - **Zero-copy reads** — heed maps LMDB pages directly into memory, no serialization overhead - **No server process** — LMDB is a memory-mapped B-tree library, not a database daemon - **ACID transactions** — single-writer, multi-reader with no lock contention on reads - **Sub-millisecond lookups** — B-tree index, not hash scanning - **Tiny footprint** — entire 138K+ memory store runs in-process with minimal RAM - **Phone-friendly** — designed for Android from day one; heed compiles cleanly on ARM64 Every tool call, brain search, and memory promotion goes through heed → LMDB. No network hops, no subprocess calls, no SQL parsing. The gate, brain, agent state, and FLINT training all share the same embedded database engine. Two agent runtimes: - **Flat files** — `LIVE/LMDB5/` (session state in markdown) - **LMDB database** — `LIVE/LMDB5.DB/` (session state in LMDB for persistence) Twin folder architecture: flat-file data uploaded via SPF CLI fs tools (user-only access). All agent tool calls are gated, validated, and audited. --- ## Architecture ``` ┌─────────────────────────────────────────────────────────────────┐ │ SPF Smart Gateway v3.0.0 │ │ 42 Rust modules │ ├─────────────────────────────────────────────────────────────────┤ │ MCP Server (JSON-RPC 2.0 over stdio) │ │ 81 tools │ tool alias map │ Qwen/LLM compatibility │ ├─────────────────────────────────────────────────────────────────┤ │ GATE (6-Step Pipeline) │ │ Step 0: Source logging │ │ Step 1: Rate limiting │ │ Step 2: Complexity calculation (SPF formula) │ │ Step 3: Validation (per-tool: paths, commands, Build Anchor) │ │ Step 4: Content inspection (credentials, injection) │ │ Step 5: Max mode escalation │ ├──────────┬──────────┬──────────┬──────────┬─────────────────────┤ │ FLINT │ Brain │ Mesh │ Voice │ Browser/RAG │ │ (encoder-│ (vectors │ (P2P QUIC│ (TTS/STT │ (reverse proxy │ │ decoder │ LMDB + │ Ed25519 │ espeak- │ search, fetch, │ │ ~5M │ MiniLM) │ iroh) │ ng FFI) │ RSS, web tools) │ │ params) │ │ │ │ │ ├──────────┴──────────┴──────────┴──────────┴─────────────────────┤ │ LMDB Storage Layer (heed) │ │ SPF_CONFIG │ TMP_DB │ AGENT_STATE │ Brain │ Gate Training │ │ All zero-copy reads via heed safe Rust bindings │ └─────────────────────────────────────────────────────────────────┘ ``` ### Module Inventory (42 modules) `paths`, `calculate`, `config`, `gate`, `inspect`, `mcp`, `session`, `storage`, `validate`, `web`, `http`, `dispatch`, `identity`, `mesh`, `fs`, `config_db`, `tmp_db`, `agent_state`, `tensor`, `tokenizer`, `framing`, `attention`, `ffn`, `encoder`, `decoder`, `transformer`, `checkpoint`, `gate_training`, `transformer_tools`, `train`, `learning`, `pipeline`, `worker`, `network`, `chat`, `voice`, `utf8_safe`, `brain_local`, `flint_memory`, `browser`, `orchestrator`, `channel` --- ## The SPF Formula ### Complexity Calculation ``` C = (basic ^ 1) + (dependencies ^ 7) + (complex ^ 10) + (files × 10) ``` ### Dynamic Analysis Allocation ``` a_optimal(C) = W_eff × (1 - 1/ln(C + e)) ``` Where `W_eff = 40,000` tokens and `e = Euler's number` ### Tier Allocation | Tier | C Range | Analyze | Build | Verify Passes | Approval | |------|---------|---------|-------|---------------|----------| | SIMPLE | < 500 | 40% | 60% | 1 | No | | LIGHT | < 2,000 | 60% | 40% | 1 | No | | MEDIUM | < 10,000 | 75% | 25% | 2 | No | | CRITICAL | > 10,000 | 95% | 5% | 3 | **Required** | ### Master Equation (Subtask Success) ``` P(success) = 1 - PRODUCT(1 - P_i) for i=1..D subtasks P_i = Q(a) × L(m) × V(v) × B(b) Q(a) = 1 - e^(-0.00004 × a) — Quality from analysis depth L(m) = 1 - 0.20^(m/2000) — Lookup from external memory V(v) = 1 - (1 - 0.75)^v — Verification accuracy B(b) = checks_done / checks_required — Build Anchor compliance ``` --- ## Security ### Gate Enforcement (6 Steps) Every tool call passes through `gate::process()` — compiled Rust, no runtime bypass. | Step | What | How | |------|------|-----| | 0 | Source logging | Identifies caller (Stdio, Transformer, Mesh, HTTP) | | 1 | Rate limiting | Per-tool limits (30–120 calls/min) | | 2 | Complexity calc | SPF formula → C value, tier, allocation | | 3 | Validation | Per-tool validator (paths, commands, anchors) | | 4 | Content inspection | Credential patterns, shell injection, path traversal | | 5 | Max mode | Escalation to CRITICAL tier on warnings | ### Build Anchor Protocol Files must be **read before they can be edited or overwritten**. Prevents AI hallucinations from blindly modifying files without understanding contents. - `Read` tracks files in `session.files_read` - `Edit` and `Write` check against this list - `Bash` write-class commands check target file reads - Violations: blocked (Max mode) or warned (Soft mode) ### Content Inspection Scans written/stored content for: - **Credential patterns**: API keys (sk-), AWS keys (AKIA), GitHub tokens (ghp_), Slack tokens, private keys, hardcoded passwords - **Shell injection**: Command substitution `$()`, backticks, eval/exec - **Path traversal**: `../` sequences - **Blocked path references**: Content mentioning system paths ### Blocked Paths Default blocked: `/tmp`, `/etc`, `/usr`, `/system`, `/data/data/com.termux/files/usr` ### Command Whitelist (Stage 0) Bash commands checked against sandbox and user-filesystem whitelists. Each command segment validated independently. Destructive commands (rm, chmod 777) blocked even if whitelisted. ### Default Deny Unknown tools blocked until explicitly added to the gate allowlist. --- ## MCP Tools (81 Total) ### Core Gate Tools | Tool | Description | |------|-------------| | `spf_calculate` | Calculate complexity score without executing. Returns C value, tier, allocation | | `spf_status` | Gateway status: session metrics, enforcement mode, complexity budget | | `spf_session` | Full session state: files read/written, action history, anchor ratio | ### Gated File Operations | Tool | Description | |------|-------------| | `Read` | Gated file read. Tracks for Build Anchor Protocol. Binary-safe | | `Write` | Gated file write. Validates Build Anchor, blocked paths, file size | | `Edit` | Gated file edit. Validates Build Anchor, blocked paths, change size | | `Bash` | Gated bash execution. Validates dangerous commands, /tmp access, git force | | `Glob` | Fast file pattern matching. Supports `**/*.rs`, `src/**/*.ts` | | `Grep` | Search file contents using regex. Built on ripgrep | ### Brain / Memory Tools | Tool | Description | |------|-------------| | `spf_brain_search` | Semantic vector search across collections (MiniLM-L6-v2, 384d) | | `spf_brain_recall` | Full document retrieval by semantic query | | `spf_brain_context` | Bounded context retrieval for prompt injection | | `spf_brain_store` | Store document in brain (FLINT-internal, source-gated) | | `spf_flint_store` | Agent memory store — bypasses brain write gate. Brain vectors + Working tier | | `spf_brain_index` | Index a file or directory into a brain collection | | `spf_brain_list` | List all indexed collections with document counts | | `spf_brain_status` | Brain system status: model state, storage size, collections | | `spf_brain_list_docs` | List stored documents in a collection | | `spf_brain_get_doc` | Retrieve a specific document by ID | ### Agent State Tools | Tool | Description | |------|-------------| | `spf_agent_stats` | AGENT_STATE LMDB statistics: memory count, sessions, state keys, tags | | `spf_agent_memory_search` | Search agent memories by content | | `spf_agent_memory_by_tag` | Get agent memories by tag | | `spf_agent_session_info` | Most recent session info | | `spf_agent_context` | Context summary for session continuity | ### FLINT Transformer Tools | Tool | Description | |------|-------------| | `spf_transformer_status` | FLINT transformer status: loaded, params, checkpoint, role | | `spf_transformer_infer` | Run inference: prompt → response. Returns generated tokens | | `spf_transformer_chat` | Multi-turn conversation with FLINT | | `spf_transformer_train` | Trigger manual training batch from accumulated gate signals | | `spf_transformer_metrics` | Learning metrics: loss, accuracy, gate alignment, training step | | `spf_flint_train_evil` | Mark a tool call as evil/harmful. Negative training signal | | `spf_flint_train_good` | Mark a tool call as good/safe. Positive training signal | | `spf_flint_execute` | Execute any SPF tool through FLINT worker mode (delegation) | ### Web Browser Tools **API tools (tested):** | Tool | Description | |------|-------------| | `spf_web_search` | Search the web (Brave API or DuckDuckGo) | | `spf_web_fetch` | Fetch URL and return clean readable text | | `spf_web_api` | Make HTTP API requests (GET/POST/PUT/DELETE/PATCH). Supports custom headers and JSON body — agents can directly interact with social media APIs (X/Twitter, Facebook, Instagram, Reddit, etc.) using stored API keys | | `spf_web_download` | Download a file from URL and save to disk | **Browser automation tools (in development — proxy starts, WebSocket bridge needs browser connection):** | Tool | Description | Status | |------|-------------|--------| | `spf_web_connect` | Initialize reverse proxy browser engine | Tested — works | | `spf_web_navigate` | Navigate browser to a URL (SSRF-validated) | Tested — works | | `spf_web_click` | Click a page element by CSS selector | In development — WebSocket timeout | | `spf_web_fill` | Type text into a form field by CSS selector | In development — WebSocket timeout | | `spf_web_select` | Query page elements by CSS selector | In development — WebSocket timeout | | `spf_web_eval` | Execute JavaScript on the current page | In development — WebSocket timeout | | `spf_web_screenshot` | Capture a screenshot of the current page | In development | | `spf_web_design` | Extract design brief: colours, fonts, spacing, components | In development | | `spf_web_page` | Structured page overview: title, headings, links, forms | In development | ### RAG Collector Tools | Tool | Description | |------|-------------| | `spf_rag_collect_web` | Search web and collect documents. Optional topic filter | | `spf_rag_collect_file` | Process a local file into brain | | `spf_rag_collect_folder` | Process all files in a folder | | `spf_rag_collect_drop` | Process files in DROP_HERE folder | | `spf_rag_index_gathered` | Index all documents in GATHERED to brain | | `spf_rag_dedupe` | Deduplicate a brain collection | | `spf_rag_status` | Collector status and stats | | `spf_rag_list_gathered` | List documents in GATHERED folder | | `spf_rag_bandwidth_status` | Bandwidth usage stats and limits | | `spf_rag_fetch_url` | Fetch a single URL with bandwidth limiting | | `spf_rag_collect_rss` | Collect from RSS/Atom feeds | | `spf_rag_list_feeds` | List configured RSS feeds | | `spf_rag_pending_searches` | Get pending SearchSeeker vectors (gaps needing fetch) | | `spf_rag_fulfill_search` | Mark a SearchSeeker as fulfilled after RAG fetch | | `spf_rag_smart_search` | Smart search with completeness check — triggers SearchSeeker if <80% | | `spf_rag_auto_fetch_gaps` | Automatically fetch data for all pending SearchSeekers | ### Mesh Network Tools | Tool | Description | |------|-------------| | `spf_mesh_status` | Mesh network status: role, team, identity | | `spf_mesh_peers` | List known/trusted mesh peers | | `spf_mesh_call` | Call a peer agent's tool via P2P mesh (Ed25519 authenticated) | ### Voice Tools | Tool | Description | |------|-------------| | `spf_voice_mode` | Voice pipeline control: start/stop audio, TTS (espeak-ng), mic capture | | `spf_voice_call` | Peer-to-peer voice calls: start, accept, reject, end, status | | `spf_voice_team` | Group voice channels: create, join, leave, add peers | ### Chat Tools | Tool | Description | |------|-------------| | `spf_chat_send` | Send text message to mesh peer via QUIC | | `spf_chat_history` | Chat message history (all conversations or specific) | | `spf_chat_rooms` | List active chat conversations with participant info | ### Network Pool Tools | Tool | Description | |------|-------------| | `spf_pool_status` | Pool status: worker roles, idle/busy counts, active tasks | | `spf_pool_assign` | Assign task to idle worker (NetAdmin only) | | `spf_pool_release` | Release worker and record proof of work receipt | ### Configuration Tools | Tool | Description | |------|-------------| | `spf_config_paths` | List all path rules (allowed/blocked) from SPF_CONFIG | | `spf_config_stats` | SPF_CONFIG LMDB statistics | ### Project Management Tools | Tool | Description | |------|-------------| | `spf_tmp_list` | List all registered projects with trust levels | | `spf_tmp_stats` | TMP_DB statistics: project count, access logs, resources | | `spf_tmp_get` | Get project info by path | | `spf_tmp_active` | Get the currently active project | ### Communication Hub | Tool | Description | |------|-------------| | `spf_channel` | Universal agent channel: create, join, leave, send, listen, history, list, connect (WS), disconnect, status | ### Notebook Tools | Tool | Description | |------|-------------| | `spf_notebook_edit` | Edit a Jupyter notebook cell (replace, insert, delete) | ### User-Only Tools (AI agents blocked) These tools are **hard-blocked** from AI agents at the gate level. User/system access only via SPF CLI: `spf_fs_exists`, `spf_fs_stat`, `spf_fs_ls`, `spf_fs_read`, `spf_fs_write`, `spf_fs_mkdir`, `spf_fs_rm`, `spf_fs_rename` --- ## FLINT Transformer Built-in encoder-decoder transformer for gate-aligned learning. | Property | Value | |----------|-------| | Architecture | Encoder-decoder | | Dimensions | 256d | | Heads | 8 | | Layers | 6 | | Parameters | ~5M | | Embeddings | all-MiniLM-L6-v2 (384d, in-process) | | Online learning | ON | | EWC lambda | 0.4 | | Learning rate | 1e-4 | | Replay buffer | 10,000 slots | | Checkpoint interval | 1,000 steps | | Training signal | Gate decisions (evil/FP labels) | ### Learning Pipeline | Phase | When | What | |-------|------|------| | PRE | Startup | init_brain() + index_knowledge_docs() + index_spf_sources() | | DURING | 30s loop | GateTrainingCollector → FLINT scores → route_signals → brain_store() | | AFTER | 1hr loop | Expire → Working→Fact → Fact→Pinned → auto-train (16+ tlog or 1hr) | ### Memory Lifecycle (Tiered Promotion) ``` Agent stores → Working (24hr) → Fact (7-day) → Pinned (permanent) ↓ ↓ ↓ Expire old Top 20% promote Never auto-expire ``` --- ## Brain System In-process vector memory using stoneshell-brain (Candle + LMDB + MiniLM-L6-v2). | Property | Value | |----------|-------| | Model | all-MiniLM-L6-v2 | | Embedding dim | 384 | | Chunk size | 512 | | Chunk overlap | 64 | | Storage | LMDB (vectors) + LIVE/BRAIN/DOCS/ (data files) | ### Collections | Collection | Purpose | |------------|---------| | `default` | General knowledge, web research, project docs | | `spf_source` | All src/*.rs modules indexed at boot | | `flint_results` | Tool call results (>2000 chars, before compression) | | `flint_training` | Gate decision signals, evil/FP labels | | `flint_knowledge` | User-dropped knowledge files (.md/.txt/.rs/.json) | | `flint_episodic` | Past FLINT Q+A pairs, behavioral patterns | | `session_state` | Current session metadata | ### Memory Triad (Redundant Persistence) Three systems — if any ONE fails, the other TWO recover: 1. **Brain** (vectors) — Semantic search, chunked knowledge 2. **STATUS** (sequential) — Current state, phase, next step 3. **Work Blocks** (structural) — Tasks, dependencies, confidence, progress 4. **Twin Folders** (evidence) — Data served for low-confidence work blocks --- ## Mesh Network P2P agent communication over QUIC (iroh library) with Ed25519 identity. **In development and testing.** | Feature | Status | |---------|--------| | P2P QUIC transport | In development | | Ed25519 identity | In development | | Peer discovery | In development | | Tool call proxying | In development | | Voice over mesh | In development | | Chat over mesh | In development | | Multi-agent coordination | In development | --- ## Voice Pipeline **Not yet tested.** Components built, awaiting integration testing. | Component | Technology | |-----------|-----------| | TTS | espeak-ng FFI (in-process) | | Codec | Opus (libopus.a) | | Audio | cpal + oboe-ext | | STT | Pending (JNI via Stone Shell Terminal) | --- ## Result Compression (FL-2) Three tiers based on result size: | Tier | Size | Behavior | |------|------|----------| | FULL | < 500 chars | Pass through unchanged | | SUMMARY | 500–5,000 | First 8 lines + last 3 lines + stats | | DIGEST | > 5,000 | First 200 chars + last 100 chars + stats + recall hint | Originals always preserved in brain (>2000 chars threshold) before compression. File reads never truncated (preserves non-Claude LLM compatibility). --- ## Build ```bash cd SPFsmartGATE cargo build --release # Deploy binary cp target/release/spf-smart-gate LIVE/BIN/spf-smart-gate/spf-smart-gate ``` ### Dependencies - Rust (stable) - **[heed](https://github.com/meilisearch/heed)** — safe Rust LMDB bindings. All persistent storage (config, agent state, brain vectors, training data) runs through heed → LMDB. Zero-copy reads, no server process, sub-millisecond lookups. The core reason SPF runs fast on a phone. - stoneshell-brain (Candle + MiniLM-L6-v2) - espeak-ng (TTS) - libopus (audio codec) - iroh (QUIC mesh) --- ## Configuration ### MCP Server Config `~/SPFsmartGATE/LIVE/LMDB5/.mcp.json` — points Claude CLI to the binary. ### Claude CLI Config `~/SPFsmartGATE/LIVE/LMDB5/.claude.json` — blocks native Claude CLI tools (26 tools denied). `~/SPFsmartGATE/LIVE/LMDB5/.claude/settings.json` — deny list for native tools. `~/SPFsmartGATE/LIVE/LMDB5/.claude/settings.local.json` — model routing (OpenRouter). ### SPF Config Enforcement mode (`soft` or `max`), blocked paths, allowed paths, formula weights — all in LMDB SPF_CONFIG database. --- ## File Structure ``` SPFsmartGATE/ ├── Cargo.toml # Rust project manifest (42 modules) ├── LICENSE # Apache-2.0 ├── README.md # This file ├── src/ │ ├── main.rs # CLI entry point │ ├── lib.rs # Library exports (42 pub mod) │ ├── gate.rs # Primary enforcement (6-step pipeline) │ ├── calculate.rs # SPF complexity formula │ ├── validate.rs # Rules validation (stages 0-6) │ ├── inspect.rs # Content inspection (creds, injection) │ ├── mcp.rs # MCP server (JSON-RPC 2.0, 81 tools) │ ├── dispatch.rs # Unified dispatch (all transports) │ ├── session.rs # Session state management │ ├── storage.rs # LMDB persistence │ ├── config.rs # Configuration types │ ├── brain_local.rs # In-process brain singleton │ ├── flint_memory.rs # Memory router + tiered promotion │ ├── agent_state.rs # Agent memory (LMDB5) │ ├── transformer.rs # FLINT model (encoder-decoder) │ ├── transformer_tools.rs # FLINT tool handlers │ ├── gate_training.rs # Training signal collection │ ├── train.rs # AdamW optimizer │ ├── tokenizer.rs # Tokenizer │ ├── tensor.rs # Tensor operations │ ├── attention.rs # Multi-head attention │ ├── ffn.rs # Feed-forward network │ ├── encoder.rs # Encoder stack │ ├── decoder.rs # Decoder stack │ ├── framing.rs # Message framing │ ├── checkpoint.rs # Model checkpoint save/load │ ├── learning.rs # Learning rate + EWC │ ├── pipeline.rs # Batch pipeline + API sessions │ ├── worker.rs # Worker pool │ ├── network.rs # Network pool + NetAdmin │ ├── mesh.rs # P2P QUIC mesh (iroh) │ ├── identity.rs # Ed25519 identity │ ├── chat.rs # Chat engine │ ├── voice.rs # Voice pipeline (TTS/STT) │ ├── web.rs # Web client │ ├── http.rs # HTTP server + reverse proxy │ ├── browser.rs # Browser automation │ ├── channel.rs # Universal channel hub │ ├── orchestrator.rs # Multi-agent orchestrator │ ├── config_db.rs # SPF_CONFIG LMDB │ ├── tmp_db.rs # TMP_DB LMDB │ ├── fs.rs # Virtual filesystem (LMDB) │ ├── paths.rs # Path utilities │ └── utf8_safe.rs # UTF-8 safe truncation ├── LIVE/ │ ├── BIN/spf-smart-gate/ # Deployed binary │ ├── BRAIN/DOCS/ # Brain data files │ ├── MODELS/ # FLINT checkpoints │ ├── SESSION/ # Session logs │ ├── LMDB5/ # Flat-file agent runtime │ └── LMDB5.DB/ # LMDB-backed agent runtime └── PROJECTS/PROJECTS/ └── DEPLOY/ # Agent workspace ``` --- ## Current Status | Component | Status | |-----------|--------| | MCP Server | 81 gated tools | | Gate Security | 6-step pipeline, compiled Rust enforcement | | Build Anchor | Read-before-write enforced | | Content Inspection | Credential + injection scanning | | FLINT Transformer | ~5M params, online learning, gate-aligned | | Brain | 7 collections, MiniLM-L6-v2, in-process | | Memory Triad | Brain + STATUS + Work Blocks + Twin Folders | | Tiered Promotion | Working → Fact → Pinned lifecycle | | Mesh Network | P2P QUIC, Ed25519, iroh — **in development and testing** | | Voice | TTS built (espeak-ng) — **not yet tested**, STT pending | | Chat | P2P messaging over mesh — **in development** | | RAG | Web search, RSS, file/folder indexing | | Web Agent | **Working** — spf_web_api tested (GET/POST with auth headers). Agents can interact with social media APIs | | Browser | API tools working (web_api, search, fetch). Browser automation (navigate/click/fill/select/eval) in development — proxy starts but WebSocket bridge needs browser connection | | Network Pool | Worker pool with proof of work | --- ## Notes - **1 developer** — not all features complete - **Gateway security**: approaching 100% - **All core tools**: 100% working - **Cross-compiles** on Android and Linux with minimal installation - **Agent cloning and specialization** supported - **50+ day continuous session** tested on Android phone - **Open source** — entire source code refreshes into transformer RAG system every reboot - Install in home folder, ensure file paths are correct in `.mcp.json` and `settings.local.json` - **Not all files have been uploaded yet** — repository is still being populated. Some modules may not be present until upload completes. --- ## License Licensed under the **Apache License 2.0**. See [LICENSE](LICENSE) for full terms. You are free to use, modify, and distribute this software, including for commercial purposes, provided you include the original copyright and license notice. **Author**: Joseph Stone **Email**: joepcstone@gmail.com *SPF (StoneCell Processing Formula), Build Anchor Protocol, and FLINT are proprietary designs of Joseph Stone.*

2 months ago
Scratchpad Mcp
@MikePressure

scratchpad-mcp is an MCP server that gives AI agents persistent, token-efficient storage. It solves a specific waste problem: agents constantly re-read files they've already seen, re-summarize documents they've already processed, and re-load context they've already understood. Every one of those round-trips burns tokens for no new information. This server fixes that with eight tools designed around how agents actually work: Versioned writes. write_file automatically versions every write and keeps the 10 most recent versions per file. Storage is append-only on success and atomic on failure partial writes can't corrupt state. Structured diffs. read_file accepts a since_version parameter and returns a JSON line-diff against that prior version instead of the full content. Agents that have already seen v1 can ask "what changed in v3?" and get a small structured payload they can reason about, not the entire file again. Append-only logs. append_log and read_log give agents an event-stream they can replay. Cursor-based pagination (since_entry + last_entry_id + has_more) means an agent can checkpoint where it left off and resume cheaply. On-demand summaries. summarize_file calls Claude Haiku to summarize files over ~2000 estimated tokens. Summaries are cached per file version, so repeat calls on an unchanged file cost nothing. The threshold is enforced server-side you can't accidentally pay to summarize something small. Per-agent isolation. Every operation is scoped by an agent_id parameter, so one server instance can serve many agents without leaking state between them. Storage limits. 1 MB per file write, 64 KB per log entry, 1000 files / 100k log entries / 100 MB total per agent sane multi-tenant guardrails out of the box. Backed by a single SQLite file (Postgres migration is on the roadmap). All SQL is parameterized, paths are validated against a strict allowlist, and the security model is documented honestly it's safe for one-user-per-process deployments today, and the V2 plan derives agent_id from the caller's API key for true multi-tenancy. Build agents that remember what they've already seen.

3 months ago
Memtrace
@syncable-dev

Memtrace — Structural Memory for AI Coding Agents The Problem Every AI coding agent — Claude Code, Cursor, Codex, Copilot — starts each turn completely blank. It re-reads raw source files and re-derives the full call graph, type hierarchy, and import tree from scratch on every single invocation. That structural rework burns 60–90% of the context window before any real reasoning begins. Less than 5% of tokens in a typical agentic coding session contribute genuine new intelligence. The rest is expensive, redundant noise — and it compounds: accuracy drops 40% as sessions grow, stale context crowds out signal, and summaries strip out the structural relationships agents need most. The Solution Memtrace is a bi-temporal structural memory layer that turns your codebase into a live, queryable knowledge graph — compiled from the AST, not guessed from embeddings. Every function, class, interface, and API endpoint becomes a typed node with deterministic relationships. Every file save becomes a queryable episode with timestamps, so agents can reason about structure, detect regressions, and time-travel through their own work without re-reading anything. One Rust binary. Zero configuration. Five-minute install. What agents can do with it Find callers, callees, and dependencies instantly — no file scanning, no token waste Compute blast radius before making a change — know exactly what breaks before anything is touched Detect structural drift between sessions — catch regressions the moment they happen, not at PR review Time-travel through code evolution — query any prior state of any symbol, not just git commits Search across the full codebase with hybrid retrieval — BM25 full-text + HNSW vector + graph traversal fused in one query Map API topology across services — cross-repo HTTP call graphs, dependency chains, dead endpoint detection Benefits −90% token cost on structural queries (Mem0) +26% accuracy on multi-step agentic tasks (Mem0) −91% p95 latency on structural lookups vs. RAG baselines +32.8% SWE-bench bug-fix success rate when agents have graph context (RepoGraph) 200–800ms per-save re-indexing — every file save is a queryable episode in under a second 40+ MCP tools covering indexing, search, relationships, impact analysis, temporal evolution, API topology, graph algorithms, and direct Cypher queries 12 languages + 3 IaC formats supported via Tree-sitter grammars Local-first, closed-source Rust — code never leaves the machine, no account required, no telemetry

3 months ago