The Glass Box Benchmark

We don't sell AI magic. We sell infrastructure.

So instead of publishing promises, we ran MemoryGate through a chaos benchmark designed to break real systems: noisy conversations, user reversals, autonomous agents, and privacy constraints that remove access to the underlying data entirely.

The result is not a perfect score. It's something better: a transparent, runnable proof of what works today—and what we are still improving.

The "Vaporware" Problem

The AI space is crowded with claims about "solving hallucinations." Very few companies show how their system behaves when users are inconsistent, wrong, or constantly changing their minds.

MemoryGate is not a chatbot, a RAG framework, or a prompt layer.
It is a Pre-LLM Trust Gate that sits between your vector database and your model.

Its only job is to score retrieved memories for trustworthiness—so outdated, contradicted, or reversed facts don't make it into the prompt at all.

To prove this isn't vaporware, we ran a full Chaos Benchmark. Then we made it harder.

Part 1: Surviving the Chaos (Logic Under Stress)

First, we tested the trust logic itself. We flooded the system with noise, indecision, and autonomous correction loops to see whether trust scores held up.

Scenario	Score	Status	Result
Long-Haul Memory	100/100	✅ PASS	A corrected fact was buried under 45+ unrelated memories. The Gate still suppressed the outdated value.
Flip-Flop Reversal	100/100	✅ PASS	User changed A → B → A. The Gate correctly recognized the reversal and restored trust in A.
Agentic Loop	100/100	✅ PASS	An autonomous agent detected its own error and issued a valid correction via API until convergence.

What this proves:
MemoryGate handles long-horizon persistence, reversals, and self-correction without relying on prompt tricks or context window size.

How Corrections Work End-to-End (Trust vs Truth)

A correction is not a single action.
It has two required parts.

MemoryGate is responsible for only one of them.

The Two-Part Correction Model

In real systems, correcting information requires both:

1. Suppressing the wrong memory
2. Injecting a new, correct memory

If you do only the first, the system can suppress bad information—but it will correctly respond with "I don't know" instead of the truth.

That's not a correction. That's silence.

MemoryGate API — Trust Suppression (What We Do)

The MemoryGate API is responsible for trust suppression.

When a correction is detected, the MemoryGate API:

• Flags or decays the trust score of the outdated memory
• Prevents it from being retrieved again
• Does not delete or overwrite any data
• Does not invent replacement facts

This answers one question only:
"Should this memory still be trusted?"

That's the product.

Client Workflow — Truth Injection (Reference Demo)

Completing a correction requires a client-side action.

Our benchmark and demo harness model how real systems already behave:
new user input is appended to the client's own database as a new memory.

In practice, the client:

• Takes the corrected user input (or agent-generated correction)
• Injects it into its own system as a new memory
• Assigns a new memory_id
• Queries again, now receiving the corrected answer

This mirrors production reality:

• Chat systems append new turns
• Knowledge bases version facts
• Agents write improved state after validation

MemoryGate does not perform this step.
The demo exists to show how clients complete the loop.

Why the Demo Workflow Exists

The MemoryGate API could stop at returning trust scores.

But then the obvious question becomes:
"What do I do with this score?"

So the Chaos Benchmark includes a reference client workflow to demonstrate:

• How clients should write new memories
• How suppression + injection produces correct answers
• Why correction is incomplete without both steps

Every Chaos Benchmark scenario follows the same pattern:

1. Detect a correction
2. Suppress outdated memory via the MemoryGate API
3. Inject corrected memory via the client workflow
4. Query again to verify truth convergence

The benchmark tests end-to-end correctness, not hidden mutation.

Part 2: The Ghost Protocol (Privacy Without Blindness)

Enterprise security teams demand zero data retention. That requirement usually breaks AI memory systems.

So we built Privacy Mode.

In Privacy Mode, MemoryGate processes content transiently in-memory to generate embeddings, with no persistence, logging, or storage of raw text. The database stores only trust metadata. Content is effectively NULL.

We re-ran the Chaos Benchmark with SVTD_PRIVACY_MODE=true.

The question:
Can you score the trustworthiness of a memory you can't read?

The answer:
Yes.

• Persistence: Even with content: NULL, MemoryGate tracked memory_id and decayed trust on outdated facts correctly.
• Identity: With "Bring Your Own ID," the client retains ownership of truth. MemoryGate manages trust, not data.

This unlocks deployment paths for SOC 2, HIPAA, and other regulated environments where content visibility is unacceptable.

Radical Transparency: Where We Are Still Improving

We don't believe in hiding edge cases. Here's what didn't score an A+—and why.

1. Multi-User State (Grade: B / 75%)

Corrections successfully propagated at the shared-tenant level (if User A corrects a fact, User B sees the update).
However, the current test harness uses a sequential query pattern rather than true concurrent user sessions.

The trust logic is sound, but test coverage for concurrent session isolation needs improvement. This is a policy-layer expansion, not a core trust failure.

2. Sentinel Correction Detection (Grade: C+)

Sentinel—the correction detector—is intentionally tuned to be conservative.

• Precision: 100%
  We did not falsely flag normal conversation as a correction.
• Recall: ~60%
  We missed some subtle corrections.

This is a deliberate trade-off. In enterprise systems, accidentally decaying good memory is worse than missing a correction. We are actively tuning Sentinel toward higher recall, but safety comes first.

Why We Publish This Anyway

Many companies would hide these grades.

We don't.

If you see a system claiming 100/100 everywhere, you should be skeptical. Real infrastructure earns trust by showing failure modes, not pretending they don't exist.

Conclusion: It's Just an API Call

This benchmark proves that MemoryGate is a functional, resilient primitive:

• It handles messy human behavior.
• It works even when content is inaccessible.
• It integrates without forcing platform rewrites.

A correction is only complete when outdated memory is suppressed and new truth is explicitly written—MemoryGate handles the first; clients control the second.

You keep your vector database.
You keep your LLM.
You add one gate: /v1/query.

We are continuing to tune, optimize, and harden the system—but the core is real, testable, and running today.

→ Request Enterprise Beta Access

If you want next:

• a CISO-grade executive summary
• a text-only architecture diagram
• or a short FAQ for security reviews

Just say the word. @MemoryGateWebsite