Institutional Intelligence Prototype

Objective

Test whether AI can make institutional analysis faster and more traceable. The prototype maps how the policies, regulations, institutions and indicators in a set of documents relate to one another, and proves each link with a quote that is checked automatically.

The AI Development Divide argues that institutional capacity decides who benefits from AI. Central banks, development finance institutions, regulators and compliance teams spend much of their time tracing these connections across long documents. A tool that speeds this up, without asking analysts to take its output on trust, is one way to build that capacity.

Technical architecture

1. Documents. The analyst picks published papers from this site, or pastes up to 30,000 characters of their own text. Pasted text is processed for that request and not stored.
2. Structured extraction. The documents go to Claude with a required output schema: a summary, entities with types, relationships with a verbatim evidence quote and source document, and an optional answer with quotes. Forcing the structure makes the output machine-checkable.
3. Verification. The server checks every quote against the source document after normalizing spacing, punctuation and formatting. Each relationship is marked as found or not found, and relationships pointing to entities that do not exist are dropped.
4. Interface. A network map with nodes coloured by type: select a node to focus on its links, and unverified links are shown dashed in red. Alongside it are an evidence table, the answer, and a traceability score (the share of quotes found verbatim).

Methodology

• Evidence by construction. Every relationship must carry a quote, so an analyst can check any link in seconds instead of rereading the document.
• Verification is mechanical, not model-based. A quote either appears in the source or it does not. This catches fabricated or paraphrased evidence without relying on a second model to judge the first.
• Domain-neutral by design. No policy domain was fixed in advance, so the prototype works on whatever documents it is given. A domain-specific version, such as financial inclusion policy, can be built on the same engine once one is chosen.

Evaluation

The verification step was tested with simulated model output:

• A quote copied from the source was marked as found.
• An invented quote was marked as not found.
• A relationship pointing to a missing entity was removed.

Planned evaluation on the live model:

Measure	How it is assessed
Traceability	Share of quotes found verbatim in the source, calculated on every run
Relationship validity	Whether a verified quote actually supports the stated relationship, judged by a reviewer
Coverage	Whether the important relationships in a document are captured, compared with a reviewer's own map
Time saved	Time to produce a reviewed map, compared with doing it manually

Results

No results yet. Results will be published here after the first evaluated runs on the live model, starting with the published papers.

Limitations

• A found quote is not proof. It shows the text exists, not that it supports the relationship drawn from it, so relationship validity still needs human review.
• Long documents must stay under 60,000 characters in total per run. Larger collections would need retrieval first, as in the Research Engine.
• Repeat runs can differ. The model chooses which entities matter, so two runs on the same documents can produce different maps.
• Few relationship types. Relationships are short verb phrases, not a fixed vocabulary, so maps from different runs are not directly comparable.

Code and resources

The prototype runs on this page through the site's server. A public repository will be linked here once it is published.