Private. Evaluated on three professional audits

Hypatia

A structured process that guides an AI model through a security audit: questions derived from the code, findings proven by a test, and a record of what was left to examine.

Solidity Runs locally ilold code map Language-agnostic process

An audit from one prompt leaves no account of itself

Asked to audit a codebase, a capable model finds real vulnerabilities. It does not say which paths it examined, which it skipped, or why it dropped an idea. Run it again and it finds a different subset. Hypatia keeps the model and replaces the single prompt with a process that can be checked at each step.

From scope to proven finding

Every audit goes through the same nine steps. Each one ends with a check, and the audit moves on only when the check passes.

Step 1 of 9

Scope accepted before anything runs

contracts/
Vault.sol412 lines
Router.sol268 lines
Oracle.sol154 lines
libraries/Math.sol96 lines
test/excluded
script/excluded
node_modules/excluded
4 files in scope  ·  930 lines  ·  3 paths excludedaccepted

676pairs of calls mapped in one 5,260-line project

withdraw()amount ≤ balance?norevertyes_update()writes balancenot paused?norevertyestransfer()sends valuecaller is owner?norevertyessetFee()writes fee
entry pointa check stops the pathwhat the path changes

Layer by layer, from one function to the rules of the system

FUNCTIONSMODULESFLOWSRULESdeposit()mint()borrow()liquidate()settle()price()VaultLendingOracleborrow → price → liquidateenforced at a lineenforced at a linenothing enforces it

804what-if questions derived for that project

state two flows sharephase-dependent callsvalue moved under a lockrules nothing enforcesdoubts a function leavesordered pairs of calls804questionsA → BA → B → CA → B → C → Da pair grows into a longer path

One outcome for every question

impossiblearguedexpandedcandidate
every candidate goes to a test in the next step

No test, no finding

❯test candidate-0172.1s
withdraw, then liquidate in the same block
the attacker ends with more than they put in
proven
❯test candidate-0181.4s
deposit, then setFee
the rule holds under every order tried
rejected
❯test candidate-0193.0s
borrow, then price
no proof either way
undecided, left for review

One report per proven finding

Withdrawal after liquidation drains the vault

Highimpact highlikelihood mediumproven by test
Root cause
Exploitation path
Test
test_withdraw_after_liquidation()   passed
Recommendation

Frozen and checked before it closes

Every question has an outcomecheck 1
Every finding has a test that still passescheck 2
Every report is completecheck 3
Counts and files agreecheck 4
A later audit can read the recordcheck 5
run sealedsha256 9f3c…a41e

13new findings when one audit was continued

First audit
→
Second audit
decidedleft opennew finding

Steps marked CODE MAP need a map of the code. Steps 5 to 7 repeat until no question is open.

Four rules the process enforces

Questions come from the code

What to investigate is derived from a map of the functions, their calls and the state they share. It does not depend on what the model happens to think of.

A finding needs a test

A candidate becomes a finding only when a test shows the harm. Rejecting a candidate also needs a test, and a second reader tries to break that rejection.

Nothing closes by assertion

A question marked impossible has to cite the check in the code that stops it. The audit cannot close while questions are undecided and budget remains.

Open work is recorded

Every audit ends with its list of questions and the status of each. A later audit of the same code starts from that list.

Measured against three professional audits

The same model audited three smart-contract projects in three ways: from one prompt, with Hypatia, and with Hypatia and a code map. The reference is the list of Critical, High and Medium vulnerabilities each project's auditors reported.

ONCHAINID

2,058 lines. Mean of three runs
Model alone
3.3 of 12
Hypatia
5 of 12
Hypatia with code map
5 of 12

Promis

3,921 lines. One run
Model alone
2 of 9
Hypatia
2 of 9
Hypatia with code map
4 of 9

GTE

5,260 lines. One run
Model alone
8 of 35
Hypatia
10 of 35
Hypatia with code map
15 of 35

When the audit was continued

GTE, second round. One run
Model alone
2 new
Hypatia
2 new
Hypatia with code map
13 new

New findings when each approach was asked to continue its own audit. Of the 13, two are in the official report and four were rated doubtful.

What the study shows

  • On the two larger projects, Hypatia with a code map found the most reported vulnerabilities.
  • On those projects the gain appears with the code map, not with the structured steps alone.
  • When one audit was continued, it produced 13 new findings. The model alone produced 2.

What it costs and where it fails

  • A first audit with the code map used 2.3 to 10.2 times the output tokens of the model alone.
  • On the largest project it rejected 8 real vulnerabilities it had asked about.
  • Scores come from running tests and from reading.

Read the full study

Where it applies

Any source code

The steps, the tests and the record of open work do not depend on the language of the target.

Solidity, with a code map

The derived question list needs a map of the code. For Solidity it is built by ilold, an open-source path analyser.

Audits that continue

A second audit starts from the questions the first left open, in place of starting again.

Read the method, the results and the limits

The study describes the three approaches, the nine experiments and everything that did not work. Hypatia is developed by Sergio Corrales and its source is not public. ilold is open source.

git.seco@protonmail.com X LinkedIn