Skip to content

Releases: lajosdeme/mole

v0.2.1

Choose a tag to compare

@github-actions github-actions released this 13 Oct 20:41

Fixes a price measurement bug and adds a new feature for max findings.

Mole was recording two of the four parts of itself that spend money at $0.00. A run that reported six cents really cost eight, and a --usd ceiling could not bind on the phase that makes most of the calls. If you enforce a dollar budget, re-check it against a single run before turning anything loose.

Fixes — every call is priced, whichever part of mole made it. Pricing was done by whoever happened to build the call… 143 of 185 calls and 127,054 tokens, all free. The rate table is consulted in the ledger now… Declared rates reach every role now too.

New — --max-findings sets how many findings the report is written from, which is what sets its length.

Changelog

  • e360c94 feat(output): the report's length follows the research, not a constant
  • cb0b482 fix(budget): the ledger prices every call, not every caller

Install

curl -fsSL https://raw.githubusercontent.com/lajosdeme/mole/main/install.sh | sh

Or see the README for Homebrew,
AUR, .deb and building from source.

v0.2.0

Choose a tag to compare

@github-actions github-actions released this 10 Oct 22:10

Four defects in how mole measured itself are fixed in this release. Each one
caused it to stop short, or to under-count what it spent. Together they mean a
session now does work it previously skipped — which is the point, and is also a
real change to your bill.

On the test corpus, the same Deep question before and after:

                 before    after
claims              317       661
contradictions       32       126
sources read         28        53
spend            $0.078    $0.173
ended         planner done,  max_wallclock,
              13 of 25 min   using its full clock

If you run mole on a budget, re-check your --usd figures against a single run
before turning anything loose.

Fixes

A run no longer ends holding work it chose to do. A verification pass
proposes a follow-up for every contradiction worth settling and queues only what
its budget takes. The rest were discarded — no later pass could find them,
because verification judges only unverified claims. One measured run found 32
contradictions, followed up 4, dropped 28, and reported itself finished. They
are now deferred and drained before the session can end.

An unpriced model is no longer a free one. A provider renaming what it
reports — deepseek-chat answering as deepseek-flash — made every call ledger
at $0.00, so --usd accumulated nothing and could never bind. The configured
model's rate now prices an unrecognised response. Related: mole ships rates for
every model it can select itself, and you can declare your own (see below).

Tokens a provider billed and did not explain are now charged. Gemini's
OpenAI-compatible endpoint reports internal thinking in total_tokens and in
neither part, so a call billed at 139 tokens was ledgered at 26 — a fivefold
undercount. Reported and diagnosed by @gavmor in #1.

The tool-call ceiling was cutting runs roughly in half. It assumed a lead
costs four calls per source; measured, the figure is a median of 43 and a
maximum of 62. It is now an explicit backstop, settable with
--max-tool-calls.

A limit of zero means every row. Six store queries read 0 as "use a
default" — 500 claims, 200 leads — so a call meaning "give me everything"
returned the first 500. Exports of large sessions were silently truncated, and
the verifier's pairing pool excluded the newest claims from ever being compared.

New

Declared model rates. A pricing block in the config takes dollars per
million tokens, overriding built-ins, so any provider — OpenRouter included — is
budgetable. mole doctor reports whether every configured model has a rate.

A session clock you can set once. max_wallclock in config, and a clamped
timeout on research.report over MCP. Requested in #2.

stopped_because on the session and in the export. budget_exhausted was
reported for five different endings, only one of them about money. The reason is
now recorded beside the status.

Follow-up lineage in the export. A sub-question spawned to settle a
contradiction carries verify_depth and root_claim_id.

Builds on Windows, via #6 from @ucsandman.

Install

curl -fsSL https://raw.githubusercontent.com/lajosdeme/mole/main/install.sh | sh

Or see the README for Homebrew,
AUR, .deb and building from source.

v0.1.0

Choose a tag to compare

@github-actions github-actions released this 13 Aug 15:19

Changelog

  • c696058 Clean up CLA by removing review markers and notes
  • eb6b873 Delete docs/DESIGN.md
  • 400adb1 Delete mole-architecture-sketch.md
  • 38122d8 Revise README to simplify claim verification section
  • 2f8a1dc Update README by removing unneeded sections
  • fcb9237 delete docs folder
  • d18a585 docs(dataset): M9 in the README, including the two measures that failed
  • 5ab1a53 docs(eval): adjudicator verdicts depend on batch size, measured across 261 pairs
  • a035a14 docs(eval): the adjudicator agrees with itself 80% of the time
  • ad9df1b docs(go): package docs read as documentation, not as a build log
  • cf2c180 docs(readme): toolkit mode, with what it costs
  • a0568b8 docs(toolkit): the proposal now describes the code, plus two smaller fixes
  • 9f64cd6 eval(corpus): re-record with the confirm pass on
  • dd49f65 eval(corpus): record the contradiction corpus, and bound the labelling to 159 pairs
  • 4ab93da eval(pairs): contradiction precision is 51%, and the scorer was hiding it
  • 95c525d eval: draft the contradiction-recall question set (§14.2)
  • dd8a61c feat(academic): PubMed search and resolve, with PMC full text
  • ecd3107 feat(academic): Unpaywall — DOI to a copy mole can actually read
  • 5f1bc71 feat(academic): arXiv search and resolve — tier 0 of the ladder
  • 5a7b31a feat(academic): the provider seam, and the contact-email gate (§10.3)
  • 63b8e04 feat(actors): consult Unpaywall at escalation, and correct its billing
  • 68275ff feat(actors): share the mining pipeline, and add the AcademicActor
  • d6e2b65 feat(actors): tier 1 — escalate to ranked full text when the abstract falls short
  • c92e1e9 feat(ask): research.ask — query a finished session's graph (§13)
  • eb022fd feat(budget): warm the cost estimator from this install's history
  • bd32de7 feat(cli): mole ask — query a finished session from a terminal
  • 60da7fb feat(cli): report progress during a run, so planning does not look like a hang
  • 408e983 feat(compute): CodeRunner — the boundary moves inside the container
  • 12fa1bd feat(compute): detect the sandbox, verify its flags, require neither
  • 43de428 feat(compute): register local data behind a handle that cannot write
  • 19cd98f feat(compute): statistical validity, so two means are not a finding
  • 4dc9cd6 feat(compute): the LocalComputeActor — a model chooses the question, never the SQL
  • b89c7b1 feat(compute): the aggregation gate, enforced by a type rather than by discipline
  • 9920dbd feat(compute): the audit trail as a table, and §14.3's exfil number measured
  • aaa8f4f feat(compute): the parse gate, half of it over tokens because Walk has holes
  • 802e7ad feat(daemon): mole serve — unix socket, peer checks, graceful stop
  • 99cd343 feat(dataset): persistence, --mode dataset end to end, and mole dataset
  • 2e3080d feat(dataset): rows extracted in the same call as claims, verified the same way
  • 3a62118 feat(dataset): the cross-source merge, and the numbers that justify it
  • d78cd14 feat(dataset): the schema, the row, and two output formats that are not equivalent
  • fe39e25 feat(dev): measure academic PDF coverage, and act on the number
  • 41e9f86 feat(doctor): check the MCP socket for real, and the config file for keys
  • 6136304 feat(eval): M6 verified live; citation accuracy no longer fails on moved offsets
  • 0e4068b feat(eval): effect-weighted accuracy, and the first labelled pair score
  • 629d61b feat(eval): measure whether the adjudicator is RIGHT, not just what it found
  • bb7bae4 feat(eval): re-judge a session's pairs with another model, and measure self-consistency
  • 443d1b2 feat(eval): score judge agreement by what it changes, not what it says
  • 4194f02 feat(eval): the dataset numbers that need no labelled data
  • 6dad9c9 feat(executor): default to four workers, and refuse a pool with a cassette
  • a7a1274 feat(executor): per-session worker pool with a replan barrier
  • 1b0f7aa feat(llm): give a reasoning model its own token allowance
  • e866060 feat(llm): make the call timeout and the adjudication batch size configurable
  • 6c27580 feat(m1): Brave and Tavily search providers, user-selectable
  • 5317b9f feat(m1): WebActor — search to cited claims, with quote verification
  • 3ba134f feat(m1): guarded fetch layer with outcome classification
  • 90bcf26 feat(m1): llm provider layer, chunker, and credential auto-detection
  • 4bea2d0 feat(m1): mole research — one lead, end to end, through the real ledger
  • 1132d1e feat(m1): readability + structured-data extractor
  • 7677b25 feat(m2): a corpus runner, so regressions can block a merge
  • 6709a1a feat(m2): cassette wiring and mole stats --fetch
  • 2f404c6 feat(m2): citation accuracy, and --always-fetch so it can be collected
  • 80585d7 feat(m2): mechanical scorer — mole eval
  • e83466c feat(m3): artifact cache — result-returning, keyed on the artifact
  • 7c5e8bc feat(m3): executor loop and the §9.5 error policy
  • a1e3e8f feat(m3): lead queue with leases and crash recovery
  • c2ac84a feat(m3): planner — rolling digest, batched replan, depth cap
  • fafa32c feat(m3): reclaim abandoned sessions at boot — the third half of §9.4
  • a270dff feat(m3): report output, and mole research on the planner loop
  • dc91f93 feat(m4): budgeted re-fetch grounding
  • 641f6df feat(m4): derived confidence from graph structure
  • 8a3d542 feat(m4): edge inference, batched and deduped from the start
  • f316589 feat(m4): lexical candidate retrieval for the claim graph
  • 2218840 feat(m4): the Verifier runs in the loop, and its caps can actually bind
  • 4da86ee feat(m4): the claim graph store, with the invariants the schema cannot hold
  • 3b447fe feat(m4): the report collapses duplicates and discloses disagreement
  • 3e12dc9 feat(mcp): mole-mcp — the disposable stdio shim
  • 672c6cb feat(mcp): the daemon speaks MCP — five tools over the socket
  • 3b881b9 feat(mcp): toolkit slice 1 — sessions, search, fetch behind --toolkit
  • c02e782 feat(mcp): toolkit slice 2 — a claim an agent's model cannot fabricate
  • 46ca552 feat(mcp): toolkit slice 3 — §12's boundary, for somebody else's model
  • fbd7b2d feat(pricing): register DeepSeek, measured rather than assumed
  • d4b2ade feat(release): five install channels from one tag
  • d665cae feat(session): background supervisor for daemon-run sessions
  • e777621 feat(stats): compare every pair of groups, with a Holm correction
  • ed40a5b feat(stats): holdout stability, the last column of §4's row
  • 2a9c95a feat(store): the document store, with a retention rule that holds (toolkit slice 0)
  • abfbd39 feat(toolkit): slice 4 — the agent judges pairs, mole keeps the graph
  • 49151cf feat(toolkit): slice 5 — a dataset the agent fills in, under the same quote rule
  • eee907a feat(verifier): confirm an edge before writing it
  • 85bbe6b feat: M0 foundations — store, ledger, cassettes, tracing
  • 8e7685a feat: a separate model for judging claims
  • c5c3837 feat: wire the AcademicActor into the CLI, behind --actors
  • c2e27d1...
Read more