AI vs Human Baseline on PostTrainBench: Why Manifold Traders Are Split at 45%

AI vs the human baseline on PostTrainBench: why Manifold traders are split at 45%. A niche Manifold market asking whether any AI model will beat the human baseline on PostTrainBench before 1 October 2026 has jumped into the platform’s most-active tier, sitting at 45% implied probability of YES as of August 18, 2026. The move follows a real event: San Francisco lab Intology said its automated research system, Locus, matched and in one setting exceeded human-level post-training on the benchmark. The catch is that the fine print of both the benchmark and the market leaves the question genuinely unresolved.

What the PostTrainBench market is

The market lives on Manifold, the play-money prediction platform, and was created by AI writer Zvi Mowshowitz. The question is blunt: “AI beats human baseline on PostTrainBench by 1 Oct 2026?” Its written resolution criteria say it settles YES if any large language model released before 1 October 2026 exceeds the human baseline of 61.77% on the official leaderboard at posttrainbench.com, with a short grace period to at most 1 November for any model released close to the deadline.

Because Manifold runs on its internal “mana” currency rather than dollars, this is not a cash betting venue. It is closer to a public forecasting board where the price is a crowd-sourced probability. If you are new to how that price becomes a forecast, our explainer on how prediction markets work covers the mechanics, and our note on implied probability explains why a 45% quote is read as a 45% chance.

The odds, and what 45% implies

As of August 18, 2026, the market shows a 45% implied probability of YES on Manifold. Activity has been heavy for a specialized technical question: the market page lists 374 trades across 71 holders, which is why it surfaced near the top of the platform’s 24-hour volume in our source scan, with roughly 77 unique bettors active over the day.

A 45% quote is a coin flip that leans slightly toward “no.” Read plainly, the crowd thinks it is nearly as likely as not that some model clears the bar in the next six weeks, but is not convinced. That is an unusual place for a benchmark market to sit. Most “will AI hit threshold X by date Y” questions trade near the extremes because the gap is either obviously closable or obviously not. The middling price here signals disagreement about a specific, recent result rather than vague uncertainty.

What is driving the move: Intology’s Locus

The catalyst is Intology, a San Francisco AI research lab, which in early August 2026 published results for Locus, its long-horizon automated research system. In a post titled “Scaling Automated Post-Training,” Intology reported that Locus is state of the art on PostTrainBench and, given enough compute, post-trains Qwen3 base models that surpass the official human instruction-tuned Qwen3-1.7B release. On social media the lab put it directly: “The models are improving the models.”

The numbers matter here. On the standard benchmark, Locus paired with the Opus 5 model scored 44.7%, versus 34.1% for Opus 5 running without the Locus harness, and ahead of a Fable 5 configuration at 41.8%. On an extended-compute variant the lab calls PostTrainBench+, using more than 4,000 hours of H100 GPU time, Locus reached 51.6%, which clears the roughly 51% human reference on that setup. Intology says the results were externally verified by the PostTrainBench authors and passed contamination and cheating checks. Jack Clark’s Import AI newsletter, which flagged the work, framed the takeaway as “AI systems are about to start building themselves,” while cautioning that the standard human baseline had not yet been beaten.

So the market moved because a credible lab showed a system crossing a human line in one configuration, while leaving the exact line named in the resolution criteria untouched.

What PostTrainBench actually measures

PostTrainBench, built by the AISA research group, is not a chatbot quiz. It measures whether command-line AI agents can automate post-training, the stage that turns a raw base model into a useful assistant. An agent is handed a small base language model in the 1-to-4 billion parameter range, a single H100 GPU, and a 10-hour window, and must improve the model however it sees fit, with no starter code, no supplied data, and no human help.

Scores are aggregated across four base models (Qwen3-1.7B, Qwen3-4B, SmolLM3-3B and Gemma-3-4B) and seven downstream tests spanning math (AIME 2025, GSM8K), code (HumanEval), function calling (BFCL), graduate-level science (GPQA) and open-ended writing (Arena-Hard), with open-ended answers judged by a GPT-5-mini grader. The accompanying paper, “PostTrainBench: Can LLM Agents Automate LLM Post-Training?” (arXiv 2603.08640), was accepted at ICLR and ICML 2026. Its headline finding: autonomous agents averaged 23.2%, far above untuned base models near 7.5% but well short of the expert human tuning baseline near 51%.

The scoreboard so far

The trajectory is steep, which is part of why the market is not pinned to zero. Every figure below is a reported PostTrainBench weighted-average score.

The pattern is a roughly fivefold jump in about a year, from under 10% to the mid-40s on the standard benchmark. Traders are extrapolating that curve against a hard mid-October date.

The resolution wrinkle: 61.77% versus 51.14%

Here is why the price is stuck in the middle rather than racing up. The market rules name a 61.77% human baseline, but the current official leaderboard reference, the “Official Instruct Models” line on PostTrainBench v1.1, sits around 51.14%. Comments on the market itself flag the discrepancy. Locus cleared roughly 51% only on the extended-compute PostTrainBench+ variant, not on the standard single-H100, 10-hour benchmark that the leaderboard tracks, and 51.6% is still nowhere near 61.77%.

That leaves three open questions a bettor has to price: which baseline number the market ultimately settles against, whether a compute-scaled harness result counts toward a question phrased about a “model released,” and whether any entrant clears the relevant bar on the standard setup before 1 October. Each of those is live, which is exactly the kind of ambiguity that produces a 45% quote instead of an 85% one.

Who is involved

The named actors are concrete. Zvi Mowshowitz, a widely read AI commentator, created and will interpret the market. Intology, the lab behind Locus, previously introduced Zochi, which it billed as the first AI system to pass peer review at a top-tier venue, and Locus itself, which the company says reached superhuman results on the RE-Bench AI research benchmark and beat 89.5% of human competitors across live Kaggle contests. The benchmark comes from the AISA group, whose authors verified Intology’s run. Frontier models in the mix include Anthropic’s Opus family and the Fable 5 and Opus 5 systems referenced in the leaderboard data. Independent trackers such as Epoch AI and llm-stats.com maintain public PostTrainBench pages.

Broader context for a tech and finance reader

The reason this obscure market matters beyond forecasting hobbyists is the underlying claim: AI systems that can improve AI systems. Post-training is skilled, expensive human labor at every major lab. A benchmark that measures whether an autonomous agent can do that job, and a result showing one crossing a human line under heavy compute, speaks directly to the recursive self-improvement debate and to the capital-expenditure race in AI. It also lands amid a crowded frontier-model field; our comparison of Claude Fable 5, Grok 4.6 and DeepSeek V4 maps the systems now being pointed at problems like this one.

For markets watchers, this is also a case study in how quickly a specialized forecast can reprice on a single lab announcement, and in why the wording of a contract matters as much as the underlying science. It echoes other AI-adjacent prediction markets we have covered, including a Manifold market pricing an AI outcome at 95%, where the resolution language did much of the heavy lifting.

What to watch next

Three things will move this market before the deadline. First, any new leaderboard entry on the standard PostTrainBench setup that pushes past the mid-40s toward the human line. Second, a public clarification from the market creator on which baseline number, 61.77% or roughly 51%, governs resolution, and whether harness-plus-model systems qualify. Third, any additional lab, beyond Intology, replicating or exceeding the Locus result on the standard benchmark. Jack Clark’s stated guess is that the current human baseline will be beaten before the end of 2026; the open question the market prices is whether that happens in the next six weeks rather than the next five months.

Frequently asked questions

Is real money on this market? No. Manifold uses its internal “mana” currency, so the 45% figure is a crowd forecast rather than a cash bet. Dollar-denominated venues such as Kalshi and Polymarket are covered in our Kalshi versus Polymarket comparison.

Has an AI already beaten the human baseline? Not on the standard leaderboard. Intology’s Locus reached about 51.6% on an extended-compute PostTrainBench+ variant, which clears the roughly 51% human reference there, but that is below the 61.77% figure named in the market rules and is not the standard single-H100, 10-hour benchmark.

Why is the price near 45% instead of higher or lower? Because the outcome hinges on unresolved wording as much as on capability. There is genuine ambiguity over which baseline number counts and whether a compute-scaled harness qualifies, on top of a fast-rising but incomplete capability curve.

Who runs the benchmark? The AISA research group publishes PostTrainBench and its leaderboard, and the associated paper was accepted at ICLR and ICML 2026.

The Bottom Line

Sources

• Manifold — AI beats human baseline on PostTrainBench by 1 Oct 2026?
• Intology — Scaling Automated Post-Training (Locus results)
• Import AI 468 (Jack Clark) — PostTrainBench+ and RSI ideas
• Epoch AI — PostTrainBench benchmark page
• arXiv — PostTrainBench: Can LLM Agents Automate LLM Post-Training?
• Morning Tick — Intology Launches Locus, an AI Research Agent
• llm-stats.com — PostTrainBench Leaderboard

Prediction markets carry risk and are provided here for information only. This is not investment, trading, or betting advice. Availability is restricted by jurisdiction: Polymarket is not available to US persons, and Kalshi is a CFTC-regulated US exchange; Manifold operates with play-money “mana.” Age restrictions of 18+ or 21+ apply where relevant. If gambling is a problem for you or someone you know, help is available at 1-800-GAMBLER.

Morgan Reeves

Prediction Markets Editor

Morgan Reeves covers prediction markets, sports analytics, and decentralized forecasting platforms. Background in quantitative finance and behavioral economics. Tracks Kalshi, Polymarket, PredictIt, and emerging US-regulated event-contract platforms since 2022.

View all articles