Skip to content

v0.3.0 — Keyword Extraction Overhaul + Chinese Support

Choose a tag to compare

@zychenpeng zychenpeng released this 26 Feb 13:43
· 172 commits to main since this release

What's New

3-Stage Keyword Extraction Pipeline (Stage A/B/C)

  • Stage A: Hard-filter boilerplate words + Chinese→English mapping
  • Stage B: Intent anchor detection (90+ anchors across 15+ domains)
  • Stage C: Synonym expansion (80+ keys) generating 3–8 anchored queries

Full Chinese/Mixed-Language Support

  • 150+ term CHINESE_TECH_MAP covering tech, SaaS, medical, legal, education, manufacturing, agriculture, aerospace, religion, art, gaming, government, and more
  • Sorted by key length to prevent compound match collisions
  • Never returns raw Chinese — fallback strips unmapped characters
  • 98%+ pass rate across 99 Taiwanese Chinese test cases

Quality Metrics

  • 93 pytest tests (up from 73)
  • 54-idea golden evaluation set: 100% anchor hit rate, 4% junk ratio
  • Zero Chinese character leakage in query output

Bug Fixes

  • Fixed synonym expansion duplicate word bug ("redis redis", "mcp server server")
  • Fixed 追蹤 mapping: general tracking instead of infra-specific tracing
  • Fixed fallback returning raw Chinese text as search queries

New Files

  • scoring/synonyms.py — SYNONYMS dict + INTENT_ANCHORS set
  • tests/golden_ideas.json — 54-idea evaluation set (EN + ZH)
  • tests/eval_keywords.py — keyword quality evaluation script
  • tests/test_tw_chinese.py — 46 TW Chinese test cases
  • tests/test_tw_niche.py — 53 niche domain Chinese test cases

Full Changelog: https://github.com/mnemox-ai/idea-reality-mcp/blob/main/CHANGELOG.md