Superseded: this model is from an earlier Jeff release and is no longer updated. The current release is Jeff v1.3: mstrasser/jeff-adapter-legal-clauses. This v1.2 adapter loads only on the v1.2 base.

Jeff-Qwen3.5-0.8B-legal-clauses

Contract clause types: labels a single contract provision with its clause type, such as governing law, notices or confidentiality.

Give it one contract provision and the LEDGAR clause types as options (up to 100, each with a one-line description), and it returns a probability for each type, such as governing law, notices or confidentiality. It was trained on provisions from US SEC filings. It is a LoRA adapter for Jeff-Qwen3.5-0.8B v1.2, a small open decision model: it answers in one forward pass, with no generated text to parse.

Results

Test set: LEDGAR test split, 9,895 rows. The official LEDGAR test split, as distributed in LexGLUE; never trained on. It is included in this repo as test.jsonl.

Calibration error is the expected calibration error over 15 confidence bins, after each model's fitted temperature: the average gap between how confident the model is and how often it is right. Lower is better. The untrained Qwen3.5-0.8B is read out from its own scores for the option letters.

Against Qwen3.8-27B

Here Jeff + legal-clauses answers first. Below a confidence threshold, the query is also sent to Qwen3.8-27B and the 27B's answer is used. Each adapter's threshold is the fastest one that still beats the 27B alone by at least 1 point of accuracy on the adapter's calibration rows (not the test rows).

The threshold is 0.00: on the calibration rows the adapter alone already beat the 27B by more than a point, so no query is passed on. The route is 35.5× faster than the 27B alone; each prompt lists all 100 clause types, which makes these the longest prompts of the nine adapters.

How this was measured: a fixed random sample of 500 held-out test rows, the same rows for both routes, on an Apple M4 Max (128 GB) with both models in MLX, one at a time. Qwen3.8-27B ran with 8-bit weights, prompted, with step-by-step reasoning off. Times are the mean per query over the whole sample, from prompt to answer. This is a 500-row sample, not the full test set.

Speed and memory

One request at a time on an NVIDIA RTX PRO 6000: 675 requests per setting, 75 test prompts of each of the nine adapters, interleaved. Measured for the adapters as a group, not for each adapter, so treat these as typical figures:

Jeff's speed and memory on the Mac: not measured yet.

Where it goes wrong

On the full test set:

• Most errors are between LEDGAR clause types that mean nearly the same thing: applicable_laws against governing_laws, defined_terms against definitions, tax_withholdings against withholdings, integration against entire_agreements, waivers against no_waivers, modifications against amendments. If you can merge such twins in your own list, do.
• general (catch-all provisions) is weak too, as it has no clear topic.

Commonest confusions (right answer → chosen instead, rows):

Weakest answers (answers with at least 20 test rows; 100 answers in all):

Harder test. Novel wording: test clauses whose wording is unlike any training clause, 7,343 rows:

On unfamiliar wording the adapter drops from 85.7% to 83.2%. Contracts unlike US SEC filings: not measured yet.

When to use it

• You have a contract split into provisions and want each one labelled with a clause type.
• Your contracts are commercial or employment agreements written in English, like those filed with the US SEC.
• You can use the 100 LEDGAR clause types, or a subset of them, as your options.

When not to use it

• Your contracts are not in English, or follow a legal system very different from the US. All training text comes from US SEC filings.
• You need legal advice or a judgement on whether a clause is fair or enforceable. The adapter only names the clause type.
• You pass a whole contract at once. It was trained on one provision at a time.
• You need your own clause types. Training used the LEDGAR labels only.

How to use it

The adapter runs with Jeff's server on the Jeff-Qwen3.5-0.8B v1.2 base. One server holds the base and any number of adapters; each request picks one by name with "model": "legal-clauses".

git clone https://github.com/firelex/jeff && cd jeff
uv sync --no-default-groups --extra lora          # add --extra cuda on NVIDIA GPUs, --extra mac on Apple silicon
uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B --revision v1.2 --local-dir Jeff-Qwen3.5-0.8B-v1.2
uv run --no-default-groups hf download mstrasser/Jeff-Qwen3.5-0.8B-legal-clauses --local-dir adapters/legal-clauses
JEFF_CHECKPOINT=Jeff-Qwen3.5-0.8B-v1.2 JEFF_ADAPTERS=adapters PORT=8765 \
  uv run --no-default-groups --extra lora jeff-serve
# add JEFF_BACKEND=mlx and --extra mac for Apple silicon

Every folder in adapters/ is served under its folder name. Add or replace an adapter while the server runs with POST /v1/adapters/reload. The server checks the base weights' checksum, so the adapter loads only on the exact v1.2 base.

from jeff import Client

legal_clauses = Client("http://localhost:8765", model="jeff-latest").with_model("legal-clauses")

Request format

State (the situation):

Questions:

• clause_type (choice): Which type of clause the provision is, judged by its main topic. Options: 100 LEDGAR clause types, each with a snake_case key such as governing_laws and a one-line description. The full list is LEDGAR in descriptions.py in the source.

Rules:

• Use the option keys and descriptions from descriptions.py; the adapter was trained on them.
• Several clause types are close in meaning (for example assignments and assigns). Expect probability to be shared between them.
• Use the instructions below word for word; the adapter was trained mostly on them.

Example

{
  "model": "legal-clauses",
  "state": {
    "document": "A commercial contract filed with the U.S. Securities and Exchange Commission (SEC)",
    "provision": "This Agreement shall be governed by and construed in accordance with the laws of the State of Delaware, without regard to its conflict of laws principles."
  },
  "questions": {
    "clause_type": {
      "type": "choice",
      "instructions": "Which type of clause is this contract provision? Choose the clause type that best matches the main topic of the provision.",
      "criteria": {
        "governing_laws": "Governing law: which state's or country's law governs the contract.",
        "notices": "Notices: how formal notices must be given, and to which addresses.",
        "confidentiality": "Confidentiality: keeping information secret and limits on using or disclosing it.",
        "terminations": "Termination: how and when the contract or employment can be ended, and what follows.",
        "assignments": "Assignments: whether and how a party may transfer its rights or duties under the contract to someone else.",
        "severability": "Severability: if one provision is invalid, the rest of the contract remains in force.",
        "entire_agreements": "Entire agreement: the contract is the whole agreement and replaces all earlier agreements on the subject.",
        "indemnifications": "Indemnification: one party must compensate and defend the other against losses and claims.",
        "counterparts": "Counterparts: the contract may be signed in separate copies, including electronic signatures, which together form one agreement.",
        "waiver_of_jury_trials": "Waiver of jury trial: the parties give up the right to a jury trial in disputes.",
        "amendments": "Amendments: how the contract can be amended, usually only in writing signed by the parties."
      }
    }
  }
}

curl -s localhost:8765/v1/systemone -H 'content-type: application/json' -d @request.json

The answer holds a probability for each option of each question, the chosen option and a confidence. (A recorded response to this example is not published yet.)

Training

The replay slice mixes in some of the base model's own training data so the adapter keeps the base's general skills. The scored checkpoint is the one at the end of the epoch.

Data and provenance

Sources:

Training data: built from the public sources above. A script to rebuild our rows from them will follow. The test set is included in this repo as test.jsonl.

Test set: test.jsonl, 9,895 rows: The official LEDGAR test split, as distributed in LexGLUE; never trained on.

Calibration set: calibration.jsonl, 958 rows: the rows the threshold for the comparison with Qwen3.8-27B is chosen on; never trained on. It is byte for byte the set the adapter was scored on.

Quality review. An independent reviewer checked the training, calibration and test files for shortcuts, duplicates, leaks between splits and junk before training. The report is at jeffhub.ai/adapters/legal-clauses/qa-report.

Limitations

• English only.
• Tied to the Jeff-Qwen3.5-0.8B v1.2 base. It will not work on any other base or version; the server refuses it if the base weights' checksum does not match. A v1.3 long-term-support base is coming, and the adapters will be retrained on it.
• Jeff chooses between the options you give it. It does not write text or reason in several steps.
• Calibration was fitted on this adapter's own calibration rows. On very different data, check it again.
• Everything listed under When not to use it above.

Links

• Adapter page: jeffhub.ai/adapters/legal-clauses
• Code and server: github.com/firelex/jeff
• Base model: mstrasser/Jeff-Qwen3.5-0.8B (revision v1.2)

Jeff is an independent project. It uses the same request format as Jev but is not affiliated with or endorsed by TypeSafe, the makers of Jev.