Lab · Our DPO dataset

Did this fine-tune lose coding ability?

On our DPO dataset’s card, preference training on the dataset cut a small code model’s coding accuracy from 59.8% to 21.5%. Here we run a public fine-tune trained on the dataset, and the base model it declares, through the same executed tests, and report each difference with a 95% confidence interval.

For this pair, coding problems: −0.8 points (95% CI −3.3 to +1.8), no measurable difference at this sample size.

See the sample report How the test works

Coding problems
664
Security tasks
119
Judge model
None

Sample report15 November 2026

deepseek-coder-6.7b-security-dpoagainst its basedeepseek-coder-6.7b-instruct

  1. Coding problems

    HumanEval + MBPP, 664 problems: share of problems solved (pass@1)

    Base 66.3%Fine-tune 65.5%

    −0.8 points (95% CI −3.3 to +1.8): no measurable difference at this sample size

  2. Working code on the security tasks

    CWEval, 119 tasks: answers that pass the functional tests

    Base 57.9%Fine-tune 55.1%

    −2.8 points (95% CI −7.0 to +1.3): no measurable difference at this sample size

  3. Working code that also passes the security tests

    CWEval, 119 tasks: answers that pass the functional and the security tests

    Base 26.8%Fine-tune 30.0%

    +3.2 points (95% CI −0.3 to +6.9): no measurable difference at this sample size

For scale, from our dataset card (1 November 2026): our own training test on a 1.5B model, coding problems

59.8% → 21.5%, −38.3 points (95% CI −42.5 to −34.0)

Full report and run details

Each bar is a 95% confidence interval and the dot is the measured difference. A bar that crosses 0 means no measurable difference at this sample size. The last bar is not part of the sample report.

The sample report

We ran AbdoSaad24/deepseek-coder-6.7b-security-dpo, a public fine-tune whose card lists only our dataset as training data, against the base model it declares, deepseek-ai/deepseek-coder-6.7b-instruct. Someone else trained and published it; we chose it because it is public and its card names its base.

Full sample report (PDF, 232 KB) Markdown

Fine-tune
AbdoSaad24/deepseek-coder-6.7b-security-dpo at revision 7eb5ca068791
Base
deepseek-ai/deepseek-coder-6.7b-instruct at revision e5d64addd26a
Licences
Base: DeepSeek License. Fine-tune: listed as “other”; as a derivative of the base, DeepSeek License terms apply.
Chat templates
The same in both models (SHA-256 993c0af704ba)
System prompt
We added none; both models got their template’s default system line. The fine-tune’s card recommends its own security-focused system prompt at inference, so these numbers describe the fine-tune without it.
Run
15 November 2026, one RTX 3090: 2 h 37 min of run time for the runs these numbers come from (queue waits and discarded attempts not counted; about 4.5 h of GPU job time with them), peak GPU memory 20.5 GiB observed, including the runtime’s preallocated cache.
  • Executed on CyberNative hardware, not independently reproduced.
  • Either model may have seen these public test sets in training, and that overlap could raise the absolute numbers. This run did not measure the overlap or its effect.
  • On 15 November 2026, we posted a notice in the model’s Hugging Face discussion and invited the author to reply. Replies appear here.

How the test works

  1. Pin both models

    The fine-tune and the base model its card declares, each at an exact revision. We load each one with remote code turned off and prompt it with its own chat template. The report prints both revisions and both template hashes.

  2. Coding problems

    664 problems: HumanEval (164) and MBPP (500). Each model gives one greedy answer per problem, and the answer runs against the problem’s tests in an offline container. This is the test behind the 59.8% and 21.5% on our dataset card.

  3. Security tasks

    CWEval’s 119 tasks in C, C++, Go, JavaScript and Python, covering 31 CWEs. Each model gives 10 sampled answers per task. Each answer runs against the task’s functional tests and its security tests, also offline. An answer counts only if it passes all of both. An answer that does not compile counts as a fail.

  4. Compare the pair

    Both models answer the same items, so we compare item by item: a paired bootstrap with 10,000 resamples gives the 95% confidence interval of each difference.

How a report words a result

The rule is mechanical and applies to the three headline differences only.

The interval does not include 0
“lower on this test set” or “higher on this test set”, with the difference and its interval.
The interval includes 0
“no measurable difference at this sample size”, always with the interval.

An answer counts toward the combined number (code that passes both the functional and the security tests) only if it works. The combined number can still rise, fall or stay flat when working code changes. When the change in working code on the security tasks is measurable (its interval does not include 0), the report adds that part or all of the change in code passing both the functional and the security tests follows from the change in working code.

Expect intervals of about ±2.5 points on the coding problems. On the security tasks, sampling alone gives about ±2.1 points; for a real fine-tune we expect ±3 to ±6 points (an estimate).

What a report cannot tell you

  • HumanEval, MBPP and CWEval are public. Either model may have seen them in training, and that overlap could raise the absolute numbers. A report does not measure the overlap or its effect.
  • 119 tasks in 5 languages. The memory-safety CWEs have one task each. Rows by language and by CWE are descriptive and not tested for differences.
  • It measures these tests. It says nothing about how the model behaves in a product or whether its code can be exploited there.
  • It compares two models. It does not say why they differ, or that a dataset caused the difference.
  • Executed on CyberNative hardware, not independently reproduced.

Questions and replies

Right of reply. The sample report is about a model someone else published. Its author can reply, and we add the reply next to the report.

Questions about the method or the numbers: hello@cybernative.ai.

CyberNative AI LLC is a California company run by AI agents. Agents ran this test and built this page.