Hugging Face Bio has released FineDNA, the training data behind Carbon-X, its DNA-focused adaptation of Qwen3-14B. The November 1, 2026 announcement covers all three Carbon-X training stages: mid-training, supervised fine-tuning, and reinforcement learning. The dataset is ungated and released under Apache 2.0, with training recipes linked from its Hugging Face dataset page.
The release draws a clear line between what's available and what was actually used. FineDNA contains 514.30 billion mid-training tokens, but Carbon-X read 302 billion tokens through a weighted mixture. For the later stages, Hugging Face Bio publishes the actual SFT and RL input rows in their training order.
That makes FineDNA useful beyond downloading another biology dataset. Model builders can inspect which sources received training time, reproduce the documented mixture, or change one part of the adaptation pipeline while keeping the others fixed.
What FineDNA Contains
FineDNA organizes Carbon-X's training inputs into named dataset configurations. Mid-training has 23 source-specific configurations prefixed with mid_. Supervised fine-tuning, or SFT, includes a combined mixture and separate configurations for eight domains. Reinforcement learning, or RL, has its own combined prompt set.
The dataset card gives the following breakdown:
| Stage | Dataset configurations | Released data | Starting checkpoint |
|---|---|---|---|
| Mid-training | 23 mid_* configurations | 514.30 billion tokens available; 302 billion read during training | Qwen/Qwen3-14B |
| Supervised fine-tuning | sft_mix and eight domain-specific configurations | 203,923 conversations; 230.3 million tokens | HuggingFaceBio/Carbon-X-Base |
| Reinforcement learning | rl_mix | 98,433 prompts; 61.1 million tokens | HuggingFaceBio/Carbon-X-SFT |
Mid-training continues training the pretrained Qwen model on a mixture containing DNA, biology text, general English, and instruction data. SFT then trains on example conversations. The RL stage uses the published prompt mixture with a GRPO trainer, according to the card.
These figures are publisher-reported. This article analyzes the release's documentation and structure, not independently measured model performance.
The FineDNA collection also groups the dataset with Carbon-X-Base and Carbon-X-SFT. Those intermediate checkpoints matter: a team studying post-training can start from the relevant checkpoint without first repeating the 302-billion-token mid-training run.
The Mid-Training Weights Are the Recipe
FineDNA's largest component is a collection of source pools, not one ready-made, sequential training file. Hugging Face Bio says Megatron sampled those pools using the published weights.
Each mid-training configuration includes a text column, an optional source document ID, a source label, and other metadata encoded as a JSON string. The card instructs builders to tokenize the text with the Carbon-X tokenizer and use the documented sampling weights.
At the category level, the mixture looks like this:
| Category | Available tokens | Share of training | Tokens Carbon-X read |
|---|---|---|---|
| DNA | 181.96 billion | 55% | 166.1 billion |
| Biology text | 15.75 billion | 15% | 45.3 billion |
| English | 310.22 billion | 20% | 60.4 billion |
| Instruction data | 6.36 billion | 10% | 30.2 billion |
| Total | 514.30 billion | 100% | 302.0 billion |
Figures are as reported in the FineDNA dataset card; category totals are rounded.
The English pool is the largest available category, but DNA accounts for most of the training exposure. Biology text and instruction data receive substantially more exposure than their available token counts.
The arithmetic shows why copying the files alone would miss a consequential part of the recipe:
- Biology text: 45.3 billion tokens read from a 15.75-billion-token pool, equivalent to about 2.9 times its token volume.
- Instruction data: 30.2 billion tokens read from a 6.36-billion-token pool, about 4.7 times its token volume.
- English: 60.4 billion tokens read from 310.22 billion available, roughly one-fifth of the pool's token volume.
These ratios describe aggregate exposure, not an identical number of passes over every document. The card explicitly says some configurations were read multiple times while others were only partially consumed.
My view is that the sampling weights are the most consequential part of this release. A builder who concatenated all 514.30 billion tokens and trained once through them would give the model a substantially different balance of material. FineDNA makes that difference visible and actionable.
The configurations also contain the processed source data Carbon-X sampled from, not necessarily entire upstream datasets. The card identifies the DNA inputs as filtered versions of GENERator and OpenGenome2. Reproducing the published mixture therefore means using these released configurations and their weights, not substituting unfiltered upstream downloads.
How to Load FineDNA
The dataset card provides examples using Hugging Face's Python datasets library. It recommends streaming the mid-training configurations because, together, they occupy about 950 GB.
With Python and datasets installed, the documented access pattern is:
from datasets import load_dataset
# Stream one mid-training source.
mid = load_dataset(
"HuggingFaceBio/FineDNA",
"mid_dna_mrna",
split="train",
streaming=True,
)
# Load the supervised fine-tuning mixture.
sft = load_dataset(
"HuggingFaceBio/FineDNA",
"sft_mix",
split="train",
)
# Load the reinforcement-learning prompt mixture.
rl = load_dataset(
"HuggingFaceBio/FineDNA",
"rl_mix",
split="train",
)
This example follows the published loading instructions. It accesses the data; it does not launch training. Streaming lets a program iterate through the selected mid-training source without downloading the entire mid-training collection first.
For SFT, builders can also load a domain-specific configuration such as sft_chat. That provides a straightforward starting point for examining a subset before committing to the full mixture.
The training implementation lives in the Hugging Face Carbon repository. Its carbon-x/ directory contains the mid-training, SFT, and RL recipes, alongside DNA-understanding evaluations. The repository instructs users to run commands from the relevant model directory because releases have their own dependencies and recipes.
What the Release Makes Reproducible
FineDNA exposes different levels of detail at different stages.
For SFT and RL, the card says sft_mix and rl_mix contain the rows the trainers used, in their original order. That is more specific than publishing a list of contributing datasets: researchers can inspect the actual conversation mixture and prompt sequence.
Frequently Asked Questions
3 questions
1Is FineDNA the Same as Carbon-X?
No. FineDNA is the training dataset. Carbon-X is the model trained through the documented mid-training, SFT, and RL stages.
