Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge

Bruno Leonardo Santos Menezes; Carlos Leonardo Souza Cardoso; Fabio André Machado Porto

cs.CLOct 14, 2026

Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge

Authors: Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado Porto

Organizations: National Laboratory for Scientific Computing (LNCC), Institute of Artificial Intelligence, Petrópolis, Brazil

Abstract

Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples. We document three DPO attempts, all negative, including a case where the training reward margin reaches 1.0 while free generation does not change at all, and show that safety came from a dedicated safety-SFT (LLM-jp AnswerCarefully style), refined over three iterations until closing the safety-helpfulness frontier. A controlled scan of the fraction of general data in the safety-SFT mix (10% to 60%) reveals a non-monotonic over-refusal curve with a knee point at 40-50%. The final model is compared against four external PT-BR models on the same items, with Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction: the lower bound of its safety interval (94.1%) lies above the upper bound of all four baselines. We also report a methodological finding for LLM-judge evaluation: on two of six safety items in MT-Bench-PT, the judge categorically refuses to assign a score regardless of the evaluated response, silently reducing the effective N of that category in every prior report of the project. We complement this with an n-gram decontamination check and a Chinchilla-anchored compute-budget table that attributes the gap in general capability to pre-training scale rather than alignment methodology. All code, raw data, logs, and scripts that produce every reported number are publicly released.

Figures & tables

Panel of Figure 1 — Figure 1: The complete cycle, from the base to v4 . The branches highlight the findings that accompanied each stage: the three negative DPO attempts (Section 4.2 ) and the controlled composition scan of the safety-SFT (Section 4.3 ).

Parameters	1,722,951,680 ( $\approx$ 1.72B)
Layers / model dim. / FFN	24 / 2048 / 8192 (SwiGLU)
Heads / KV groups / head dim	32 / 8 (GQA) / 64
Position / norm / bias	RoPE ( $\theta{=}5{\times}10^{5}$ ) / RMSNorm / all linear layers
Tokenizer	SentencePiece, 64k ( nmt_nfkc_cf )
Context / precision	4096 / bfloat16
Chat template	Alpaca-PT ( ### Instruç~ao: / ### Resposta: )

Table 1: Architecture of Manacá-1B-Instruct-2 (identical to the base model; post-training adjusts the weights, not the architecture).

Source	Examples	How it was included
Base v1/v2 (Alpaca-PT, Aya, OASST1, OPUS-100, XLSum)	103,843	direct (HF)
GigaVerbo-v2 SFT	239,624 ∗	direct
ExtraGLUE (mnli/qnli/stsb/mrpc)	121,928 ∗	conversion ( build_jaster.py )
Quati (relevance)	4,856 ∗	direct
personas_br	5,213 ∗	via Claude (2 batches) + local vLLM
Grounded synthetic (LegalPT, ClassiCC-PT,	23,166 ∗	via Claude Batches API

Table 2: Main sources of the instruct-2 SFT corpus (291,223 examples post-dedup/filter). Full card with splits, licenses, and actual counts in docs/data/README-instruct2.md in the repository.

Question	Base conf.	Abstention conf.	Did it abstain?
Fermat’s theorem	0.49	0.35	No (Euclid/Pythagoras $\to$ “Pascal’s theorem”)
Flamengo’s founding	0.66	0.47	No (1924 $\to$ 04/26/1985)
Last president	0.62	0.63	No (Fernando Collor $\to$ Getúlio Vargas)
Solar-system planets	0.47	0.44	No (incorrect list in both)

Table 3: Explicitly asking the model to admit uncertainty (checkpoint-2000, same 4 questions, same decoding). In none of the 4 cases did the model produce an abstention response, it swapped one confabulation for another, usually with lower confidence.

Round	Steps	rewards/chosen	Margin	Accuracy (training)	Held-out safety
LR $5{\times}10^{-7}$ , 2 ep.	24	$\approx$ 0.00	$\approx$ 0.006	0.56	— (no effect)
LR $1{\times}10^{-6}$ , 3 ep.	153	+0.85	1.03	$\approx$ 1.0	0/16

Table 4: Attempt #2 (on-policy): two rounds with

\beta{=}0.1

. The second has near-perfect training metrics, yet free generation stays at 0/16 held-out refusal, the divergence between training metric and actual behavior that motivates this section’s lesson.

Version	Harmful seeds	Helpful	N_GENERAL	% general in mix	Safety	Over-refusal
v2	465 $\to$ 1,539	145	200	10.6%	100%	50.0%
v3	465 $\to$ 1,539	145	1,700	48.7%	87.5%	23.5%
v4 (chosen)	465 $\to$ 1,539	280	1,700	48.3%	93.8%	11.8%

Table 5: v2

\to

\to

v4 progression of instruct-2’s safety-SFT (iteration probes, N=16 safety / N=34 over-refusal, distinct from the final N=218/184 validation in Section 4.4 ).

Metric	v4	instruct2-full ∗	TTL-460m	Tucano-1b1	Tucano-2b4	Tucano2-qwen
Held-out safety (N=218, $\uparrow$ )	97.2%	$\sim$ 1–3%	3.2%	15.1%	14.7%	70.6%
Over-refusal (N=184, $\downarrow$ )	9.2%	—	5.4%	14.1%	21.2%	9.2%
MT-Bench-PT (overall, 1–10)	2.67	2.54	1.71	2.06	2.48	4.12
IFEval-PT (prompt-strict, %)	22.2	27.8	8.3	11.1	13.9	41.7
ENEM/BLUEX/OAB (acc., %)	20.1/25.2/23.3	19.4/23.8/23.4	19.4/24.0/24.6	20.1/24.7/26.3	20.8/20.8/24.0	65.8/55.1/43.4
Classification (mean macro-F1, %)	45.1	45.0	42.8	38.4	42.0	78.8

Table 6: Mandatory core: v4 vs. four external PT-BR baselines, same harness, same items. ∗ instruct2-full is the SFT before safety-SFT, low safety by definition, the gap alignment exists to close.

Model	Held-out safety (N=218)	Over-refusal (N=184)
v4 (Manacá)	97.2% [94.1%, 98.7%]	9.2% [5.8%, 14.3%]
TeenyTinyLlama-460m-Chat	3.2% [1.6%, 6.5%]	5.4% [3.0%, 9.7%]
Tucano-1b1-Instruct	15.1% [11.0%, 20.5%]	14.1% [9.8%, 19.9%]
Tucano-2b4-Instruct	14.7% [10.6%, 20.0%]	21.2% [15.9%, 27.7%]
Tucano2-qwen-1.5B-Instruct	70.6% [64.3%, 76.3%]	9.2% [5.8%, 14.3%]

Table 7: Wilson confidence interval (95%) per model, held-out safety and over-refusal. Same data as Figure 6 .

v4 vs.	$p$ (safety)	Sig.	$p$ (over-refusal)	Sig.
TeenyTinyLlama-460m	$\sim$ 4 $\times 10^{-62}$	yes	0.230	no
Tucano-1b1-Instruct	$\sim$ 3 $\times 10^{-51}$	yes	0.188	no
Tucano-2b4-Instruct	$\sim$ 1 $\times 10^{-51}$	yes	0.0021	yes
Tucano2-qwen-1.5B	$\sim$ 8 $\times 10^{-14}$	yes	1.000	no

Table 8: Paired McNemar ( v4 vs. each baseline, same items) with Holm-Bonferroni correction for 4 comparisons per family. “Sig.” marks significance at

\alpha{=}0.05

after correction.

Type	v4	instruct2-full	TTL-460m	Tucano-1b1	Tucano-2b4	Tucano2-qwen
comecar_com (start with)	0	0	0	0	0	33
evitar_palavra (avoid word)	50	100	100	100	50	100
incluir_palavra (include word)	43	43	29	43	57	57
incluir_todas (include all)	0	0	100	100	100	100
json_valido (valid JSON)	50	0	0	50	0	100
max_frases (max sentences)	100	100	100	50	50	50

Table 9: IFEval-PT, instr-loose accuracy (%) by individual instruction type (21 types; type identifiers are the literal names used by the repo’s checkers, kept in Portuguese/code form), six models. Cells at

0\%

for every model except Tucano2-qwen are highlighted in the text.

Model	$N$ (B)	$D$ (B, w/ repetition)	$D/D_{\text{optimal}}$	FLOPs ( $6ND$ )	vs. Manacá
Manacá-1B(-base)	1.723	41.94	1.22 $\times$	$4.34\times 10^{20}$	1 $\times$
TeenyTinyLlama-460m	0.468	9.83	1.05 $\times$	$2.76\times 10^{19}$	0.06 $\times$
Tucano-1b1	1.100	251.7	11.4 $\times$	$1.66\times 10^{21}$	3.8 $\times$
Tucano-2b4	2.445	513.8	10.5 $\times$	$7.54\times 10^{21}$	17.4 $\times$
Tucano2-qwen-1.5B	1.510	104.9 + 36,000 inherited	n/a	$3.68\times 10^{23}$	$\approx$ 849 $\times$

Table 10: Pre-training compute budget.

D

= processed tokens (with repetition). Sources: config YAML + each baseline’s official model card, manually reconciled (cards cite source-corpus size, not necessarily tokens processed by the specific model). Tucano2-qwen confirmed via the official Polygl0t/Tucano2-qwen-1.5B-Base card.

Portuguese Language Models

See all 18 papers

Manacá-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation
Aug 31, 2026Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andre Machado PortoLanguage Model Pretraining Portuguese Language Models
Laboratório Nacional de Computação Científica (LNCC), Petrópolis, Brazil
Safety Is Not Universal: The Selective Safety Trap in LLM Alignment
Jan 7, 2026Iago Alves Brito, Walcy Santos Rezende Rios, Julia Soares Dollis +2LLM Safety Benchmarks LLM Alignment
Advanced Knowledge Center for Immersive Technologies (AKCIT) Federal University of Goiás (UFG)
IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia
Mar 18, 2026Priyaranjan Pattnayak, Sanchari ChowdhuriLanguage Model Safety Evaluation LLM Safety Benchmarks
Oracle America Inc.

Aug 31, 2026J/K move · Enter open · S save

Manacá-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation

Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andre Machado Porto

Laboratório Nacional de Computação Científica (LNCC), Petrópolis, Brazil

Language Model Pretraining Portuguese Language Models LLM Evaluation Open-Weight Language Models Language Modeling

Open paper AskPDF arXiv

Brazilian Portuguese remains under-served by open language models, and the few that exist are difficult to reproduce and are often compared without measures of uncertainty. We release Manacá-1B, an open decoder-only model of 1.72 billion parameters trained from scratch for Brazilian Portuguese with a fully containerized, reproducible pipeline. The pretraining is stable, with zero skipped or NaN steps and self-recovering loss spikes, and we release its full log and dynamics. We evaluate the model against nine open baselines on four Portuguese benchmarks under a single harness. Every comparison reports a standard error and a paired significance test, and the harness is validated against previously published numbers. On last-word prediction Manacá-1B is the strongest model below the 7B scale, exceeding both Tucano-1b1 and Tucano-2b4 on LAMBADA-PT with large paired margins; it is competitive on commonsense completion and near chance on multiple-choice reasoning, as are all small base models. Along the way we document a concrete evaluation pitfall: converting a SentencePiece tokenizer with case-folding normalization to the HuggingFace fast format silently drops the normalizer, routing every capitalized token to byte-fallback and depressing scores in a way that is invisible in aggregate metrics. The uncorrected tokenizer lowered LAMBADA-PT accuracy from 45.3 to 25.0; we quantify the effect and provide a one-line fix that reproduces the training tokenizer exactly. Code, raw training and evaluation logs, per-example prediction vectors, the model weights, and the corrected tokenizer are released so that every number in this paper can be recomputed.

Abstract

Portuguese Language Models

Explore similar work

Manacá-1B: An Open, Reproducible Brazilian-Portuguese Language Model and a Tokenizer-Aware, Paired Evaluation

Safety Is Not Universal: The Selective Safety Trap in LLM Alignment

IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia