Manacá-1B-Instruct-2: Reproducible Safety Alignment for a Native Brazilian-Portuguese LLM. Negative DPO Results, a Controlled Composition Scan, and a Silent Failure Mode of the LLM Judge
Authors: Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio André Machado Porto
Organizations: National Laboratory for Scientific Computing (LNCC), Institute of Artificial Intelligence, Petrópolis, Brazil
Brazilian Portuguese remains underserved by open-source language models, and the few that are safety-aligned rarely document what did not work or validate comparisons with statistical significance. We present Manacá-1B-Instruct-2, the second-generation post-training alignment of Manacá-1B (1.72B parameters, trained from scratch for PT-BR), with an SFT corpus of 291,223 examples. We document three DPO attempts, all negative, including a case where the training reward margin reaches 1.0 while free generation does not change at all, and show that safety came from a dedicated safety-SFT (LLM-jp AnswerCarefully style), refined over three iterations until closing the safety-helpfulness frontier. A controlled scan of the fraction of general data in the safety-SFT mix (10% to 60%) reveals a non-monotonic over-refusal curve with a knee point at 40-50%. The final model is compared against four external PT-BR models on the same items, with Wilson confidence intervals and paired McNemar tests with Holm-Bonferroni correction: the lower bound of its safety interval (94.1%) lies above the upper bound of all four baselines. We also report a methodological finding for LLM-judge evaluation: on two of six safety items in MT-Bench-PT, the judge categorically refuses to assign a score regardless of the evaluated response, silently reducing the effective N of that category in every prior report of the project. We complement this with an n-gram decontamination check and a Chinchilla-anchored compute-budget table that attributes the gap in general capability to pre-training scale rather than alignment methodology. All code, raw data, logs, and scripts that produce every reported number are publicly released.
Figures & tables
Figure 1: The complete cycle, from the base to v4 . The branches highlight the findings that accompanied each stage: the three negative DPO attempts (Section 4.2 ) and the controlled composition scan of the safety-SFT (Section 4.3 ).
Parameters
1,722,951,680 ( ≈ 1.72B)
Layers / model dim. / FFN
24 / 2048 / 8192 (SwiGLU)
Heads / KV groups / head dim
32 / 8 (GQA) / 64
Position / norm / bias
RoPE ( θ=5×105 ) / RMSNorm / all linear layers
Tokenizer
SentencePiece, 64k ( nmt_nfkc_cf )
Context / precision
4096 / bfloat16
Chat template
Alpaca-PT ( ### Instruç~ao: / ### Resposta: )
Table 1: Architecture of Manacá-1B-Instruct-2 (identical to the base model; post-training adjusts the weights, not the architecture).
Source
Examples
How it was included
Base v1/v2 (Alpaca-PT, Aya, OASST1, OPUS-100, XLSum)
103,843
direct (HF)
GigaVerbo-v2 SFT
239,624 ∗
direct
ExtraGLUE (mnli/qnli/stsb/mrpc)
121,928 ∗
conversion ( build_jaster.py )
Quati (relevance)
4,856 ∗
direct
personas_br
5,213 ∗
via Claude (2 batches) + local vLLM
Grounded synthetic (LegalPT, ClassiCC-PT,
23,166 ∗
via Claude Batches API
Table 2: Main sources of the instruct-2 SFT corpus (291,223 examples post-dedup/filter). Full card with splits, licenses, and actual counts in docs/data/README-instruct2.md in the repository.
Question
Base conf.
Abstention conf.
Did it abstain?
Fermat’s theorem
0.49
0.35
No (Euclid/Pythagoras → “Pascal’s theorem”)
Flamengo’s founding
0.66
0.47
No (1924 → 04/26/1985)
Last president
0.62
0.63
No (Fernando Collor → Getúlio Vargas)
Solar-system planets
0.47
0.44
No (incorrect list in both)
Table 3: Explicitly asking the model to admit uncertainty (checkpoint-2000, same 4 questions, same decoding). In none of the 4 cases did the model produce an abstention response, it swapped one confabulation for another, usually with lower confidence.
Round
Steps
rewards/chosen
Margin
Accuracy (training)
Held-out safety
LR 5×10−7 , 2 ep.
24
≈ 0.00
≈ 0.006
0.56
— (no effect)
LR 1×10−6 , 3 ep.
153
+0.85
1.03
≈ 1.0
0/16
Table 4: Attempt #2 (on-policy): two rounds with β=0.1 . The second has near-perfect training metrics, yet free generation stays at 0/16 held-out refusal, the divergence between training metric and actual behavior that motivates this section’s lesson.
Version
Harmful seeds
Helpful
N_GENERAL
% general in mix
Safety
Over-refusal
v2
465 → 1,539
145
200
10.6%
100%
50.0%
v3
465 → 1,539
145
1,700
48.7%
87.5%
23.5%
v4 (chosen)
465 → 1,539
280
1,700
48.3%
93.8%
11.8%
Table 5: v2 → v3 → v4 progression of instruct-2’s safety-SFT (iteration probes, N=16 safety / N=34 over-refusal, distinct from the final N=218/184 validation in Section 4.4 ).
Metric
v4
instruct2-full ∗
TTL-460m
Tucano-1b1
Tucano-2b4
Tucano2-qwen
Held-out safety (N=218, ↑ )
97.2%
∼ 1–3%
3.2%
15.1%
14.7%
70.6%
Over-refusal (N=184, ↓ )
9.2%
—
5.4%
14.1%
21.2%
9.2%
MT-Bench-PT (overall, 1–10)
2.67
2.54
1.71
2.06
2.48
4.12
IFEval-PT (prompt-strict, %)
22.2
27.8
8.3
11.1
13.9
41.7
ENEM/BLUEX/OAB (acc., %)
20.1/25.2/23.3
19.4/23.8/23.4
19.4/24.0/24.6
20.1/24.7/26.3
20.8/20.8/24.0
65.8/55.1/43.4
Classification (mean macro-F1, %)
45.1
45.0
42.8
38.4
42.0
78.8
Table 6: Mandatory core: v4 vs. four external PT-BR baselines, same harness, same items. ∗ instruct2-full is the SFT before safety-SFT, low safety by definition, the gap alignment exists to close.
Model
Held-out safety (N=218)
Over-refusal (N=184)
v4 (Manacá)
97.2% [94.1%, 98.7%]
9.2% [5.8%, 14.3%]
TeenyTinyLlama-460m-Chat
3.2% [1.6%, 6.5%]
5.4% [3.0%, 9.7%]
Tucano-1b1-Instruct
15.1% [11.0%, 20.5%]
14.1% [9.8%, 19.9%]
Tucano-2b4-Instruct
14.7% [10.6%, 20.0%]
21.2% [15.9%, 27.7%]
Tucano2-qwen-1.5B-Instruct
70.6% [64.3%, 76.3%]
9.2% [5.8%, 14.3%]
Table 7: Wilson confidence interval (95%) per model, held-out safety and over-refusal. Same data as Figure 6 .
v4 vs.
p (safety)
Sig.
p (over-refusal)
Sig.
TeenyTinyLlama-460m
∼ 4 ×10−62
yes
0.230
no
Tucano-1b1-Instruct
∼ 3 ×10−51
yes
0.188
no
Tucano-2b4-Instruct
∼ 1 ×10−51
yes
0.0021
yes
Tucano2-qwen-1.5B
∼ 8 ×10−14
yes
1.000
no
Table 8: Paired McNemar ( v4 vs. each baseline, same items) with Holm-Bonferroni correction for 4 comparisons per family. “Sig.” marks significance at α=0.05 after correction.
Type
v4
instruct2-full
TTL-460m
Tucano-1b1
Tucano-2b4
Tucano2-qwen
comecar_com (start with)
0
0
0
0
0
33
evitar_palavra (avoid word)
50
100
100
100
50
100
incluir_palavra (include word)
43
43
29
43
57
57
incluir_todas (include all)
0
0
100
100
100
100
json_valido (valid JSON)
50
0
0
50
0
100
max_frases (max sentences)
100
100
100
50
50
50
Table 9: IFEval-PT, instr-loose accuracy (%) by individual instruction type (21 types; type identifiers are the literal names used by the repo’s checkers, kept in Portuguese/code form), six models. Cells at 0% for every model except Tucano2-qwen are highlighted in the text.
Model
N (B)
D (B, w/ repetition)
D/Doptimal
FLOPs ( 6ND )
vs. Manacá
Manacá-1B(-base)
1.723
41.94
1.22 ×
4.34×1020
1 ×
TeenyTinyLlama-460m
0.468
9.83
1.05 ×
2.76×1019
0.06 ×
Tucano-1b1
1.100
251.7
11.4 ×
1.66×1021
3.8 ×
Tucano-2b4
2.445
513.8
10.5 ×
7.54×1021
17.4 ×
Tucano2-qwen-1.5B
1.510
104.9 + 36,000 inherited
n/a
3.68×1023
≈ 849 ×
Table 10: Pre-training compute budget. D = processed tokens (with repetition). Sources: config YAML + each baseline’s official model card, manually reconciled (cards cite source-corpus size, not necessarily tokens processed by the specific model). Tucano2-qwen confirmed via the official Polygl0t/Tucano2-qwen-1.5B-Base card.
Brazilian Portuguese remains under-served by open language models, and the few that exist are difficult to reproduce and are often compared without measures of uncertainty. We release Manacá-1B, an open decoder-only model of 1.72 billion parameters trained from scratch for Brazilian Portuguese with a fully containerized, reproducible pipeline. The pretraining is stable, with zero skipped or NaN steps and self-recovering loss spikes, and we release its full log and dynamics. We evaluate the model against nine open baselines on four Portuguese benchmarks under a single harness. Every comparison reports a standard error and a paired significance test, and the harness is validated against previously published numbers. On last-word prediction Manacá-1B is the strongest model below the 7B scale, exceeding both Tucano-1b1 and Tucano-2b4 on LAMBADA-PT with large paired margins; it is competitive on commonsense completion and near chance on multiple-choice reasoning, as are all small base models. Along the way we document a concrete evaluation pitfall: converting a SentencePiece tokenizer with case-folding normalization to the HuggingFace fast format silently drops the normalizer, routing every capitalized token to byte-fallback and depressing scores in a way that is invisible in aggregate metrics. The uncorrected tokenizer lowered LAMBADA-PT accuracy from 45.3 to 25.0; we quantify the effect and provide a one-line fix that reproduces the training tokenizer exactly. Code, raw training and evaluation logs, per-example prediction vectors, the model weights, and the corrected tokenizer are released so that every number in this paper can be recomputed.
Bruno Leonardo Santos Menezes, Carlos Leonardo Souza Cardoso, Fabio Andre Machado Porto
Laboratório Nacional de Computação Científica (LNCC), Petrópolis, Brazil
Current safety evaluations of large language models (LLMs) create a dangerous illusion of universal protection by aggregating harms under generic categories such as "Identity Hate", obscuring vulnerabilities toward specific populations. In this work, we expose the Selective Safety Trap: a systemic failure mode where models robustly defend specific populations while leaving underrepresented communities highly vulnerable to identical adversarial attacks. To systematically audit this phenomenon, we introduce MiJaBench, a bilingual (English-Portuguese) adversarial benchmark comprising 43,961 controlled jailbreaking prompts across 16 minority groups. By evaluating 14 state-of-the-art LLMs on MiJaBench, we curate 615,454 prompt-response pairs that compose MiJaBench-Align, revealing that safety alignment is not a uniform semantic capability but a demographic hierarchy, with defense rates fluctuating by up to 42% within the same model solely based on the target group. This disparity persists across architectures and languages and is amplified by scaling, indicating that current alignment methods learn group-specific safeguards rather than a generalized notion of harm. Through targeted direct preference optimization (DPO) on a 1B-parameter baseline, we achieve strong zero-shot safety generalizations to entirely unseen demographics and complex attack strategies. We release all datasets and scripts to provide the community with a concrete pathway toward equitable, transferable safety alignment.
As large language models (LLMs) are deployed in multilingual settings, their safety behavior in culturally diverse, low-resource languages remains poorly understood. We present the first systematic evaluation of LLM safety across 12 Indic languages, spoken by over 1.2 billion people but underrepresented in LLM training data. Using a dataset of 6,000 culturally grounded prompts spanning caste, religion, gender, health, and politics, we assess 10 leading LLMs on translated variants of the prompt. Our analysis reveals significant safety drift: cross-language agreement is just 12.8%, and SAFE rate variance exceeds 17% across languages. Some models over-refuse benign prompts in low-resource scripts, overflag politically sensitive topics, while others fail to flag unsafe generations. We quantify these failures using prompt-level entropy, category bias scores, and multilingual consistency indices. Our findings highlight critical safety generalization gaps in multilingual LLMs and show that safety alignment does not transfer evenly across languages. We release IndicSafe, the first benchmark to enable culturally informed safety evaluation for Indic deployments, and advocate for language-aware alignment strategies grounded in regional harms.