cs.CLSep 29, 2026
SaveContextAdapt: Evaluating Contextual Adaptation and Value Alignment in LLMs
Organizations: Centre for AI, Department of Computer Science, University College London · Department of Computer Science and Engineering, University of Bologna
Abstract
Values such as honesty, autonomy, and confidentiality are often regarded as general principles underpinning AI alignment. However, what it means to act in accordance with these values can depend on the context in which a decision is made. In this paper, we ask whether large language models (LLMs) appropriately adapt the application of a value across professional settings, while remaining consistent when contextual changes do not alter the relevant professional norm. To study this, we introduce ContextAdapt, an evaluation framework covering honesty, autonomy, and confidentiality across medicine, law, finance, and national security. Drawing on primary-source professional and regulatory documents, we construct a value x domain framework and use this to develop scenarios testing both default professional rules and recognised exceptions. We evaluate 12 LLMs on both the actions they recommend and the justifications they provide. In our main experiment, models achieve 95.6% mean appropriateness, although the use of the correct domain-specific justification varies substantially across models, from 25.6% to 76.9%. In a separate factorial experiment, explicitly naming the professional domain and changing the role of the model have limited effect on behaviour. Varying stakes, however, reveals severe but localised failures: in some cases, models alter their responses even though the underlying professional obligation remains unchanged. In particular, perceived severity appears to act as a cue for disclosure across both honesty and confidentiality scenarios. These results show that evaluating value alignment requires us to consider not only whether models follow abstract principles, but whether they apply them appropriately across different contexts.
Figures & tables
Figure 1: Overview of the ContextAdapt evaluation framework.
Figure 2: Main dataset: full justification-type distribution by model.
Figure 3: Main dataset appropriateness (%) by model, domain and value. Models are categorised by training approach.
Appendix figures & tables28 assets
Supplementary material from the paper’s appendix.
Appendix
| Domain | Jurisdiction | Source | Type |
|---|---|---|---|
| Finance | International | Code of Ethics and Standards of Professional Conduct CFA Institute | Professional self-regulatory code |
| Finance | International | Objectives and Principles of Securities Regulation International Organization of Securities Commissions (IOSCO) | Regulatory framework |
| Finance | International | CISI Code of Conduct Chartered Institute for Securities & Investment | Professional self-regulatory code |
| Finance | International | ACCA Code of Ethics and Conduct Association of Chartered Certified Accountants | Professional self-regulatory code |
| Finance | International | IESBA International Code of Ethics for Professional Accountants International Ethics Standards Board for Accountants | Professional self-regulatory code |
| Finance | International | CIMA Code of Ethics Chartered Institute of Management Accountants | Professional self-regulatory code |
Table 1: Full professional-norm source corpus (24 documents).
| Domain | Honesty | Autonomy | Confidentiality |
|---|---|---|---|
| General (domain-agnostic) | Default: Telling the truth / able to be trusted. | Default: Ability to make one’s own decisions without being controlled by anyone else. | Default: Fact of private information being kept secret. |
| Medicine | Default: Full truthful disclosure to the patient by default, layered with an affirmative duty of candour about errors. | Default: Patient autonomy via informed consent is the organising concept: free, uncoerced, adequately informed choice, gated by capacity, including the right to refuse “for any reason.” Contested internally — the Royal College of Physicians explicitly rejects “clinical autonomy” as a value for the doctor (treated as an edge case, see Scenario type taxonomy). | Default: Strong default duty surviving death. Distinctively, ANA states protecting confidentiality can be ethically justified even against a state reporting mandate — ethics outranking law. (HIPAA is the operative US statutory backstop but is deliberately not treated as a primary source, consistent with prioritising ethics/self-regulatory codes over statute.) |
| Exception: Therapeutic privilege triggers only on a clinician’s genuine, individualised belief that disclosure itself — not the diagnosis — would cause serious harm, not general patient distress. Gated by mandatory peer consultation and a documented benefit/harm review; must remain temporary, with a plan to disclose once the patient can tolerate it. Separately, a patient’s own advance request not to be told is honoured — patient-initiated, not clinician-invoked. | Exception: Default shifts to a surrogate (not suspended) when capacity is impaired, and the patient must still be involved “to the greatest extent possible.” Can be narrowed in public-health emergencies or where continued self-determination risks others, but any such limitation is “always considered a serious departure from the standard of care,” justified only where no less-restrictive means exist. | Exception: Triggered by identifiable, foreseeable risk of serious harm to the patient or a third party, legal mandate, or patient consent — a graduated severity threshold, not a categorical one (see Stakes as an exception-triggering mechanism). Gated by a minimum-necessary-disclosure principle and a duty to attempt discussion with the patient first where feasible. | |
| Law | Default: Bifurcated by audience: near-absolute candour to the court (cannot knowingly mislead, must correct false statements) outweighs any duty to volunteer uncomfortable truths to the client. Honesty runs to the tribunal first, the client second. | Default: Two distinct, partly conflicting strands under one word: client autonomy (client sets objectives, instructs, can terminate at will) versus advocate independence (lawyer resists client pressure, may decline or withdraw). | Default: The most absolutely worded duty in the corpus: a “primary and fundamental right,” indefinite in time, backed by evidentiary privilege exempting the lawyer from testifying; bar associations “opposed in principle” to legislative erosion. |
| Exception: No single-trigger exception structure — honesty to third parties is already bounded by confidentiality by default, and is displaced only where the confidentiality exception itself is triggered, or where continuing would mean assisting an ongoing crime/fraud, in which case withdrawal — not disclosure — is the required first step. | Exception: Client autonomy is overridden where the lawyer cannot obtain instructions but has legal authority to act — the “overriding obligation to protect the client’s best interests” governs instead. Advocate independence, conversely, has essentially no exception: the duty to the court cannot be waived by client instruction. | Exception: Triggered by: prevention of reasonably certain death or substantial bodily harm — a graduated severity threshold — or prevention of an ongoing or future crime/fraud using the lawyer’s services (a categorical, not severity-based, trigger), the lawyer’s own self-defence in proceedings, or client consent. Gated tightly — “limited to information absolutely indispensable.” | |
| Finance | Default: Operationalised as information integrity for markets/counterparties collectively (accurate disclosure, no misrepresentation) rather than a dyadic truth-telling duty to one named person. | Default: Client “autonomy” is operationalised thinly, mainly via suitability assessment and mandate-following. The dominant content is actually the professional’s own independence: non-subordination of judgment to client or employer. | Default: Runs to the employer as much as the client — a target absent in medicine/law. Anti-money-laundering obligations (FATF Recommendations — not yet reviewed in depth) are expected to be the domain’s most distinctive confidentiality-limiting mechanism once reviewed: proactive and rule-triggered (a suspicious-transaction threshold) rather than a case-by-case judgment call. |
Table 2: Value domain matrix: default operationalisation and documented exception for each value in each domain.
| Model | Identifier | Provider |
|---|---|---|
| GPT-5.6 Luna | openrouter/openai/gpt-5.6-luna | OpenRouter |
| Gemini 3.5 Flash-Lite | openrouter/google/gemini-3.5-flash-lite | OpenRouter |
| Command A | cohere/command-a-03-2025 | Cohere |
| Command A+ | cohere/command-a-plus-05-2026 | Cohere |
| DeepSeek V4 Flash | openrouter/deepseek/deepseek-v4-flash-0731 | OpenRouter |
| Llama 3.1 8B Instruct | openrouter/meta-llama/llama-3.1-8b-instruct | OpenRouter |
Table 3: Model identifiers and providers.
| Judge pair | Dataset | Outcome agree. | Justif. agree. | ||
|---|---|---|---|---|---|
| GPT-5.6 Luna vs. Gemini 3.5 FL | Main | 0.457 | 88.5% | 0.524 | 69.4% |
| GPT-5.6 Luna vs. Command A | Main | 0.375 | 88.0% | 0.300 | 56.7% |
| Gemini 3.5 FL vs. Command A | Main | 0.559 | 93.6% | 0.310 | 58.3% |
| GPT-5.6 Luna vs. Gemini 3.5 FL | Secondary | 0.481 | 81.1% | 0.492 | 65.1% |
| GPT-5.6 Luna vs. Command A | Secondary | 0.315 | 78.1% | 0.229 | 53.1% |
| Gemini 3.5 FL vs. Command A | Secondary | 0.508 | 86.3% | 0.271 | 56.8% |
Table 4: Pairwise judge agreement by dataset.
Figure 4: Main dataset: unanimous, majority, and split-panel proportions for Outcome Correctness and Justification Type, by dataset.
Figure 5: Secondary experiment: unanimous, majority, and split-panel proportions for Outcome Correctness and Justification Type, by dataset.
| Model | Appropriateness | Matches | Partial | Does not match |
|---|---|---|---|---|
| Claude Sonnet 5 | 97.9% | 97.1% | 1.5% | 1.3% |
| Llama 3.3 70B | 97.6% | 95.6% | 3.9% | 0.5% |
| Claude Haiku 4.5 | 97.5% | 96.2% | 2.7% | 1.1% |
| GPT-5.6 Luna | 97.4% | 95.7% | 3.4% | 0.9% |
| Gemini 3.5 Flash-Lite | 97.0% | 95.7% | 2.7% | 1.6% |
| Llama 4 Scout | 96.7% | 94.7% | 4.1% | 1.2% |
Table 5: main dataset: appropriateness and outcome-label distribution by model.
Figure 6: Main dataset appropriateness (%) by domain and value. Confidentiality shows the widest cross-domain range, while Honesty is comparatively stable across domains.
| Domain | Autonomy | Confidentiality | Honesty |
|---|---|---|---|
| Finance | 96.4 | 97.7 | 99.4 |
| Law | 95.9 | 93.8 | 98.4 |
| Medicine | 95.9 | 90.7 | 99.2 |
| National Security | 92.0 | 93.5 | 97.9 |
Table 6: main dataset appropriateness (%) by domain and value.
Figure 7: Main dataset appropriateness (%) by domain and value, per model. One shared colour scale is used across all panels, so panels are directly comparable to one another (unlike Figure 6 , which is scaled to its own pooled range). Sorted by each model’s overall appropriateness rate.
Figure 8: Main dataset appropriateness against the proportion of correct domain-specific justifications for each model.
Figure 9: Main dataset: appropriateness (%) for each exception probe type (clean-positive, insufficient-evidence, ambiguous-boundary), by domain and value. Only domain–value cells with exception probes are shown.
Figure 12: Domain-label manipulation: pooled effect and per-model breakdown. The near-null pooled effect (Section 4.2 ) holds broadly across models rather than masking large offsetting per-model effects.
Figure 15: Agent-role manipulation: pooled effect and per-model breakdown.
| Domain | Autonomy | Confidentiality | Honesty |
|---|---|---|---|
| Finance | 91.5 | 89.1 | 90.3 |
| Law | 87.2 | 98.9 | 82.6 |
| Medicine | 99.6 | 79.7 | 83.0 |
| National Security | 99.1 | 70.9 | 84.5 |
Table 7: Secondary experiment appropriateness (%) by domain and value. These values describe the internal structure of the secondary experiment and are not intended for comparison with the main dataset.
Figure 16: Distribution of stakes effects.
| Model | Construct | High | Low | Gap |
|---|---|---|---|---|
| Llama 3.1 70B | Medicine Honesty | 100.0 | 8.8 | 91.2 |
| Gemini 3.5 Flash-Lite | Law Honesty | 100.0 | 19.7 | 80.3 |
| Command A | Medicine Honesty | 97.4 | 17.9 | 79.4 |
| Llama 3.3 70B | Medicine Honesty | 100.0 | 23.8 | 76.2 |
| Gemma 2 27B | Medicine Honesty | 100.0 | 35.0 | 65.0 |
Table 8: Examples of the largest positive High-minus-Low stakes differences.
| Model | High stakes | Low stakes | Gap |
|---|---|---|---|
| Gemma 3 27B | 25.0% | 92.3% | 67.3pp |
| Command A | 51.3% | 96.3% | 44.9pp |
| Gemini 3.5 Flash-Lite | 73.1% | 100.0% | 26.9pp |
| Gemma 2 27B | 73.8% | 100.0% | 26.3pp |
| Llama 4 Scout | 68.0% | 87.2% | -19.2pp |
Table 9: Largest reverse-direction effects for Medicine Confidentiality.
| Construct | High stakes | Low stakes | Difference |
|---|---|---|---|
| Person-facing Autonomy | 99.4% | 98.9% | 0.5pp |
| Institution-facing Confidentiality | 88.5% | 10.1% | 78.4pp |
| Person-facing Confidentiality | 76.0% | 95.5% | 19.5pp |
| Institution-facing Honesty | 99.2% | 92.4% | 6.9pp |
| Person-facing Honesty | 91.7% | 54.2% | 37.5pp |
Table 10: Secondary experiment: national-security constructs by stakes.
Figure 17: Secondary experiment appropriateness by national-security construct and stakes. Institution-facing Confidentiality exhibits a large Low-stakes failure that is not shared by institution-facing Honesty or person-facing Confidentiality.
Figure 20: Appropriateness (%) by broad post-training category, by dataset. Descriptive only; see discussion of coarseness of these categories in the main text.