Dataset Deep Dive

UniMoral: The First Dataset That Knows Who Is Judging, Not Just What They Judge

Every other moral dataset asks: "Is this action right or wrong?" UniMoral asks the harder question: "Is this action right or wrong for someone with these specific moral foundations?" That makes it the perfect validation target for MHF.

What It Is

Moral judgments with moral profiles attached

UniMoral is a multilingual moral evaluation dataset covering six languages: Arabic, Chinese, English, Hindi, Russian, and Spanish. But what makes it extraordinary is not the breadth -- it is the depth. Every annotator in UniMoral completed the MFQ2 (Moral Foundations Questionnaire, Second Edition), giving researchers their individual Haidt moral profile. Then those same annotators judged a set of moral dilemmas.

This means, for the first time, we can ask: does knowing someone's moral foundations profile predict their moral judgment? If it does, then MHF's architecture -- which parameterizes moral reasoning via Haidt foundation weights -- is validated at the individual level, not just the cultural one.

6
Languages
MFQ2
Haidt profile per annotator
VSM
Cultural values measured
1st
Profile-to-judgment dataset
The Killer Experiment

Parameterize MHF with their profile. Predict their verdict.

This is MHF's most direct validation opportunity. The experimental design is simple and devastating:

Individual Moral Prediction Pipeline

Take an annotator's own Haidt foundation weights. Plug them into MHF's parameterization. Run the dilemma through the graph. Compare the output to their actual judgment.

Step 1

Take annotator's MFQ2 scores

Care: 4.2 / Auth: 3.8 / Sanc: 4.5 ...

Step 2

Parameterize MHF graph with their profile as theta vector

theta = normalize(MFQ2_scores)

Step 3

Run constraint propagation on the same dilemma they judged

predict(dilemma, theta) vs actual_judgment

If MHF can predict individual annotator judgments better than a flat classifier that ignores moral profiles, the core thesis holds: moral reasoning is parameterized constraint propagation, not flat classification.

Why this is different from what anyone else has done: Delphi predicts the crowd average. ETHICS checks against philosophical principles. Neither can explain why two equally thoughtful people disagree. MHF, parameterized per-annotator via UniMoral's MFQ2 data, can predict which person will say "yes" and which will say "no" -- and explain the difference in terms of their Care vs. Authority vs. Sanctity weights.

Cross-Cultural Profiles

How moral foundations differ across six languages

UniMoral's language coverage maps onto genuinely different moral cultures. Arabic and Hindi annotators tend to score higher on Authority and Sanctity. English annotators weight Care and Fairness more heavily. Chinese annotators show a distinctive pattern in Loyalty. These are not stereotypes -- they are measured MFQ2 distributions. The bars below are representative profiles based on published Haidt research for similar populations.

Arabic

ar / High Authority + Sanctity
Care
3.6
Fair
3.2
Loyal
4.0
Auth
4.3
Sanct
4.4

Chinese

zh / High Loyalty
Care
3.4
Fair
3.3
Loyal
4.2
Auth
3.7
Sanct
3.0

English

en / High Care + Fairness
Care
4.2
Fair
4.1
Loyal
2.6
Auth
2.3
Sanct
2.1

Hindi

hi / High Authority + Sanctity
Care
3.8
Fair
3.4
Loyal
3.9
Auth
4.1
Sanct
4.2

Russian

ru / Moderate across all axes
Care
3.5
Fair
3.4
Loyal
3.6
Auth
3.5
Sanct
3.3

Spanish

es / High Care, moderate Sanctity
Care
4.0
Fair
3.8
Loyal
3.3
Auth
2.8
Sanct
3.1

The pattern is clear. WEIRD (Western, Educated, Industrialized, Rich, Democratic) populations weight Care and Fairness heavily while deprioritizing Authority and Sanctity. Non-WEIRD populations have a more balanced profile -- or even reverse the hierarchy. This is not a bug in the data. It is the core phenomenon MHF is designed to capture.

MHF Validation

What UniMoral tells us about parameterization

MHF Prediction What UniMoral Can Test Expected Result
High-Authority annotators differ from low-Authority Split annotators by Authority MFQ2 score, compare judgments on authority-relevant dilemmas Systematic divergence in obedience/conformity scenarios
High-Sanctity annotators differ from low-Sanctity Same split on Sanctity, compare purity/sexuality dilemmas Systematic divergence in disgust/purity scenarios
Individual theta predicts better than population mean Compare prediction accuracy: per-annotator MHF vs. flat classifier MHF per-annotator > flat by measurable margin
Cross-language profiles match Haidt literature Compare average MFQ2 by language to published norms Arabic/Hindi high Auth+Sanct; English high Care+Fair
MHF Christian params match high-Auth+Sanct annotators Cluster annotators by profile similarity to Christian weights High-similarity cluster disproportionately from Arabic, Hindi subsets
Strengths & Weaknesses

Honest assessment

Strengths

  • Individual profiles. MFQ2 per annotator -- first dataset to connect moral psychology to moral judgment at the individual level
  • Multilingual. Six languages spanning four language families and multiple moral cultures
  • Cultural values. VSM (Values Survey Module) provides Hofstede-compatible cultural dimensions alongside Haidt foundations
  • Clean experimental design. Same dilemmas across languages enable cross-cultural comparison
  • Directly validates MHF. The parameterization architecture maps perfectly to MFQ2 profiles

Weaknesses

  • Smaller scale. Far fewer entries than AITA's 270K -- may lack statistical power for fine-grained splits
  • Annotator sample bias. Who volunteers for moral annotation studies is not random
  • No relational structure. Dilemmas do not include stakeholder graphs (same limitation as AITA)
  • MFQ2 captures stated values, not revealed preferences. What people say they value vs. how they actually judge may diverge
  • Six languages is not six cultures. Hindi-speaking India contains more moral diversity than this simplification captures
So What?

Why UniMoral matters for moral AI

The entire field of moral AI has been building systems that predict the average judgment of a culturally narrow population. This is useful for content moderation. It is useless for moral advice.

When someone asks "Should I leave my alcoholic father?", the right answer depends on their moral commitments -- not the average American's. A person high in Authority and Loyalty faces a genuinely different moral landscape than a person high in Care and Liberty. UniMoral is the first dataset that lets us test whether MHF can navigate that difference.

If it works -- if per-annotator parameterization predicts judgments better than flat classification -- then we have demonstrated something the field has not seen: a moral reasoning system that respects moral diversity without collapsing into relativism. The hierarchy is not "anything goes." It is "given your commitments, here is what consistency demands." UniMoral lets us prove it.