[Aman (its-amann)]

Description

Fixes #3430.

When dataset patterns overlap, the same CSV can get matched more than once. It then gets loaded twice, which gives it extra weight in the evaluation and changes the reported accuracy. This PR removes duplicate paths right before the existing sorted loading step, so each file is only counted once.

Different files are still counted separately, even if they share the same dataset version. Sorting, version concatenation and registry behavior don't change.

Roman Lutz (@romanlutz), I kept the fix in the shared evaluator path, so it covers both objective and harm evaluations.

Tests and Documentation

• Added regression tests for overlapping and repeated patterns, reversed order, patterns that match nothing, recursive globs and ./ paths. They use real CSV loading, SubStringScorer, the metric calculation and an isolated SQLite memory.
• Evaluator tests: 111 passed.
• Scorer unit tests: 4,541 passed, 115 skipped (make unit-test-junit CMD='python -m' UNIT_TESTS=tests/unit/score).
• Pre-commit hooks on the changed files, Ruff and ty on the changed modules all pass. I ran ty only on the changed files instead of the full ty-check hook, since the full one needs every optional dependency installed.
• Updated the dataset config docstring to explain what happens with overlapping patterns. No notebooks changed, so I didn't run JupyText.

[Aman (its-amann)]

@microsoft-github-policy-service agree

[Richard Lundeen (richlundeen)]

Looks good. Deduplicating the matched paths before loading prevents overlapping patterns from counting the same CSV twice, while keeping deterministic ordering.

Reviewed with GitHub Copilot.