The challenge is not recognizing a name, but deciding whether that name can stay in the text. Contextual PII masking systems need examples that distinguish maskable from preservable mentions even when they belong to the same entity type. ReCoPII starts from this point: privacy restrictions make access to real data difficult, and author-based augmentation techniques such as SPY introduce fragile labeling because they assume a single data subject.
ReCoPII overturns that premise. The framework plans ontology-constrained entity relations, generates insertion sentences with placeholders, reconstructs annotations in code, and has a separate LLM check the result. MASK/KEEP labels then derive from entity relations, not from author identity. On Korean documents from six domains, this yields ReCoPII-Ko, a corpus with relation, privacy, and MASK/KEEP annotations.
The most telling figure is not just annotator agreement: 88.42% on sampled KEEP labels against 26.27% for SPY-style augmentation of the same sources. That gap signals the author-centric assumption is not a technical detail but a structural flaw. If an organization uses SPY to build masking data, it risks labeling as non-PII mentions that should be preserved, or training a system on inconsistent examples. In a compliance context, this does not just add noise; it erodes the ability to decide what to mask and what to keep.
The second result shifts the perspective on the balance between protection and utility. Joint fine-tuning with a PII detector, within a fixed leakage budget, removes most of the over-masking that detection alone would introduce. On human-verified entity records, the four detectors meeting that budget show at most 5.04% over-masking with at most 4.18% leakage. This is not an abstract benchmark: it is an operational trade-off. Over-masking does not protect more; it degrades text for downstream models. Under-masking exposes personal data. The relationship between these two errors is what a deployment team must govern.
There is also a signal for efficiency. The compact detector is comparable to XLM-R-large on KDPII with about a quarter of the parameters. In a self-hosted architecture, that translates into lower VRAM pressure, more contained latency, and lower inference costs, without sacrificing detection capability on that benchmark. That is not a recommendation; it is a structural constraint that shifts attention from larger models to data quality and fine-tuning.
The broader lesson of ReCoPII is that privacy in LLMs is not solved by a masking layer alone, but by training data that reflects real relations among entities. Moving from author-based to relation-based labels reduces arbitrariness and brings masking closer to document semantics. Regulated environments, where data cannot leave the local perimeter, need exactly this: mechanisms that decide what to protect without destroying the context the LLM needs.
๐ฌ Comments (0)
๐ Log in or register to comment on articles.
No comments yet. Be the first to comment!