Dataset Rights, Provenance and Privacy
Separate possession from permission, bind evidence to intended uses, and propagate source restrictions through a synthetic data pipeline.
A support team can export its ticket archive in seconds. That technical ability does not answer whether the archive may be used to train a model, sent to a labelling service, retained indefinitely or included in a public dataset. These are different acts, potentially involving different people, purposes and obligations. A permissive answer to one question cannot safely be copied into the other fields.
The miniature experiment in Build a Fine-Tune Without a Training Framework avoided this difficulty by constructing a few letter rules. A useful data pipeline must make the assumptions visible. Before asking whether an example improves a model, we need to establish where it came from, which use is proposed, and which evidence supports that use. An impressive loss curve cannot supply missing permission retrospectively.
Our running dataset is entirely fictional. We will nevertheless build a source ledger that can represent an unknown licence, an evaluation-only permission, a deletion request and inherited restrictions. Learning to represent a refusal is as important as learning to represent an approval.
Permission is a relation, not a property of a file
Consider a synthetic incident: “A fictional user cannot authenticate.” One copy is generated from an original teaching specification. Another is copied from an online incident report. Their text could be identical while their provenance differs. Conversely, two very different strings could share the same restrictive source agreement. Content similarity does not establish permission, and content identity does not establish origin.
Represent an eligibility decision as a relation $E(s,u,t)$ over source $s$, intended use $u$ and the state known at time $t$. The time argument matters: a source may be revoked or the proposed purpose may change. Saving an old eligible=true flag beside a record is therefore insufficient for admitting a new dataset revision.
Possession, ownership, copyright permission, confidentiality and personal-data obligations should be assessed separately. An organisation may possess records without owning every contribution. Its agreement with a supplier may restrict onward disclosure. A licence can permit one form of analysis while forbidding redistribution. A release process must retain the evidence and scope of the decision rather than compress these questions into “public data”.
The engineering rule is deliberately conservative: an unavailable permission is unknown, not implicitly granted. This is an organisational admission rule. It is not a claim that every legally possible use requires a particular licence document, or that a Python validator can decide the applicable law.
Identify the legal question before automating its evidence
In the EU, copyright and personal-data analysis are separate. The Digital Single Market Directive, Articles 3 and 4 distinguishes scientific-research text and data mining from a broader exception with conditions including lawful access and rights reservation. Applying an exception requires attention to the activity, beneficiary and national implementation. Neither exception should be treated as a blanket dataset-publication permission. Website terms and contractual restrictions need their own assessment; public accessibility alone does not settle those questions.
For personal data, the GDPR requires a lawful basis, purpose limitation, minimisation and appropriate retention and security. Special-category processing has additional conditions. Erasure rights have conditions and exceptions; a pipeline should support requests without promising that every request has the same legal outcome. Data protection by design belongs in preparation, rather than being attached after training.
The Irish Data Protection Commission’s lawful-basis guidance lists consent, contract, legal obligation, vital interests, public task and legitimate interests. Consent is consequently not the only possible basis, while having collected data for one purpose does not automatically establish a basis for a different training purpose. A responsible assessment identifies the controller, proposed processing and supporting reasoning.
Ireland’s 2021 implementing regulations insert research and commercial text-and-data-mining provisions, including rules about reserved rights and contractual terms contrary to particular exceptions. This is why “the terms prohibit it” also needs legal analysis rather than replacing it. The engineering gate conservatively records unresolved questions; it does not override statutory exceptions.
Jurisdiction matters. EU provisions are not a worldwide rule for every source, user or deployment. A project crossing jurisdictions needs an assessment of the applicable requirements and agreements. Our code records decisions made at that boundary; it does not issue legal advice or determine that boundary from a URL.
Removing names does not establish anonymity
A fictional replacement such as person-17 can preserve links between records. A rare job role, time, location and incident description can also identify someone without naming them. The Irish DPC’s anonymisation guidance distinguishes pseudonymisation from data that no longer identifies individuals. Pseudonymised records remain personal data. Assess identifiability through available linking information, rather than checking only whether a name column is absent.
The EDPB’s Opinion 28/2024 also treats anonymity of AI models as an evidence-dependent assessment. Generating synthetic text from private records is not, by its name alone, proof that those records cannot be recovered or linked.
Our source specification uses no actual tickets, conversations or people. NONE_EXPECTED records that bounded assessment, not a privacy-compliance certificate. In a different project, suspected personal information should trigger review before use. Pattern matching can flag a possible identifier; failure to match one cannot prove absence. Sensitive attributes and trade secrets can appear in ordinary prose, screenshots and derived labels as well as explicit database fields.
Build a ledger that keeps missing evidence visible
FineTuneLab’s source manifest separates source identity from its content digest. It records origin, revision, generation date, licence identifier and source, supporting evidence, allowed and prohibited uses, personal-data assessment, retention, revocation, derivation parents, reviewer state and unresolved evidence. A digest binds that particular declaration. It does not certify the declaration’s truth.
The validator uses four outcomes:
| Status | Engineering meaning |
|---|---|
RIGHTS_VERIFIED |
An external authenticated, use-specific approval binds the declared evidence |
RIGHTS_RESTRICTED |
The proposed use is excluded or requires additional review |
RIGHTS_UNKNOWN |
Required evidence or trusted approval is missing |
RIGHTS_REJECTED |
Revocation prevents this admission |
“Trusted” refers to the caller’s approval store, not a signature invented by the submitted manifest. Changing review.state to APPROVED and recomputing its hash does not insert a matching approval into that store. A production service would need authenticated approval capture and access controls around it. This small library deliberately exposes that trust assumption.
The diagram follows one (source, use, time) triple through the eligibility check to its four terminal states. Notice that unknown is the default outcome rather than a gap to be filled later, and that a submitted approval field can never satisfy the store lookup the arrow requires.
The original pilot used an engineering approval declaration supplied by its own generator. That is circular authority: the submitted object cannot establish who approved its use. Its synthetic origin establishes provenance, not independent rights approval. The current mechanism policy permits local construction and QA of these fictional examples; it grants neither training nor benchmark authority.
The source ledger therefore retains declarations separately from approval evidence. Candidates may reference approval IDs. The auditor receives an external authority context, whose verifier must authenticate independently persisted records bound to source digest, evidence, parent restrictions and the particular use. Merely passing an approval-shaped JSON object, setting a review flag or recomputing hashes cannot satisfy that boundary. Unknown or revoked authority does not become permission.
Training, evaluation and redistribution are separate requests. The replacement candidate declares future training and evaluation purposes, checks each, and reports RIGHTS_APPROVAL_PENDING. No production approval backend is configured. Test-only verifier fixtures exercise refusals; they cannot authorise a real release. Parent restrictions and revocations remain effective even when a child claims permission.
Choose a new output path. The result includes the source declarations and the exact code identities used. A later candidate audit calls the same eligibility function again; it does not inherit an old green flag as authority for a new purpose.
Eligibility establishes a necessary boundary around the examples. It does not establish that their answers are correct. The next chapter examines that separate source of authority: the label definition, the evidence behind each answer and the decisions that remain unresolved even after two annotators agree.
Further Reading
- Timnit Gebru et al., “Datasheets for Datasets” (2018). Read the source. Motivates the chapter ledger: document motive, composition, collection and permitted uses so a digest can bind what was declared without pretending to certify its truth.
- Emily M. Bender and Batya Friedman, “Data Statements for Natural Language Processing” (2018). Read the source. Defines curation, speaker and annotator context for NLP data, clarifying why content identity cannot establish permission.
- Shayne Longpre et al., “The Data Provenance Initiative” (2023). Read the source. Audits licence lineage across more than 1800 text datasets and finds pervasive omission and error, supporting the conservative unknown-until-evidenced gate.