
<!-- README.md is generated from README.Rmd. Please edit README.Rmd. -->

# gp3ml <img src="man/figures/logo.png" align="right" height="139" alt="gp3ml logo" />

<!-- badges: start -->

[![R-CMD-check](https://github.com/stefanosbalaskas/gp3ml/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/stefanosbalaskas/gp3ml/actions/workflows/R-CMD-check.yaml)
[![CRAN
status](https://www.r-pkg.org/badges/version/gp3ml)](https://CRAN.R-project.org/package=gp3ml)
[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.21487272.svg)](https://doi.org/10.5281/zenodo.21487272)
<!-- badges: end -->

`gp3ml` is a governance-first R package for leakage-resistant and
group-aware predictive modelling and validation using Gazepoint-derived
research data.

The package is designed for explicitly observed, non-sensitive outcomes
and declared scientific purposes. It is not a generic automatic machine-
learning wrapper and does not perform autonomous model selection.

## Installation

Install the current CRAN release with:

``` r
install.packages("gp3ml")
```

The GitHub repository may contain a later development or
release-candidate version. Install it with:

``` r
install.packages("pak")
pak::pak("stefanosbalaskas/gp3ml")
```

## Scope

The package provides:

- task, purpose, outcome, predictor, and identifier-role governance;
- feature-provenance manifests and leakage auditing;
- deterministic group-aware holdout splitting;
- repeated grouped resampling with materialized folds;
- fold-balance, coverage, exclusion, and outcome diagnostics;
- fold-local preprocessing contracts;
- governed statistical and optional machine-learning engines;
- discrimination, error, calibration, and uncertainty assessment;
- external-validation reports;
- model cards and reproducibility reports.

Supported predictive-generalization targets are:

- new trials among known participants;
- new participants;
- new stimuli;
- simultaneous new-participant and new-stimulus generalization.

## Scientific safeguards

Participant overlap is treated as a failure when the declared target
requires generalization to new participants. Stimulus overlap is treated
as a failure when the declared target requires generalization to unseen
stimuli. Preprocessing must be estimated inside the relevant analysis
partition or resampling fold. Package examples and tests use
deterministic synthetic data.

## Leakage-audit workflow

`audit_gazepoint_ml_leakage()` audits already-defined analysis and
assessment partitions before predictive evaluation. It checks:

- compatibility with the declared generalization target;
- participant, participant-trial, and stimulus overlap;
- outcome and declared identifiers included as predictors;
- declared target-derived and post-outcome predictors;
- exact row overlap and repeated predictor profiles;
- duplicated rows and missing grouping identifiers; and
- identifier-like predictor names requiring manual review.

The returned audit has an overall `pass`, `review`, or `fail` status, a
complete check table, a non-passing issue table, and partition
summaries. Audit tables can be exported using
`write_gazepoint_ml_leakage_audit_csv()`.

``` r
analysis <- data.frame(
  participant_id = c("P01", "P02"),
  trial_id = c("T01", "T02"),
  stimulus_id = c("S01", "S02"),
  outcome = c(0, 1),
  fixation_duration = c(210, 245)
)

assessment <- data.frame(
  participant_id = c("P03", "P04"),
  trial_id = c("T03", "T04"),
  stimulus_id = c("S03", "S04"),
  outcome = c(1, 0),
  fixation_duration = c(275, 230)
)

audit <- audit_gazepoint_ml_leakage(
  analysis = analysis,
  assessment = assessment,
  outcome = "outcome",
  predictors = "fixation_duration",
  participant_id = "participant_id",
  trial_id = "trial_id",
  stimulus_id = "stimulus_id",
  generalization_target = "new_participants"
)

audit
```

The audit identifies structural risks visible in the supplied partitions
and declared variable roles. It does not establish that preprocessing or
feature selection was estimated within resampling folds; those
safeguards require separate provenance and resampling infrastructure.

## Feature-provenance workflow

`create_gazepoint_feature_manifest()` records the declared origin,
transformation, availability, role, and preprocessing scope of each
intended predictor.

`validate_gazepoint_feature_manifest()` checks whether the manifest
contains sufficient provenance metadata and identifies predictors
declared as outcome-derived, post-outcome, unavailable at prediction
time, identifiers, or incompatible with required fold-local
preprocessing.

``` r
manifest <- create_gazepoint_feature_manifest(
  features = c("fixation_duration", "pupil_change"),
  scientific_source = c(
    "Gazepoint fixation export",
    "Gazepoint all-gaze export"
  ),
  source_table = c("fixations", "all_gaze"),
  transformation = c(
    "Trial-level mean",
    "Baseline-adjusted change"
  ),
  availability_stage = "during_exposure",
  prediction_time_available = TRUE,
  preprocessing_scope = c("none", "resampling_fold"),
  fold_local_required = c(FALSE, TRUE)
)

validation <- validate_gazepoint_feature_manifest(manifest)
validation
```

Validation returns an overall `pass`, `review`, or `fail` status,
together with complete check and issue tables. The manifest or its
validation tables can be exported using
`write_gazepoint_feature_manifest_csv()`.

The manifest records declared provenance. It does not independently
verify that preprocessing was executed within the stated partition or
resampling scope.

## Group-aware holdout splitting

`split_gazepoint_ml_data()` creates deterministic analysis and
assessment partitions that preserve the grouping unit implied by an
explicit predictive-generalization target. A passing feature-provenance
manifest is required before splitting, and the resulting partitions are
checked using the leakage audit.

``` r
split_data <- expand.grid(
  participant_id = sprintf("P%02d", 1:8),
  trial_id = sprintf("T%02d", 1:4),
  KEEP.OUT.ATTRS = FALSE,
  stringsAsFactors = FALSE
)

split_data$outcome <- as.integer(
  seq_len(nrow(split_data)) %% 2L
)

split_data$fixation_duration <-
  200 + seq_len(nrow(split_data))

split_data$pupil_change <-
  seq_len(nrow(split_data)) / 100

split <- split_gazepoint_ml_data(
  data = split_data,
  outcome = "outcome",
  predictors = c(
    "fixation_duration",
    "pupil_change"
  ),
  feature_manifest = manifest,
  generalization_target = "new_participants",
  participant_id = "participant_id",
  trial_id = "trial_id",
  assessment_prop = 0.25,
  seed = 17
)

split
split$validation
```

Supported generalization targets are:

- `new_trials_known_participants`;
- `new_participants`;
- `new_stimuli`; and
- `new_participants_and_new_stimuli`.

For simultaneous participant and stimulus generalization, rows belonging
to only one held-out dimension are placed in the `excluded` partition.
This preserves strict separation between the analysis and assessment
participant–stimulus blocks while retaining complete source-row
accounting.

`validate_gazepoint_ml_split()` checks partition structure, source-row
accounting, provenance status, and the embedded leakage audit. Split
assignments, summaries, validation checks, and materialized partitions
can be exported using `write_gazepoint_ml_split_csv()`.

The splitter creates one deterministic holdout split. It does not
perform preprocessing, automated feature selection, cross-validation,
nested resampling, or model fitting.

## Group-aware resampling

`create_gazepoint_group_folds()` creates deterministic repeated grouped
V-fold plans for the same explicit generalization targets supported by
the holdout splitter. A passing feature-provenance manifest is required,
and every analysis-assessment pair is checked using the leakage audit.

``` r
folds <- create_gazepoint_group_folds(
  data = split_data,
  outcome = "outcome",
  predictors = c(
    "fixation_duration",
    "pupil_change"
  ),
  feature_manifest = manifest,
  generalization_target = "new_participants",
  participant_id = "participant_id",
  trial_id = "trial_id",
  v = 4,
  repeats = 2,
  seed = 17
)

folds
folds$validation
folds$audit
```

Supported targets are new trials among known participants, new
participants, new stimuli, and simultaneous new-participant and
new-stimulus generalization. For simultaneous participant and stimulus
generalization, `v` may contain separate participant and stimulus fold
counts. Crossed assessment blocks preserve strict separation in both
dimensions, while cross-block rows are explicitly assigned to the
excluded partition.

`validate_gazepoint_group_folds()` checks fold counts, source-row
accounting, assessment coverage, materialized partitions, feature
provenance, and embedded leakage audits. `audit_gazepoint_group_folds()`
aggregates the fold-level audits. Assignments, summaries, validation
tables, audit tables, and optionally materialized partitions can be
exported using `write_gazepoint_group_folds_csv()`.

The fold planner does not perform preprocessing, automated feature
selection, tuning, nested resampling, or model fitting.

## Resampling diagnostics

`diagnose_gazepoint_group_folds()` evaluates the balance, coverage,
outcome representation, and exclusions of an existing grouped resampling
plan. Diagnostics remain downstream of fold construction and do not
modify assignments or fit models.

``` r
fold_diagnostics <- diagnose_gazepoint_group_folds(folds)

fold_diagnostics
fold_diagnostics$repeat_metrics
head(fold_diagnostics$outcome_balance)
head(fold_diagnostics$assessment_coverage)
```

`validate_gazepoint_fold_diagnostics()` reports `pass`, `review`, or
`fail` findings for fold counts, partition accounting, non-empty
partitions, assessment coverage, fold-size imbalance, group presence,
outcome-level presence, embedded leakage audits, and source-plan status.

For crossed new-participant and new-stimulus designs,
`exclusion_summary` reports the rows intentionally excluded because they
belong to only one held dimension rather than the strict crossed
assessment block.

All diagnostic tables can be exported for review and reporting:

``` r
write_gazepoint_fold_diagnostics_csv(
  fold_diagnostics,
  directory = "resampling-diagnostics",
  prefix = "gazepoint_folds"
)
```

Diagnostics do not perform preprocessing, automated feature selection,
hyperparameter tuning, nested resampling, or model fitting.

## Governed modelling and reporting

The governed modelling core provides related interfaces for:

- declaring tasks and validating intended uses;
- validating outcome, predictor, and identifier roles;
- fitting and applying fold-local preprocessors;
- fitting reviewable model engines and integrating externally fitted
  black-box models under explicit governance;
- calculating classification and regression performance metrics;
- assessing and applying calibration;
- producing explicitly labelled bootstrap metric uncertainty;
- evaluating genuinely external datasets;
- generating model cards and reproducibility reports.

These interfaces do not convert row-level metrics into participant-level
estimates merely because a split was participant-grouped. Version
`0.2.0` adds governed tuning and nested grouped resampling while
retaining explicit human-reviewed selection. Autonomous winner selection
remains outside the package’s scope.

## Package maturity and interoperability

The development line adds explicit API contracts, lightweight handoff
contracts for `gp3tools`, `gpbiometrics`, and `gp3sequences`,
deterministic documentation-output controls, optional-engine portability
audits, and an integrated synthetic cross-package research workflow.
These additions preserve the governance boundary: upstream packages
remain responsible for preprocessing and feature derivation, while
`gp3ml` remains responsible for declared roles, leakage-safe resampling,
governed modelling, evaluation, uncertainty, and reporting.

## Prohibited uses

The package does not support person identification, health inference,
protected-attribute prediction, or direct inference of emotion, stress,
mental state, cognition, comprehension, personality, deception, or
intent.

See:

- [`GOVERNANCE.md`](https://github.com/stefanosbalaskas/gp3ml/blob/master/inst/governance/GOVERNANCE.md)
- [`PROHIBITED-USE.md`](https://github.com/stefanosbalaskas/gp3ml/blob/master/inst/governance/PROHIBITED-USE.md)

## Citation

Use the Zenodo concept DOI to cite `gp3ml` generally:

- Concept DOI: <https://doi.org/10.5281/zenodo.21487272>
- CRAN package DOI: <https://doi.org/10.32614/CRAN.package.gp3ml>

For the archived `0.2.0` release, use its version-specific DOI:

- `gp3ml 0.2.0`: <https://doi.org/10.5281/zenodo.21532057>

The version-specific DOI for `0.3.0` will be added after the release is
archived.

The citation metadata installed with the package can be displayed with:

``` r
citation("gp3ml")
```

## Release status

Version `0.3.0` is the third formal source release of `gp3ml`.

It preserves the grouped evaluation, governed tuning, nested resampling,
target-aligned uncertainty, external-validation reporting, and release
model-card infrastructure established in version `0.2.0`. Version
`0.3.0` adds explicit API contracts, cross-package handoffs,
prediction-to-decision governance, target-aware conformal prediction,
dataset-shift auditing, locked analysis plans, portable model artifacts,
robustness diagnostics, environment provenance, research-object export,
release manifests, and standards-oriented evidence profiles.

The stable compatibility contract protects 71 exports and 38 public
classes. The 56 newer exports remain explicitly experimental and are not
promoted merely because they are included in this release.

Model selection remains explicitly reviewable and human-governed. The
package does not perform autonomous winner selection or relax its
prohibited-use boundaries.

## Analytical roadmap

gp3ml now extends its materialized grouped-fold infrastructure with a
complete, reviewable evaluation and reporting layer:

- fold-local evaluation across `gazepoint_group_folds`;
- explicit deterministic candidate grids and human-reviewed selection;
- nested participant/stimulus-aware resampling;
- observation, participant, stimulus, two-way, fold, and repeat
  uncertainty;
- independent external-dataset declarations and transportability
  reports;
- release-ready model cards recording selection, uncertainty,
  limitations, and external-validation status.

The package’s synthetic-data articles demonstrate predefined
recording-quality review, assigned-condition discrimination, observed
non-sensitive endpoints, all supported generalization targets,
contaminated provenance, nested resampling, and external validation.
These workflows retain the package’s prohibited-use boundaries and do
not perform autonomous model selection.

## Project resources

- [Package website](https://stefanosbalaskas.github.io/gp3ml/)
- [CRAN package page](https://CRAN.R-project.org/package=gp3ml)
- [Source repository](https://github.com/stefanosbalaskas/gp3ml)
- [Issue tracker](https://github.com/stefanosbalaskas/gp3ml/issues)

## Governance and reproducibility in 0.3.0

Version `0.3.0` extends `gp3ml` beyond model fitting into explicit
prediction-to-decision governance, target-aware conformal prediction,
dataset-shift auditing, frozen analysis plans, model-artifact
portability, robustness diagnostics, environment provenance,
research-object export, and standards-oriented evidence crosswalks.

These features preserve gp3ml’s existing governance boundary. They do
not support identification, authentication, diagnosis,
protected-attribute inference, or direct or indirect inference of
emotion, stress, personality, deception, cognition, comprehension,
intent, or other mental states. Standards-oriented crosswalks are
documentation aids only and do not imply NIST endorsement, ISO
conformity, certification, or legal compliance.
