Missing-Data

Setup

First, lets load the package:

library(bvarnet)

Data

Now, we can load the example data:

data(studentlife)

Missing Data in bvarnet

Inspecting the dataset, we can see that there are quite a lot of missing values:

head(studentlife)
#> # A tibble: 6 × 76
#>      id   day happy happyornot   sad sadornot social_number stress_level sleep_hour sleep_quality difficult_stay_awake anxious  calm
#>   <dbl> <dbl> <dbl>      <int> <dbl>    <dbl>         <dbl>        <dbl>      <dbl>         <dbl>                <int>   <dbl> <dbl>
#> 1     0     1    NA         NA    NA       NA         NA               3       7.5           1.5                     0      NA    NA
#> 2     0     2    NA         NA    NA       NA          4               2       5.33          2.33                    0      NA    NA
#> 3     0     4    NA         NA    NA       NA          2               4       9             2                       0      NA    NA
#> 4     0     5    NA         NA    NA       NA          2              NA       3             4                       0      NA    NA
#> 5     0     6    NA         NA    NA       NA         NA               1      NA            NA                      NA      NA    NA
#> 6     0     7    NA         NA    NA       NA          2.67            2       5             3                       1      NA    NA
#> # ℹ 63 more variables: conventional <dbl>, critical <dbl>, dependable <dbl>, disorganized <dbl>, enthusiastic <dbl>,
#> #   open_to_experiences <dbl>, reserved <dbl>, sympathetic <dbl>, act_running_ep_0 <dbl>, act_running_ep_1 <dbl>, act_running_ep_2 <dbl>,
#> #   act_running_ep_3 <dbl>, act_running_ep_4 <dbl>, act_still_ep_0 <dbl>, act_still_ep_1 <dbl>, act_still_ep_2 <dbl>, act_still_ep_3 <dbl>,
#> #   act_still_ep_4 <dbl>, act_unknown_ep_0 <dbl>, act_unknown_ep_1 <dbl>, act_unknown_ep_2 <dbl>, act_unknown_ep_3 <dbl>,
#> #   act_unknown_ep_4 <dbl>, act_walking_ep_0 <dbl>, act_walking_ep_1 <dbl>, act_walking_ep_2 <dbl>, act_walking_ep_3 <dbl>,
#> #   act_walking_ep_4 <dbl>, act_on_foot_ep_0 <dbl>, act_on_foot_ep_1 <dbl>, act_on_foot_ep_2 <dbl>, act_on_foot_ep_3 <dbl>,
#> #   act_on_foot_ep_4 <dbl>, audio_convo_duration_ep_0 <dbl>, audio_convo_duration_ep_1 <dbl>, audio_convo_duration_ep_2 <dbl>, …

If we look at the subset of variables that we use in the Vignette("bvarnet") c(“anxious”, “calm”, “conventional”, “critical”, “dependable”), we see there is approximately \(90\%\) of missing data:

sum(is.na(studentlife[c("anxious", "calm", "conventional", "critical", "dependable")])) / (5*nrow(studentlife))
#> [1] 0.8988494

The Skipping-Lag Mechanism

Before constructing lagged predictors, bvarnet applies listwise deletion at the measurement-occasion level. Within each subject, a row is removed if any selected outcome or covariate is missing. The subject is not removed simply because one or more measurement occasions are incomplete. Entirely absent time points and rows removed through listwise deletion can therefore create gaps in the remaining time series. Listwise deletion is computationally simple, but it reduces the available sample size and can produce biased estimates when the retained observations differ systematically from the omitted observations. Missing completely at random (MCAR) provides a straightforward justification for this procedure, although its validity ultimately depends on the missingness process, model, and estimand. After listwise deletion, bvarnet constructs the lagged predictors. For a VAR(\(K\)) model, the outcome at time \(t\) has a valid lag window only when observations are available at every required time \(t-1,\ldots,t-K\). The time_col variable must therefore be integer-valued, with one unit corresponding to one lag step. When skip_lag = TRUE (the default), a current observation with an incomplete lag window is retained, but its entire set of lagged predictors is set to zero. Consequently, that row provides no direct information about the temporal coefficients. It can still inform non-temporal components of the model, such as covariate effects and non-temporal random effects. In Gaussian and binary models, it can also inform the intercept; in ordinal models, it can inform the category thresholds. Gaussian observations additionally inform the residual standard deviation. When skip_lag = FALSE, a current observation with an incomplete lag window is removed from the likelihood. Thus, all retained observations have a complete and correctly spaced lag window. This prevents effects spanning irregular time gaps from being interpreted as ordinary one-step or higher-order lagged relationships. This mechanism does not impute missing values or recover temporal information from incomplete lag windows. It determines whether current observations with unavailable lagged predictors are retained for estimating the non-temporal parts of the model.

Comparison: With and Without Lag Skipping

In the bvar function the skipping-lag mechanism can be turned on and off by using the skip_lag argument. To illustrate this, we can fit the model once using the lag skipping mechanism, and once without:

fit_no_skip_lag <- bvar(
  id_col = "id",
  time_col = "day",
  y_cols = c("anxious", "calm", "conventional", "critical", "dependable"),
  x_cols = NULL,
  re_temporal = FALSE,
  K = 1,
  skip_lag = FALSE,
  data = studentlife,
  family = c("ordinal"),
  priors = set_priors(),
  iter = 4000,
  warmup = 1000,
  chains = 4,
  cores = 4,
  seed = 1337
)

fit_skip_lag <- bvar(
  id_col = "id",
  time_col = "day",
  y_cols = c("anxious", "calm", "conventional", "critical", "dependable"),
  x_cols = NULL,
  re_temporal = FALSE,
  K = 1,
  skip_lag = TRUE,
  data = studentlife,
  family = c("ordinal"),
  priors = set_priors(),
  iter = 4000,
  warmup = 1000,
  chains = 4,
  cores = 4,
  seed = 1337
)

Model Output

We can print an overview of the models we just ran using the print(fit) functions:

print(fit_no_skip_lag)
#> BVAR Network fit
#> ======================================== 
#> Family:      ordinal
#> Outcomes (p): 5 
#> Lags (K):     1 
#> Fixed eff.:   0 
#> Observations: 67 
#> Rhat max:    1.001
#> Divergences: 4  WARNING: check model/priors.
#> Priors:       beta ~ Normal(0, 1), phi ~ Normal(0, 0.5), kappa ~ Normal(0, 2) (all defaults)
#> Total time:  8.3 sec
#> ========================================
print(fit_skip_lag)
#> BVAR Network fit
#> ======================================== 
#> Family:      ordinal
#> Outcomes (p): 5 
#> Lags (K):     1 
#> Fixed eff.:   0 
#> Observations: 147 
#> Rhat max:    1.001
#> Divergences: 2  WARNING: check model/priors.
#> Priors:       beta ~ Normal(0, 1), phi ~ Normal(0, 0.5), kappa ~ Normal(0, 2) (all defaults)
#> Total time:  14.7 sec
#> ========================================

Here we get information about the number of variables and lags, the number of observations and some first indications if the model did converge.

To further inspect the model parameters we can use the summary(fit) function:

summary(fit_no_skip_lag)
#> Error in `q[1L, ]`:
#> ! incorrect number of dimensions
summary(fit_skip_lag)
#> Error in `q[1L, ]`:
#> ! incorrect number of dimensions

Why does skip_lag = FALSE produce fewer observations?

For the demonstrated VAR(1) model, an observation has a valid lag only when an observation is available exactly one time unit earlier. More generally, a VAR(\(K\)) model requires observations at every time \(t-1,\ldots,t-K\).

When skip_lag = FALSE, a current observation with an incomplete lag window is removed from the likelihood. When skip_lag = TRUE, the current observation is retained, but all its lagged predictors are set to zero. The row therefore provides no direct information about the temporal coefficients, temporal random effects, or interactions involving lagged predictors. It can still inform the non-temporal components of the model.

The reported number of observations is the number of rows entering the likelihood after listwise deletion, removal of the first \(K\) observations per subject, and application of the selected skip-lag rule.

Practical implications

Neither option imputes missing values or recovers temporal information from incomplete lag windows. If listwise deletion is inappropriate for the missingness process, users should consider a suitable imputation procedure or an explicit model for the missing data.