Function for Fitting Multi-Season Single-Species Spatial Integrated Occupancy Models Using Polya-Gamma Latent Variables

Function for fitting single-species multi-season spatial integrated occupancy models using Polya-Gamma latent variables. Data integration is done using a joint likelihood framework, assuming distinct detection models for each data source that are each conditional on a single latent occurrence process. Models are fit using Nearest Neighbor Gaussian Processes.

Usage

stIntPGOcc(occ.formula, det.formula, data, inits, priors, tuning, 
           cov.model = 'exponential', NNGP = TRUE, n.neighbors = 15, 
           search.type = 'cb', n.batch, batch.length, accept.rate = 0.43, 
           n.omp.threads = 1, verbose = TRUE, ar1 = FALSE, n.report = 100, 
           n.burn = round(.10 * n.batch * batch.length), n.thin = 1, n.chains = 1, ...)

Arguments

occ.formula: a symbolic description of the model to be fit for the occurrence portion of the model using R's model syntax. Only right-hand side of formula is specified. See example below. Random intercepts are allowed using lme4 syntax (Bates et al. 2015).
det.formula: a list of symbolic descriptions of the models to be fit for the detection portion of the model using R's model syntax for each data set. Each element in the list is a formula for the detection model of a given data set. Only right-hand side of formula is specified. See example below. Random intercepts are allowed using lme4 syntax (Bates et al. 2015).
data: a list containing data necessary for model fitting. Valid tags are y, occ.covs, det.covs, sites, seasons, and coords. y is a list of three-dimensional arrays with first dimensional equal to the number of sites surveyed in that data set, second dimension equal to the number of primary time periods (i.e., years or seasons), and third dimension equal to the maximum number of replicate surveys at a site within a given season. occ.covs is a list of variables included in the occurrence portion of the model. Each list element is a different occurrence covariate, which can be site level or site/primary time period level. Site-level covariates are specified as a vector of length \(J\) while site/primary time period level covariates are specified as a matrix with rows corresponding to sites and columns corresponding to primary time periods. det.covs is a list of variables included in the detection portion of the model for each data source. det.covs should have the same number of elements as y, where each element is itself a list. Each element of the list for a given data source is a different detection covariate, which can be site-level , site-season-level, or observation-level. Site-level covariates and site/primary time period level covariates are specified in the same manner as occ.covs. Observation-level covariates are specified as a three-dimensional array with first dimension corresponding to sites, second dimension corresponding to primary time period, and third dimension corresponding to replicate. sites is a list of site indices with number of elements equal to the number of data sources being modeled. Each element contains a vector of length equal to the number of sites that specific data source contains. Each value in the vector indicates the corresponding site in occ.covs covariates that corresponds with the specific row of the detection-nondetection data for the data source. This is used to properly link sites across data sets. Similarly, seasons is a list of season indices with number of elements equal to the number of data sources being modeled. Each element contains a vector of length equal to the number of seasons that a specific data source is available for. This is used to properly link seasons across data sets. Each value in the vector indicates the corresponding season in occ.covs covariates that correspond with the specific column of the detection-nondetection data for the given data source. This is used to properly link seasons across data sets, which can have a differing number of seasons surveyed. coords is a matrix of the observation site coordinates. Note that spOccupancy assumes coordinates are specified in a projected coordinate system.
inits: a list with each tag corresponding to a parameter name. Valid tags are z, beta, alpha, sigma.sq.psi, sigma.sq.p, sigma.sq.t, rho, phi, w, nu, sigma.sq. The value portion of each tag is the parameter's initial value. sigma.sq.psi and sigma.sq.p are only relevant when including random effects in the occurrence and detection portion of the occupancy model, respectively. sigma.sq.t and rho are only relevant when ar1 = TRUE. The tag alpha is a list comprised of the initial values for the detection parameters for each data source. Each element of the list should be a vector of initial values for all detection parameters in the given data source or a single value for each data source to assign all parameters for a given data source the same initial value. See priors description for definition of each parameter name. Additionally, the tag fix can be set to TRUE to fix the starting values across all chains. If fix is not specified (the default), starting values are varied randomly across chains.
priors: a list with each tag corresponding to a parameter name. Valid tags are beta.normal, alpha.normal, sigma.sq.psi.ig, sigma.sq.p.ig, sigma.sq.t.ig, rho.unif, phi.unif, nu.unif, sigma.sq.ig, and sigma.sq.unif. Occupancy (beta) and detection (alpha) regression coefficients are assumed to follow a normal distribution. For beta hyperparameters of the normal distribution are passed as a list of length two with the first and second elements corresponding to the mean and variance of the normal distribution, which are each specified as vectors of length equal to the number of coefficients to be estimated or of length one if priors are the same for all coefficients. For the detection coefficients alpha, the mean and variance hyperparameters are themselves passed in as lists, with each element of the list corresponding to the specific hyperparameters for the detection parameters in a given data source. If not specified, prior means are set to 0 and prior variances set to 2.72. sigma.sq.psi and sigma.sq.p are the random effect variances for any unstructured occurrence or detection random effects, respectively, and are assumed to follow an inverse Gamma distribution. The hyperparameters of the inverse-Gamma distribution are passed as a list of length two with first and second elements corresponding to the shape and scale parameters, respectively, which are each specified as vectors of length equal to the number of random intercepts or of length one if priors are the same for all random effect variances. sigma.sq.t and rho are the AR(1) variance and correlation parameters for the AR(1) zero-mean temporal random effects, respectively. sigma.sq.t is assumed to follow an inverse-Gamma distribution, where the hyperparameters are specified as a vector with elements corresponding to the shape and scale parameters, respectively. rho is assumed to follow a uniform distribution, where the hyperparameters are specified in a vector of length two with elements corresponding to the lower and upper bounds of the uniform prior. sigma.sq, is assumed to follow an inverse-Gamma distribution or a uniform distribution (default is inverse-Gamma). The spatial decay phi and smoothness nu parameters are assumed to follow Uniform distributions. The hyperparameters of the inverse-Gamma are passed as a vector of length two, with the first and second elements corresponding to the shape and scale, respectively. The hyperparameters of the Uniform are also passed as a vector of length two with the first and second elements corresponding to the lower and upper support, respectively.
tuning: a list with each tag corresponding to a parameter name. Valid tags are rho, phi, and nu. The value portion of each tag defines the initial tuning variance of the Adaptive sampler. See Roberts and Rosenthal (2009) for details.
cov.model: a quoted keyword that specifies the covariance function used to model the spatial dependence structure among the observations. Supported covariance model key words are: "exponential", "matern", "spherical", and "gaussian".
NNGP: if TRUE, model is fit with an NNGP. If FALSE, a full Gaussian process is used. Currently only NNGP models are supported.
n.neighbors: number of neighbors used in the NNGP. Only used if NNGP = TRUE. Datta et al. (2016) showed that 15 neighbors is usually sufficient, but that as few as 5 neighbors can be adequate for certain data sets, which can lead to even greater decreases in run time. We recommend starting with 15 neighbors (the default) and if additional gains in computation time are desired, subsequently compare the results with a smaller number of neighbors using WAIC.
search.type: a quoted keyword that specifies the type of nearest neighbor search algorithm. Supported method key words are: "cb" and "brute". The "cb" should generally be much faster. If locations do not have identical coordinate values on the axis used for the nearest neighbor ordering then "cb" and "brute" should produce identical neighbor sets. However, if there are identical coordinate values on the axis used for nearest neighbor ordering, then "cb" and "brute" might produce different, but equally valid, neighbor sets, e.g., if data are on a grid.
n.batch: the number of MCMC batches in each chain to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
batch.length: the length of each MCMC batch in each chain to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
accept.rate: target acceptance rate for Adaptive MCMC. Default is 0.43. See Roberts and Rosenthal (2009) for details.
n.omp.threads: a positive integer indicating the number of threads to use for SMP parallel processing within chains. The package must be compiled for OpenMP support. For most Intel-based machines, we recommend setting n.omp.threads up to the number of hyperthreaded cores. Note, n.omp.threads > 1 might not work on some systems. Currently only relevant for spatial models.
verbose: if TRUE, messages about data preparation, model specification, and progress of the sampler are printed to the screen. Otherwise, no messages are printed.
ar1: logical value indicating whether to include an AR(1) zero-mean temporal random effect in the model. If FALSE, the model is fit without an AR(1) temporal autocovariance structure. If TRUE, an AR(1) random effect is included in the model to account for temporal autocorrelation across the primary time periods.
n.report: the interval to report MCMC progress. Note this is specified in terms of batches, not MCMC samples.
n.burn: the number of samples out of the total n.samples to discard as burn-in for each chain. By default, the first 10% of samples is discarded.
n.thin: the thinning interval for collection of MCMC samples. The thinning occurs after the n.burn samples are discarded. Default value is set to 1.
n.chains: the number of chains to run in sequence.
...: currently no additional arguments

Note

Some of the underlying code used for generating random numbers from the Polya-Gamma distribution is taken from the pgdraw package written by Daniel F. Schmidt and Enes Makalic. Their code implements Algorithm 6 in PhD thesis of Jesse Bennett Windle (2013) https://repositories.lib.utexas.edu/handle/2152/21842.

References

Polson, N.G., J.G. Scott, and J. Windle. (2013) Bayesian Inference for Logistic Models Using Polya-Gamma Latent Variables. Journal of the American Statistical Association, 108:1339-1349.

Bates, Douglas, Martin Maechler, Ben Bolker, Steve Walker (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01 .

Author

Jeffrey W. Doser doserjef@msu.edu

Value

An object of class stIntPGOcc that is a list comprised of:

beta.samples: a coda object of posterior samples for the occupancy regression coefficients.
alpha.samples: a coda object of posterior samples for the detection regression coefficients for all data sources.
z.samples: a three-dimensional array of posterior samples for the latent occupancy values, with dimensions corresponding to posterior sample, site, and primary time period. Note this object will contain predicted occupancy values for sites/primary time periods that were not sampled.
psi.samples: a three-dimensional array of posterior samples for the latent occupancy probability values, with dimensions corresponding to posterior sample, site, and primary time period. Note this object will contain predicted occupancy probabilities for sites/primary time periods that were not sampled.
sigma.sq.psi.samples: a coda object of posterior samples for variances of random intercepts included in the occupancy portion of the model. Only included if random intercepts are specified in occ.formula.
sigma.sq.p.samples: a coda object of posterior samples for variances of random intercpets included in the detection portion of the model. Includes random effect variances for all data sources. Only included if random intercepts are specified in det.formula.
beta.star.samples: a coda object of posterior samples for the occurrence random effects. Only included if random intercepts are specified in occ.formula.
alpha.star.samples: a coda object of posterior samples for the detection random effects in any of the data sources. Only included if random intercepts are specified in at least one of the individual data set detection formulas in det.formula.
theta.samples: a coda object of posterior samples for spatial covariance parameters and temporal covariance parameters if ar1 = TRUE.
w.samples: a coda object of posterior samples for latent spatial random effects.
eta.samples: a coda object of posterior samples for the AR(1) random effects for each primary time period. Only included if ar1 = TRUE.
p.samples: a list of four-dimensional arrays consisting of the posterior samples of detection probability for each data source. For each data source, the dimensions of the four-dimensional array correspond to MCMC sample, site, season, and replicate within season.
like.samples: a two-dimensional array of posterior samples for the likelihood values associated with each site and primary time period, for each individual data source. Used for calculating WAIC.
rhat: a list of Gelman-Rubin diagnostic values for some of the model parameters.
ESS: a list of effective sample sizes for some of the model parameters.
run.time: execution time reported using proc.time().

The return object will include additional objects used for subsequent prediction and/or model fit evaluation.

Examples

set.seed(332)

# Simulate Data -----------------------------------------------------------
# Number of locations in each direction. This is the total region of interest
# where some sites may or may not have a data source.
J.x <- 15 
J.y <- 15
J.all <- J.x * J.y
# Number of data sources.
n.data <- 3
# Sites for each data source.
J.obs <- sample(ceiling(0.2 * J.all):ceiling(0.4 * J.all), n.data, replace = TRUE)
# Maximum number of years for each data set
n.time.max <- c(4, 8, 10)
# Number of years each site in each data set is sampled
n.time <- list()
for (i in 1:n.data) {
  n.time[[i]] <- sample(1:n.time.max[i], J.obs[i], replace = TRUE)
}
# Replicates for each data source.
n.rep <- list()
for (i in 1:n.data) {
  n.rep[[i]] <- matrix(NA, J.obs[i], n.time.max[i])
  for (j in 1:J.obs[i]) {
    n.rep[[i]][j, sample(1:n.time.max[i], n.time[[i]][j], replace = FALSE)] <- 
      sample(1:4, n.time[[i]][j], replace = TRUE)
  }
}
# Total number of years across all data sets
n.time.total <- 10
# List denoting the specific years each data set was sampled during. 
data.seasons <- list()
for (i in 1:n.data) {
  data.seasons[[i]] <- sort(sample(1:n.time.total, n.time.max[i], replace = FALSE))
}

# Occupancy covariates
beta <- c(0, 0.4, 0.3)
trend <- TRUE
# Random occupancy covariates
psi.RE <- list(levels = c(20),
               sigma.sq.psi = c(0.6))
p.occ <- length(beta)
# Detection covariates
alpha <- list()
alpha[[1]] <- c(0, 0.2, -0.5)
alpha[[2]] <- c(-1, 0.5, 0.3, -0.8)
alpha[[3]] <- c(-0.5, 1)

p.RE <- list()
p.det.long <- sapply(alpha, length)
p.det <- sum(p.det.long)

# Spatial parameters
sigma.sq <- 0.9
phi <- 3 / .5

# Simulate occupancy data.
dat <- simTIntOcc(n.data = n.data, J.x = J.x, J.y = J.y, J.obs = J.obs,
                  n.time = n.time, data.seasons = data.seasons, n.rep = n.rep,
                  beta = beta, alpha = alpha, trend = trend, 
                  psi.RE = psi.RE, p.RE = p.RE, sp = TRUE, 
                  sigma.sq = sigma.sq, phi = phi, cov.model = 'exponential')

y <- dat$y
X <- dat$X.obs
X.re <- dat$X.re.obs
X.p <- dat$X.p
sites <- dat$sites
coords <- dat$coords.obs

# Package all data into a list
occ.covs <- list(trend = X[, , 2], 
                 occ.cov.1 = X[, , 3], 
                 occ.factor.1 = X.re[, , 1])
det.covs <- list()
# Add covariates one by one
det.covs[[1]] <- list(det.cov.1.1 = X.p[[1]][, , , 2],
                      det.cov.1.2 = X.p[[1]][, , , 3])
det.covs[[2]] <- list(det.cov.2.1 = X.p[[2]][, , , 2],
                      det.cov.2.2 = X.p[[2]][, , , 3],
                      det.cov.2.3 = X.p[[2]][, , , 4])
det.covs[[3]] <- list(det.cov.3.1 = X.p[[3]][, , , 2])
data.list <- list(y = y, occ.covs = occ.covs, det.covs = det.covs,
                  sites = sites, seasons = data.seasons, coords = coords)

# Testing
occ.formula <- ~ trend + occ.cov.1 + (1 | occ.factor.1)
# Note that the names are not necessary.
det.formula <- list(f.1 = ~ det.cov.1.1 + det.cov.1.2,
                    f.2 = ~ det.cov.2.1 + det.cov.2.2 + det.cov.2.3,
                    f.3 = ~ det.cov.3.1)

# NOTE: this is a short run of the model, in reality we would run the 
#       model for much longer.
out <- stIntPGOcc(occ.formula = occ.formula,
                 det.formula = det.formula,
                 data = data.list,
                 NNGP = TRUE, 
                 n.neighbors = 15, 
                 cov.model = 'exponential',
                 n.batch = 3,
                 batch.length = 25, 
                 n.report = 1,
                 n.burn = 25,
                 n.thin = 1,
                 n.chains = 1)
#> ----------------------------------------
#> 	Preparing the data
#> ----------------------------------------
#> No prior specified for beta.normal.
#> Setting prior mean to 0 and prior variance to 2.72
#> No prior specified for alpha.normal.
#> Setting prior mean to 0 and prior variance to 2.72
#> No prior specified for phi.unif.
#> Setting uniform bounds based on the range of observed spatial coordinates.
#> No prior specified for sigma.sq.
#> Using an inverse-Gamma prior with the shape parameter set to 2 and scale parameter to 1.
#> No prior specified for sigma.sq.psi.ig.
#> Setting prior shape to 0.1 and prior scale to 0.1
#> z is not specified in initial values.
#> Setting initial values based on observed data
#> beta is not specified in initial values.
#> Setting initial values to random values from the prior distribution
#> alpha is not specified in initial values.
#> Setting initial values to random values from the prior distribution
#> phi is not specified in initial values.
#> Setting initial value to random value from the prior distribution
#> sigma.sq is not specified in initial values.
#> Setting initial value to random value from the prior distribution
#> sigma.sq.psi is not specified in initial values.
#> Setting initial values to random values between 0.5 and 10
#> ----------------------------------------
#> 	Building the neighbor list
#> ----------------------------------------
#> ----------------------------------------
#> Building the neighbors of neighbors list
#> ----------------------------------------
#> ----------------------------------------
#> 	Model description
#> ----------------------------------------
#> Spatial integrated Multi-season Occupancy Model with Polya-Gamma latent
#> variable fit with 157 sites and 10 primary time periods.
#> 
#> Integrating 3 occupancy data sets.
#> 
#> Samples per chain: 75 (3 batches of length 25)
#> Burn-in: 25 
#> Thinning Rate: 1 
#> Number of Chains: 1 
#> Total Posterior Samples: 50 
#> 
#> Using the exponential spatial correlation model.
#> 
#> Using 15 nearest neighbors.
#> 
#> Source compiled with OpenMP support and model fit using 1 thread(s).
#> 
#> Adaptive Metropolis with target acceptance rate: 43.0
#> ----------------------------------------
#> 	Chain 1
#> ----------------------------------------
#> Sampling ... 
#> Batch: 1 of 3, 33.33%
#> 	Parameter	Acceptance	Tuning
#> 	phi		40.0		0.98020
#> -------------------------------------------------
#> Batch: 2 of 3, 66.67%
#> 	Parameter	Acceptance	Tuning
#> 	phi		32.0		0.97045
#> -------------------------------------------------
#> Batch: 3 of 3, 100.00%
summary(out)
#> 
#> Call:
#> stIntPGOcc(occ.formula = occ.formula, det.formula = det.formula, 
#>     data = data.list, cov.model = "exponential", NNGP = TRUE, 
#>     n.neighbors = 15, n.batch = 3, batch.length = 25, n.report = 1, 
#>     n.burn = 25, n.thin = 1, n.chains = 1)
#> 
#> Samples per Chain: 75
#> Burn-in: 25
#> Thinning Rate: 1
#> Number of Chains: 1
#> Total Posterior Samples: 50
#> Run Time (min): 0.0025
#> 
#> ----------------------------------------
#> Occurrence
#> ----------------------------------------
#> Fixed Effects (logit scale):
#>                Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept) -0.7920 0.2071 -1.1245 -0.8288 -0.4297   NA   5
#> trend        0.4833 0.1235  0.2555  0.4759  0.7343   NA  25
#> occ.cov.1    0.0045 0.1450 -0.2394  0.0116  0.2752   NA  19
#> 
#> Random Effect Variances (logit scale):
#>                Mean     SD   2.5%    50%  97.5% Rhat ESS
#> occ.factor.1 0.1792 0.0842 0.0657 0.1657 0.3709   NA  25
#> 
#> ----------------------------------------
#> Data source 1 Detection
#> ----------------------------------------
#> Fixed Effects (logit scale):
#>                Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept)  0.1448 0.2638 -0.3686  0.1603  0.5314   NA  18
#> det.cov.1.1  0.2159 0.1581 -0.0161  0.2210  0.5083   NA  49
#> det.cov.1.2 -0.5212 0.1742 -0.8115 -0.5428 -0.1920   NA  63
#> 
#> ----------------------------------------
#> Data source 2 Detection
#> ----------------------------------------
#> Fixed Effects (logit scale):
#>                Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept) -0.6702 0.1144 -0.8598 -0.6641 -0.4619   NA  20
#> det.cov.2.1  0.7097 0.1670  0.4116  0.6803  1.0473   NA  27
#> det.cov.2.2  0.3416 0.1253  0.1294  0.3370  0.5851   NA  50
#> det.cov.2.3 -0.6946 0.1549 -0.9372 -0.7242 -0.4307   NA  25
#> 
#> ----------------------------------------
#> Data source 3 Detection
#> ----------------------------------------
#> Fixed Effects (logit scale):
#>                Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept) -0.3646 0.1799 -0.7199 -0.3735 -0.0147   NA  33
#> det.cov.3.1  1.2427 0.1960  0.8974  1.2678  1.5629   NA  27
#> 
#> ----------------------------------------
#> Occurrence Spatio-temporal Covariance: 
#> ----------------------------------------
#>             Mean     SD   2.5%     50%   97.5% Rhat ESS
#> sigma.sq  0.9139 0.2586 0.5134  0.9372  1.2962   NA   3
#> phi      17.7453 4.8288 8.2630 17.5119 24.9751   NA   2