Function for Fitting Multi-Species Spatial Occupancy Models Using Polya-Gamma Latent Variables

The function spMsPGOcc fits multi-species spatial occupancy models using Polya-Gamma latent variables. Models can be fit using either a full Gaussian process or a Nearest Neighbor Gaussian Process for large data sets.

Usage

spMsPGOcc(occ.formula, det.formula, data, inits, priors, tuning, 
          cov.model = 'exponential', NNGP = TRUE, 
          n.neighbors = 15, search.type = 'cb', n.batch, 
          batch.length, accept.rate = 0.43, n.omp.threads = 1, 
          verbose = TRUE, n.report = 100, 
          n.burn = round(.10 * n.batch * batch.length), n.thin = 1, 
          n.chains = 1, k.fold, k.fold.threads = 1, k.fold.seed, 
          k.fold.only = FALSE, ...)

Arguments

occ.formula: a symbolic description of the model to be fit for the occurrence portion of the model using R's model syntax. Only right-hand side of formula is specified. See example below. Random intercepts are allowed using lme4 syntax (Bates et al. 2015).
det.formula: a symbolic description of the model to be fit for the detection portion of the model using R's model syntax. Only right-hand side of formula is specified. See example below. Random intercepts are allowed using lme4 syntax (Bates et al. 2015).
data: a list containing data necessary for model fitting. Valid tags are y, occ.covs, det.covs, coords. y is a three-dimensional array with first dimension equal to the number of species, second dimension equal to the number of sites, and third dimension equal to the maximum number of replicates at a given site. occ.covs is a matrix or data frame containing the variables used in the occurrence portion of the model, with \(J\) rows for each column (variable). det.covs is a list of variables included in the detection portion of the model. Each list element is a different detection covariate, which can be site-level or observational-level. Site-level covariates are specified as a vector of length \(J\) while observation-level covariates are specified as a matrix or data frame with the number of rows equal to \(J\) and number of columns equal to the maximum number of replicates at a given site. coords is a \(J \times 2\) matrix of the observation coordinates. Note that spOccupancy assumes coordinates are specified in a projected coordinate system.
inits: a list with each tag corresponding to a parameter name. Valid tags are alpha.comm, beta.comm, beta, alpha, tau.sq.beta, tau.sq.alpha, sigma.sq.psi, sigma.sq.p, z, sigma.sq, phi, w, and nu. nu is only specified if cov.model = "matern", sigma.sq.psi is only specified if there are random intercepts in occ.formula, and sigma.sq.p is only specified if there are random intercpets in det.formula. The value portion of each tag is the parameter's initial value. See priors description for definition of each parameter name. Additionally, the tag fix can be set to TRUE to fix the starting values across all chains. If fix is not specified (the default), starting values are varied randomly across chains.
priors: a list with each tag corresponding to a parameter name. Valid tags are beta.comm.normal, alpha.comm.normal, tau.sq.beta.ig, tau.sq.alpha.ig, phi.unif, sigma.sq.ig, sigma.sq.unif, nu.unif, sigma.sq.psi, sigma.sq.p. Community-level occurrence (beta.comm) and detection (alpha.comm) regression coefficients are assumed to follow a normal distribution. The hyperparameters of the normal distribution are passed as a list of length two with the first and second elements corresponding to the mean and variance of the normal distribution, which are each specified as vectors of length equal to the number of coefficients to be estimated or of length one if priors are the same for all coefficients. If not specified, prior means are set to 0 and prior variances set to 2.73. Community-level variance parameters for occupancy (tau.sq.beta) and detection (tau.sq.alpha) are assumed to follow an inverse Gamma distribution. The hyperparameters of the inverse gamma distribution are passed as a list of length two with the first and second elements corresponding to the shape and scale parameters, which are each specified as vectors of length equal to the number of coefficients to be estimated or a single value if priors are the same for all parameters. If not specified, prior shape and scale parameters are set to 0.1. The species-specific spatial variance parameter, sigma.sq, is assumed to follow an inverse-Gamma distribution or a uniform distribution (default is inverse-Gamma). sigma.sq of all species can also be fixed at its initial value by setting the prior value to "fixed". The spatial decay phi and smoothness nu parameters are assumed to follow Uniform distributions. The hyperparameters of the inverse-Gamma are passed as a list of length two, with the list elements being vectors of length N corresponding to the species-specific shape and scale parameters, respectively, or a single value if the same value is assigned for all species. The hyperparameters of the Uniform are also passed as a list with two elements, with both elements being vectors of length N corresponding to the lower and upper support, respectively, or as a single value if the same value is assigned for all species. sigma.sq.psi and sigma.sq.p are the random effect variances for any occurrence or detection random effects, respectively, and are assumed to follow an inverse Gamma distribution. The hyperparameters of the inverse-Gamma distribution are passed as a list of length two with first and second elements corresponding to the shape and scale parameters, respectively, which are each specified as vectors of length equal to the number of random intercepts or of length one if priors are the same for all random effect variances.
tuning: a list with each tag corresponding to a parameter name. Valid tags are phi and nu. The value portion of each tag defines the initial variance of the adaptive sampler. We assume the initial variance of the adaptive sampler is the same for each species, although the adaptive sampler will adjust the tuning variances separately for each species. See Roberts and Rosenthal (2009) for details.
cov.model: a quoted keyword that specifies the covariance function used to model the spatial dependence structure among the observations. Supported covariance model key words are: "exponential", "matern", "spherical", and "gaussian".
NNGP: if TRUE, model is fit with an NNGP. If FALSE, a full Gaussian process is used. See Datta et al. (2016) and Finley et al. (2019) for more information.
n.neighbors: number of neighbors used in the NNGP. Only used if NNGP = TRUE. Datta et al. (2016) showed that 15 neighbors is usually sufficient, but that as few as 5 neighbors can be adequate for certain data sets, which can lead to even greater decreases in run time. We recommend starting with 15 neighbors (the default) and if additional gains in computation time are desired, subsequently compare the results with a smaller number of neighbors using WAIC or k-fold cross-validation.
search.type: a quoted keyword that specifies the type of nearest neighbor search algorithm. Supported method key words are: "cb" and "brute". The "cb" should generally be much faster. If locations do not have identical coordinate values on the axis used for the nearest neighbor ordering then "cb" and "brute" should produce identical neighbor sets. However, if there are identical coordinate values on the axis used for nearest neighbor ordering, then "cb" and "brute" might produce different, but equally valid, neighbor sets, e.g., if data are on a grid.
n.batch: the number of MCMC batches in each chain to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
batch.length: the length of each MCMC batch to run for the Adaptive MCMC sampler. See Roberts and Rosenthal (2009) for details.
accept.rate: target acceptance rate for Adaptive MCMC. Defaul is 0.43. See Roberts and Rosenthal (2009) for details.
n.omp.threads: a positive integer indicating the number of threads to use for SMP parallel processing. The package must be compiled for OpenMP support. For most Intel-based machines, we recommend setting n.omp.threads up to the number of hyperthreaded cores. Note, n.omp.threads > 1 might not work on some systems.
verbose: if TRUE, messages about data preparation, model specification, and progress of the sampler are printed to the screen. Otherwise, no messages are printed.
n.report: the interval to report Metropolis sampler acceptance and MCMC progress. Note this is specified in terms of batches and not overall samples for spatial models.
n.burn: the number of samples out of the total n.samples to discard as burn-in for each chain. By default, the first 10% of samples is discarded.
n.thin: the thinning interval for collection of MCMC samples. The thinning occurs after the n.burn samples are discarded. Default value is set to 1.
n.chains: the number of chains to run in sequence.
k.fold: specifies the number of k folds for cross-validation. If not specified as an argument, then cross-validation is not performed and k.fold.threads and k.fold.seed are ignored. In k-fold cross-validation, the data specified in data is randomly partitioned into k equal sized subsamples. Of the k subsamples, k - 1 subsamples are used to fit the model and the remaining k samples are used for prediction. The cross-validation process is repeated k times (the folds). As a scoring rule, we use the model deviance as described in Hooten and Hobbs (2015). Cross-validation is performed after the full model is fit using all the data. Cross-validation results are reported in the k.fold.deviance object in the return list.
k.fold.threads: number of threads to use for cross-validation. If k.fold.threads > 1 parallel processing is accomplished using the foreach and doParallel packages. Ignored if k.fold is not specified.
k.fold.seed: seed used to split data set into k.fold parts for k-fold cross-validation. Ignored if k.fold is not specified.
k.fold.only: a logical value indicating whether to only perform cross-validation (TRUE) or perform cross-validation after fitting the full model (FALSE). Default value is FALSE.
...: currently no additional arguments

Note

Some of the underlying code used for generating random numbers from the Polya-Gamma distribution is taken from the pgdraw package written by Daniel F. Schmidt and Enes Makalic. Their code implements Algorithm 6 in PhD thesis of Jesse Bennett Windle (2013) https://repositories.lib.utexas.edu/handle/2152/21842.

References

Datta, A., S. Banerjee, A.O. Finley, and A.E. Gelfand. (2016) Hierarchical Nearest-Neighbor Gaussian process models for large geostatistical datasets. Journal of the American Statistical Association, doi:10.1080/01621459.2015.1044091 .

Finley, A.O., A. Datta, B.D. Cook, D.C. Morton, H.E. Andersen, and S. Banerjee. (2019) Efficient algorithms for Bayesian Nearest Neighbor Gaussian Processes. Journal of Computational and Graphical Statistics, doi:10.1080/10618600.2018.1537924 .

Finley, A. O., Datta, A., and Banerjee, S. (2020). spNNGP R package for nearest neighbor Gaussian process models. arXiv preprint arXiv:2001.09111.

Polson, N.G., J.G. Scott, and J. Windle. (2013) Bayesian Inference for Logistic Models Using Polya-Gamma Latent Variables. Journal of the American Statistical Association, 108:1339-1349.

Roberts, G.O. and Rosenthal J.S. (2009) Examples of adaptive MCMC. Journal of Computational and Graphical Statistics, 18(2):349-367.

Bates, Douglas, Martin Maechler, Ben Bolker, Steve Walker (2015). Fitting Linear Mixed-Effects Models Using lme4. Journal of Statistical Software, 67(1), 1-48. doi:10.18637/jss.v067.i01 .

Hooten, M. B., and Hobbs, N. T. (2015). A guide to Bayesian model selection for ecologists. Ecological Monographs, 85(1), 3-28.

Author

Jeffrey W. Doser doserjef@msu.edu,
Andrew O. Finley finleya@msu.edu

Value

An object of class spMsPGOcc that is a list comprised of:

beta.comm.samples: a coda object of posterior samples for the community level occurrence regression coefficients.
alpha.comm.samples: a coda object of posterior samples for the community level detection regression coefficients.
tau.sq.beta.samples: a coda object of posterior samples for the occurrence community variance parameters.
tau.sq.alpha.samples: a coda object of posterior samples for the detection community variance parameters.
beta.samples: a coda object of posterior samples for the species level occurrence regression coefficients.
alpha.samples: a coda object of posterior samples for the species level detection regression coefficients.
theta.samples: a coda object of posterior samples for the species level covariance parameters.
z.samples: a three-dimensional array of posterior samples for the latent occurrence values for each species.
psi.samples: a three-dimensional array of posterior samples for the latent occupancy probability values for each species.
w.samples: a three-dimensional array of posterior samples for the latent spatial random effects for each species.
sigma.sq.psi.samples: a coda object of posterior samples for variances of random intercepts included in the occurrence portion of the model. Only included if random intercepts are specified in occ.formula.
sigma.sq.p.samples: a coda object of posterior samples for variances of random intercpets included in the detection portion of the model. Only included if random intercepts are specified in det.formula.
alpha.star.samples: a coda object of posterior samples for the detection random effects. Only included if random intercepts are specified in det.formula.
beta.star.samples: a coda object of posterior samples for the occurrence random effects. Only included if random intercepts are specified in occ.formula.
like.samples: a three-dimensional array of posterior samples for the likelihood value associated with each site and species. Used for calculating WAIC.
rhat: a list of Gelman-Rubin diagnostic values for some of the model parameters.
ESS: a list of effective sample sizes for some of the model parameters.
run.time: MCMC sampler execution time reported using proc.time().
k.fold.deviance: vector of scoring rules (deviance) from k-fold cross-validation. A separate value is reported for each species. Only included if k.fold is specified in function call.

The return object will include additional objects used for subsequent prediction and/or model fit evaluation. Note that detection probability estimated values are not included in the model object, but can be extracted using fitted().

Examples

set.seed(400)

# Simulate Data -----------------------------------------------------------
J.x <- 7
J.y <- 7
J <- J.x * J.y
n.rep <- sample(2:4, size = J, replace = TRUE)
N <- 5
# Community-level covariate effects
# Occurrence
beta.mean <- c(0.2, -0.15)
p.occ <- length(beta.mean)
tau.sq.beta <- c(0.6, 0.3)
# Detection
alpha.mean <- c(0.5, 0.2, -.2)
tau.sq.alpha <- c(0.2, 0.3, 0.8)
p.det <- length(alpha.mean)
# Draw species-level effects from community means.
beta <- matrix(NA, nrow = N, ncol = p.occ)
alpha <- matrix(NA, nrow = N, ncol = p.det)
for (i in 1:p.occ) {
  beta[, i] <- rnorm(N, beta.mean[i], sqrt(tau.sq.beta[i]))
}
for (i in 1:p.det) {
  alpha[, i] <- rnorm(N, alpha.mean[i], sqrt(tau.sq.alpha[i]))
}
phi <- runif(N, 3/1, 3/.4)
sigma.sq <- runif(N, 0.3, 3)
sp <- TRUE

dat <- simMsOcc(J.x = J.x, J.y = J.y, n.rep = n.rep, N = N, beta = beta, alpha = alpha,
                phi = phi, sigma.sq = sigma.sq, sp = TRUE, cov.model = 'exponential')

# Number of batches
n.batch <- 30
# Batch length
batch.length <- 25
n.samples <- n.batch * batch.length

y <- dat$y
X <- dat$X
X.p <- dat$X.p
coords <- as.matrix(dat$coords)

# Package all data into a list
occ.covs <- X[, 2, drop = FALSE]
colnames(occ.covs) <- c('occ.cov')
det.covs <- list(det.cov.1 = X.p[, , 2], 
                 det.cov.2 = X.p[, , 3])
data.list <- list(y = y, 
                  occ.covs = occ.covs,
                  det.covs = det.covs, 
                  coords = coords)
# Priors
prior.list <- list(beta.comm.normal = list(mean = 0, var = 2.72), 
                   alpha.comm.normal = list(mean = 0, var = 2.72), 
                   tau.sq.beta.ig = list(a = 0.1, b = 0.1), 
                   tau.sq.alpha.ig = list(a = 0.1, b = 0.1),
                   phi.unif = list(a = 3/1, b = 3/.1), 
                   sigma.sq.ig = list(a = 2, b = 2)) 
# Initial values
inits.list <- list(alpha.comm = 0, 
                   beta.comm = 0, 
                   beta = 0, 
                   alpha = 0,
                   tau.sq.beta = 1, 
                   tau.sq.alpha = 1, 
                   phi = 3 / .5, 
                   sigma.sq = 2,
                   w = matrix(0, nrow = N, ncol = nrow(X)),
                   z = apply(y, c(1, 2), max, na.rm = TRUE))
# Tuning
tuning.list <- list(phi = 1) 

out <- spMsPGOcc(occ.formula = ~ occ.cov, 
                 det.formula = ~ det.cov.1 + det.cov.2, 
                 data = data.list,
                 inits = inits.list, 
                 n.batch = n.batch, 
                 batch.length = batch.length, 
                 accept.rate = 0.43, 
                 priors = prior.list, 
                 cov.model = "exponential", 
                 tuning = tuning.list, 
                 n.omp.threads = 1, 
                 verbose = TRUE, 
                 NNGP = TRUE, 
                 n.neighbors = 5, 
                 search.type = 'cb', 
                 n.report = 10, 
                 n.burn = 500, 
                 n.thin = 1, 
                 n.chains = 1)
#> ----------------------------------------
#> 	Preparing to run the model
#> ----------------------------------------
#> ----------------------------------------
#> 	Building the neighbor list
#> ----------------------------------------
#> ----------------------------------------
#> Building the neighbors of neighbors list
#> ----------------------------------------
#> ----------------------------------------
#> 	Model description
#> ----------------------------------------
#> NNGP Multi-species Occupancy Model with Polya-Gamma latent
#> variable fit with 49 sites and 5 species.
#> 
#> Samples per chain: 750 (30 batches of length 25)
#> Burn-in: 500 
#> Thinning Rate: 1 
#> Number of Chains: 1 
#> Total Posterior Samples: 250 
#> 
#> Using the exponential spatial correlation model.
#> 
#> Using 5 nearest neighbors.
#> 
#> Source compiled with OpenMP support and model fit using 1 thread(s).
#> 
#> Adaptive Metropolis with target acceptance rate: 43.0
#> ----------------------------------------
#> 	Chain 1
#> ----------------------------------------
#> Sampling ... 
#> Batch: 10 of 30, 33.33%
#> 	Species		Parameter	Acceptance	Tuning
#> 	1		phi		56.0		1.11628
#> 	2		phi		88.0		1.11628
#> 	3		phi		68.0		1.11628
#> 	4		phi		64.0		1.09417
#> 	5		phi		64.0		1.11628
#> -------------------------------------------------
#> Batch: 20 of 30, 66.67%
#> 	Species		Parameter	Acceptance	Tuning
#> 	1		phi		72.0		1.20925
#> 	2		phi		68.0		1.23368
#> 	3		phi		72.0		1.23368
#> 	4		phi		68.0		1.20925
#> 	5		phi		60.0		1.23368
#> -------------------------------------------------
#> Batch: 30 of 30, 100.00%

summary(out, level = 'both')
#> 
#> Call:
#> spMsPGOcc(occ.formula = ~occ.cov, det.formula = ~det.cov.1 + 
#>     det.cov.2, data = data.list, inits = inits.list, priors = prior.list, 
#>     tuning = tuning.list, cov.model = "exponential", NNGP = TRUE, 
#>     n.neighbors = 5, search.type = "cb", n.batch = n.batch, batch.length = batch.length, 
#>     accept.rate = 0.43, n.omp.threads = 1, verbose = TRUE, n.report = 10, 
#>     n.burn = 500, n.thin = 1, n.chains = 1)
#> 
#> Samples per Chain: 750
#> Burn-in: 500
#> Thinning Rate: 1
#> Number of Chains: 1
#> Total Posterior Samples: 250
#> Run Time (min): 0.0134
#> 
#> ----------------------------------------
#> 	Community Level
#> ----------------------------------------
#> Occurrence Means (logit scale): 
#>                Mean     SD    2.5%     50%  97.5% Rhat ESS
#> (Intercept) -0.5603 0.3459 -1.3230 -0.5522 0.0667   NA 132
#> occ.cov      0.2236 0.4675 -0.5884  0.1915 1.1159   NA 159
#> 
#> Occurrence Variances (logit scale): 
#>               Mean     SD   2.5%    50% 97.5% Rhat ESS
#> (Intercept) 0.5255 0.7204 0.0490 0.3262 2.504   NA  84
#> occ.cov     0.9119 1.0404 0.1136 0.5661 3.561   NA 151
#> 
#> Detection Means (logit scale): 
#>                Mean     SD    2.5%     50%  97.5% Rhat ESS
#> (Intercept)  0.6923 0.4014 -0.1512  0.7097 1.3619   NA 191
#> det.cov.1   -0.1406 0.3732 -0.8184 -0.1407 0.5838   NA 250
#> det.cov.2   -0.7371 0.3899 -1.4805 -0.7458 0.0281   NA 198
#> 
#> Detection Variances (logit scale): 
#>               Mean     SD   2.5%    50%  97.5% Rhat ESS
#> (Intercept) 0.5720 0.8625 0.0773 0.3117 3.1149   NA  94
#> det.cov.1   0.5636 1.0550 0.0669 0.3142 2.2950   NA 139
#> det.cov.2   0.7929 2.0518 0.0666 0.3680 3.6109   NA 207
#> 
#> ----------------------------------------
#> 	Species Level
#> ----------------------------------------
#> Occurrence (logit scale): 
#>                    Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept)-sp1 -0.4986 0.3732 -1.4015 -0.4591  0.1607   NA  36
#> (Intercept)-sp2  0.0315 0.5399 -0.9897  0.0071  1.2356   NA  27
#> (Intercept)-sp3 -0.8632 0.4581 -1.8238 -0.8321 -0.1312   NA  70
#> (Intercept)-sp4 -0.5621 0.3875 -1.3699 -0.5510  0.1501   NA  76
#> (Intercept)-sp5 -0.9349 0.3737 -1.6936 -0.9412 -0.2332   NA  55
#> occ.cov-sp1     -0.3039 0.3692 -1.0844 -0.2975  0.4305   NA  85
#> occ.cov-sp2      0.3756 0.4800 -0.4693  0.3478  1.4301   NA  59
#> occ.cov-sp3     -0.3903 0.3892 -1.1544 -0.3885  0.3021   NA 109
#> occ.cov-sp4      0.8338 0.4191  0.0959  0.8414  1.7321   NA  82
#> occ.cov-sp5      0.6081 0.3673 -0.0874  0.6080  1.3166   NA 109
#> 
#> Detection (logit scale): 
#>                    Mean     SD    2.5%     50%   97.5% Rhat ESS
#> (Intercept)-sp1  0.8192 0.2854  0.2383  0.8298  1.4426   NA 149
#> (Intercept)-sp2  1.3120 0.2794  0.7893  1.2950  1.8684   NA 108
#> (Intercept)-sp3 -0.0206 0.4127 -0.8751 -0.0249  0.6941   NA  81
#> (Intercept)-sp4  0.6477 0.2607  0.1112  0.6665  1.0804   NA 120
#> (Intercept)-sp5  0.8066 0.3430  0.2020  0.8024  1.5481   NA 119
#> det.cov.1-sp1    0.0348 0.2227 -0.4183  0.0354  0.4701   NA 250
#> det.cov.1-sp2   -0.4639 0.2622 -0.9770 -0.4623  0.0229   NA  86
#> det.cov.1-sp3    0.4585 0.3141 -0.1611  0.4626  1.0326   NA  85
#> det.cov.1-sp4   -0.5733 0.2999 -1.1670 -0.5452 -0.0396   NA 128
#> det.cov.1-sp5   -0.2566 0.3257 -0.8902 -0.2541  0.4113   NA 119
#> det.cov.2-sp1   -0.3335 0.2838 -0.8683 -0.3442  0.2468   NA 156
#> det.cov.2-sp2   -0.8214 0.2594 -1.3761 -0.8133 -0.3119   NA 113
#> det.cov.2-sp3   -1.6268 0.5070 -2.8275 -1.5573 -0.8664   NA  48
#> det.cov.2-sp4   -0.6323 0.2364 -1.1286 -0.6295 -0.1775   NA 139
#> det.cov.2-sp5   -0.5728 0.2866 -1.1826 -0.5772 -0.0280   NA 140
#> 
#> Spatial Covariance: 
#>                 Mean      SD   2.5%     50%   97.5% Rhat ESS
#> sigma.sq-sp1  2.4237  2.3820 0.7931  1.4636  8.0265   NA  12
#> sigma.sq-sp2  8.8736 11.4231 0.5523  3.3895 37.9279   NA   4
#> sigma.sq-sp3  1.2398  0.7618 0.4545  1.0201  3.2219   NA  24
#> sigma.sq-sp4  2.2128  2.3557 0.4650  1.3911 10.1891   NA  10
#> sigma.sq-sp5  1.0453  0.6638 0.2436  0.9251  2.6913   NA  25
#> phi-sp1      13.7409  7.2005 4.4405 11.8798 27.6629   NA  17
#> phi-sp2      18.5615  6.1327 7.4803 18.2681 29.4330   NA  30
#> phi-sp3      19.7496  6.3174 7.1826 19.8693 29.2511   NA  28
#> phi-sp4      15.4098  6.5759 5.8415 14.2049 27.5318   NA  16
#> phi-sp5      12.6210  6.8492 3.5967 10.8899 26.3085   NA  21