R/epoisCensored.R
epoisCensored.RdEstimate the mean of a Poisson distribution given a sample of data that has been subjected to Type I censoring, and optionally construct a confidence interval for the mean.
epoisCensored(x, censored, method = "mle", censoring.side = "left",
ci = FALSE, ci.method = "profile.likelihood", ci.type = "two-sided",
conf.level = 0.95, n.bootstraps = 1000, pivot.statistic = "z",
ci.sample.size = sum(!censored))Extended methods and derivations are available in
Estimation with censored data: epoisCensored.
Open the vignette locally with
vignette("censored-estimation", package = "EnvStats").
numeric vector of observations. Missing (NA), undefined (NaN), and
infinite (Inf, -Inf) values are allowed but will be removed.
numeric or logical vector indicating which values of x are censored.
This must be the same length as x. If the mode of censored is
"logical", TRUE values correspond to elements of x that
are censored, and FALSE values correspond to elements of x that
are not censored. If the mode of censored is "numeric",
it must contain only 1's and 0's; 1 corresponds to
TRUE and 0 corresponds to FALSE. Missing (NA)
values are allowed but will be removed.
character string specifying the method of estimation. The possible values are:
"mle" (maximum likelihood; the default), and
"half.cen.level" (moment estimation based on setting the censored
observations to half the censoring level).
character string indicating on which side the censoring occurs. The possible
values are "left" (the default) and "right".
logical scalar indicating whether to compute a confidence interval for the
mean or variance. The default value is ci=FALSE.
character string indicating what method to use to construct the confidence interval
for the mean. The possible values are "profile.likelihood"
(profile likelihood; the default),
"normal.approx" (normal approximation),
and
"bootstrap" (based on bootstrapping).
See the DETAILS section for more information.
This argument is ignored if ci=FALSE.
character string indicating what kind of confidence interval to compute. The
possible values are "two-sided" (the default), "lower", and
"upper". This argument is ignored if ci=FALSE.
a scalar between 0 and 1 indicating the confidence level of the confidence interval.
The default value is conf.level=0.95. This argument is ignored if
ci=FALSE.
numeric scalar indicating how many bootstraps to use to construct the
confidence interval for the mean when ci.type="bootstrap". This
argument is ignored if ci=FALSE and/or ci.method does not
equal "bootstrap".
character string indicating which pivot statistic to use in the construction
of the confidence interval for the mean when ci.method="normal.approx"
(see the DETAILS section). The possible
values are pivot.statistic="z" (the default) and pivot.statistic="t".
When pivot.statistic="t" you may supply the argument
ci.sample size (see below). The argument pivot.statistic is
ignored if ci=FALSE.
numeric scalar indicating what sample size to assume to construct the
confidence interval for the mean if pivot.statistic="t" and
ci.method="normal.approx". The default value is the number of
uncensored observations.
a list of class "estimateCensored" containing the estimated parameters
and other information. See estimateCensored.object for details.
Cohen, A.C. (1991). Truncated and Censored Samples. Marcel Dekker, New York, New York, 312pp.
Cox, D.R. (1970). Analysis of Binary Data. Chapman & Hall, London. 142pp.
Efron, B. (1979). Bootstrap Methods: Another Look at the Jackknife. The Annals of Statistics 7, 1–26.
Efron, B., and R.J. Tibshirani. (1993). An Introduction to the Bootstrap. Chapman and Hall, New York, 436pp.
Forbes, C., M. Evans, N. Hastings, and B. Peacock. (2011). Statistical Distributions, Fourth Edition. John Wiley and Sons, Hoboken, NJ.
Helsel, D.R. (2012). Statistics for Censored Environmental Data Using Minitab and R, Second Edition. John Wiley & Sons, Hoboken, New Jersey.
Johnson, N. L., S. Kotz, and A. Kemp. (1992). Univariate Discrete Distributions, Second Edition. John Wiley and Sons, New York, Chapter 4.
Millard, S.P., P. Dixon, and N.K. Neerchal. (2014; in preparation). Environmental Statistics with R. CRC Press, Boca Raton, Florida.
Nelson, W. (1982). Applied Life Data Analysis. John Wiley and Sons, New York, 634pp.
Royston, P. (2007). Profile Likelihood for Estimation and Confdence Intervals. The Stata Journal 7(3), pp. 376–387.
Stryhn, H., and J. Christensen. (2003). Confidence Intervals by the Profile Likelihood Method, with Applications in Veterinary Epidemiology. Contributed paper at ISVEE X (November 2003, Chile).
Venzon, D.J., and S.H. Moolgavkar. (1988). A Method for Computing Profile-Likelihood-Based Confidence Intervals. Journal of the Royal Statistical Society, Series C (Applied Statistics) 37(1), pp. 87–94.
A sample of data contains censored observations if some of the observations are reported only as being below or above some censoring level. In environmental data analysis, Type I left-censored data sets are common, with values being reported as “less than the detection limit” (e.g., Helsel, 2012). Data sets with only one censoring level are called singly censored; data sets with multiple censoring levels are called multiply or progressively censored.
Statistical methods for dealing with censored data sets have a long history in the field of survival analysis and life testing. More recently, researchers in the environmental field have proposed alternative methods of computing estimates and confidence intervals in addition to the classical ones such as maximum likelihood estimation. Helsel (2012, Chapter 6) gives an excellent review of past studies of the properties of various estimators for parameters of a normal or lognormal distribution based on censored environmental data.
In practice, it is better to use a confidence interval for the mean or a joint confidence region for the mean and standard deviation (or coefficient of variation), rather than rely on a single point-estimate of the mean. Few studies have been done to evaluate the performance of methods for constructing confidence intervals for the mean or joint confidence regions for the mean and coefficient of variation of a Poisson distribution when data are subjected to single or multiple censoring.
# Generate 20 observations from a Poisson distribution with
# parameter lambda=10, and censor the values less than 10.
# Then generate 20 more observations from the same distribution
# and censor the values less than 20. Then estimate the mean
# using the maximum likelihood method.
# (Note: the call to set.seed simply allows you to reproduce this example.)
set.seed(300)
dat.1 <- rpois(20, lambda=10)
censored.1 <- dat.1 < 10
dat.1[censored.1] <- 10
dat.2 <- rpois(20, lambda=10)
censored.2 <- dat.2 < 20
dat.2[censored.2] <- 20
dat <- c(dat.1, dat.2)
censored <- c(censored.1, censored.2)
epoisCensored(dat, censored, ci = TRUE)
#>
#> Results of Distribution Parameter Estimation
#> Based on Type I Censored Data
#> --------------------------------------------
#>
#> Assumed Distribution: Poisson
#>
#> Censoring Side: left
#>
#> Censoring Level(s): 10 20
#>
#> Estimated Parameter(s): lambda = 11.05402
#>
#> Estimation Method: MLE
#>
#> Data: dat
#>
#> Censoring Variable: censored
#>
#> Sample Size: 40
#>
#> Percent Censored: 65%
#>
#> Confidence Interval for: lambda
#>
#> Confidence Interval Method: Profile Likelihood
#>
#> Confidence Interval Type: two-sided
#>
#> Confidence Level: 95%
#>
#> Confidence Interval: LCL = 9.842894
#> UCL = 12.846484
#>
#Results of Distribution Parameter Estimation
#Based on Type I Censored Data
#--------------------------------------------
#
#Assumed Distribution: Poisson
#
#Censoring Side: left
#
#Censoring Level(s): 10 20
#
#Estimated Parameter(s): lambda = 11.05402
#
#Estimation Method: MLE
#
#Data: dat
#
#Censoring Variable: censored
#
#Sample Size: 40
#
#Percent Censored: 65%
#
#Confidence Interval for: lambda
#
#Confidence Interval Method: Profile Likelihood
#
#Confidence Interval Type: two-sided
#
#Confidence Level: 95%
#
#Confidence Interval: LCL = 9.842894
# UCL = 12.846484
#----------
# Clean up
#---------
rm(dat.1, censored.1, dat.2, censored.2, dat, censored)