vignettes/prediction-intervals.Rmd
prediction-intervals.RmdThis vignette describes normal, nonparametric, and Poisson prediction intervals for future environmental observations. Derivations of prediction factors and simultaneous coverage are followed by the associated power calculations for normal and nonparametric monitoring procedures.
Related vignettes:
Help topic: predIntNormK.
A prediction interval for some population is an interval on the real line constructed so that it will contain future observations or averages from that population with some specified probability , where and is some pre-specified positive integer. The quantity is called the confidence coefficient or confidence level associated with the prediction interval.
Let
denote a vector of
observations from a normal
distribution with parameters
mean=
and
sd=.
Also, let
denote the sample size associated with the
future averages (i.e.,
n.mean=).
When
,
each average is really just a single observation, so in the rest of this
section the term “averages” will replace the phrase “observations or
averages”.
For a normal distribution, the form of a two-sided prediction interval is:
where denotes the sample mean:
denotes the sample standard deviation:
and
denotes a constant that depends on the sample size
,
the confidence level, the number of future averages
,
and the sample size associated with the future averages,
.
Do not confuse the constant
(uppercase K) with the number of future averages
(lowercase k). The symbol
is used here to be consistent with the notation used for tolerance
intervals (see tolIntNorm).
Similarly, the form of a one-sided lower prediction interval is:
and the form of a one-sided upper prediction interval is:
but
differs for one-sided versus two-sided prediction intervals. The
derivation of the constant
is explained below. The function predIntNormK computes the
value of
and is called by predIntNorm.
The Derivation of K for One Future Observation or Average (k
= 1)
Let
denote a random variable from a normal
distribution with parameters
mean=
and
sd=,
and let
denote the
’th
quantile of
.
A true two-sided prediction interval for the next observation of is given by:
where denotes the ’th quantile of a standard normal distribution.
More generally, a true two-sided prediction interval for the next average based on a sample of size is given by:
Because the values of and are unknown, they must be estimated, and a prediction interval then constructed based on the estimated values of and .
For a two-sided prediction interval
(pi.type="two-sided"), the constant
for a
prediction interval for the next
average based on a sample size of
is computed as:
where
denotes the
’th
quantile of the Student’s
t-distribution with
degrees of freedom. For a one-sided prediction interval
(pi.type="lower" or pi.type="lower"), the
prediction interval is given by:
.
The formulas for these prediction intervals are derived as follows. Let denote the future average based on observations. Then the quantity has a normal distribution with expectation and variance given by:
so the quantity
has a Student’s t-distribution with degrees of freedom.
The Derivation of K for More than One Future Observation or
Average (k >1)
When
,
the function predIntNormK allows for two ways to compute
:
an exact method due to Dunnett (1955) (method="exact"), and
an approximate (conservative) method based on the Bonferroni inequality
(method="Bonferroni"; see Miller, 1981a, pp.8, 67-70;
Gibbons et al., 2009, p.4). Each of these methods is explained
below.
Exact Method Due to Dunnett (1955)
(method="exact")
Dunnett (1955) derived the value of
in the context of the multiple comparisons problem of comparing several
treatment means to one control mean. The value of
is computed as:
where is a constant that depends on the sample size , the number of future observations (averages) , the sample size associated with the future averages , and the confidence level .
When pi.type="lower" or pi.type="upper",
the value of
is the number that satisfies the following equation (Gupta and Sobel,
1957; Hahn, 1970a):
where
and and denote the cumulative distribution function and probability density function, respectively, of the standard normal distribution. Note that the function is the probability density function of a chi random variable with degrees of freedom.
When pi.type="two-sided", the value of
is the number that satisfies the following equation:
where
Approximate Method Based on the Bonferroni Inequality
(method="Bonferroni")
As shown above, when
,
the value of
is given by Equation (8) or Equation (9) for two-sided or one-sided
prediction intervals, respectively. When
,
a conservative way to construct a
prediction interval for the next
observations or averages is to use a Bonferroni correction (Miller,
1981a, p.8) and set
in Equation (8) or (9) (Chew, 1968). This value of
will be conservative in that the computed prediction intervals will be
wider than the exact predictions intervals. Hahn (1969, 1970a) compared
the exact values of
with those based on the Bonferroni inequality for the case of
and found the approximation to be quite satisfactory except when
is small,
is large, and
is large. For example, Gibbons (1987a) notes that for a 99% prediction
interval (i.e.,
)
for the next
observations, if
,
the bias of
is never greater than 1% no matter what the value of
.
Help topic: predIntNormSimultaneousK.
What is a Simultaneous Prediction Interval?
A prediction interval for some population is an interval on the real
line constructed so that it will contain
future observations from that population with some specified probability
,
where
and
is some pre-specified positive integer. The quantity
is called the confidence coefficient or confidence level associated with
the prediction interval. The function predIntNorm
computes a standard prediction interval based on a sample from a normal
distribution.
The function predIntNormSimultaneous computes a
simultaneous prediction interval that will contain a certain number of
future observations with probability
for each of
future sampling “occasions”, where
is some pre-specified positive integer. The quantity
may refer to
distinct future sampling occasions in time, or it may for example refer
to sampling at
distinct locations on one future sampling occasion, assuming that the
population standard deviation is the same at all of the
distinct locations.
The function predIntNormSimultaneous
computes a simultaneous prediction interval based on one of three
possible rules:
rule="k.of.m"), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If obserations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: When
and
,
the results of predIntNormSimultaneous are equivalent to
the results of predIntNorm.rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.rule="Modified.CA"),
with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.Simultaneous prediction intervals can be extended to using averages
(means) in place of single observations (USEPA, 2009, Chapter 19). That
is, you can create a simultaneous prediction interval that will contain
a specified number of averages (based on which rule you choose) on each
of
future sampling occassions, where each each average is based on
individual observations. For the functions predIntNormSimultaneous
and predIntNormSimultaneousK, the argument
n.mean corresponds to
.
The Form of a Prediction Interval for 1 Future
Observation
Let
denote a vector of
observations from a normal
distribution with parameters
mean=
and
sd=.
Also, let
denote the sample size associated with the future averages (i.e.,
n.mean=).
When
,
each average is really just a single observation, so in the rest of this
section the term “averages” will sometimes replace the phrase
“observations or averages”.
For a normal distribution, the form of a two-sided simultaneous prediction interval is:
where denotes the sample mean:
denotes the sample standard deviation:
and
denotes a constant that depends on the sample size
,
the confidence level, the number of future sampling occassions
,
and the sample size associated with the future averages,
.
Do not confuse the constant
(uppercase K) with the number of future averages
(lowercase k) in the
-of-
rule. The symbol
is used here to be consistent with the notation used for tolerance
intervals (see tolIntNorm).
Similarly, the form of a one-sided lower prediction interval is:
and the form of a one-sided upper prediction interval is:
The derivation of the constant
for 1 future observation is explained in the section on predIntNormK.
The Form of a Simultaneous Prediction Interval
For simultaneous prediction intervals, only lower
(pi.type="lower") and upper (pi.type="upper")
prediction intervals are available. Two-sided simultaneous prediction
intervals were available in Versions 2.4.0 - 2.8.1 of EnvStats,
but these prediction intervals were based on an incorrect algorithm for
.
Equations (4) and (5) above hold for simultaneous prediction intervals, but the derivation of the constant is more difficult, and is explained below.
The Derivation of K for Future Observations
First we will show the derivation based on future observations (i.e.,
,
n.mean=1), and then extend the formulas to future
averages.
The Derivation of K for the k-of-m Rule
(rule="k.of.m")
For the
-of-
rule (rule="k.of.m") with
(i.e., n.mean=1), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If observations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: When
and
,
this kind of simultaneous prediction interval becomes the same as a
standard prediction interval for the next
observations (see predIntNorm).
For the case when
future sampling occasion, both Hall and Prairie (1973) and Fertig and
Mann (1977) discuss the derivation of
.
Davis and McNichols (1987) extend the derivation to the case where
is a positive integer. They show that for a one-sided upper prediction
interval (pi.type="upper"), the probability
that at least
of the next
future observations will be contained in the interval given in Equation
(5) above, for each of
future sampling occasions, is given by:
where
denotes the cdf of the non-central
Student’s t-distribution with parameters
df=
and
ncp=
evaluated at
;
denotes the cdf of the standard normal
distribution evaluated at
;
denotes the cdf of the beta
distribution with parameters
shape1=
and
shape2=;
and
denotes the value of the beta
function with parameters
a=
and
b=.
The quantity
(upper case delta) denotes the difference between the mean of the
population that was sampled to construct the prediction interval, and
the mean of the population that will be sampled to produce the future
observations. The quantity
(sigma) denotes the population standard deviation of both of these
populations. Usually you assume
unless you are interested in computing the power of the rule to detect a
change in means between the populations (see predIntNormSimultaneousTestPower).
For given values of the confidence level
(),
sample size
(),
minimum number of future observations to be contained in the interval
per sampling occasion
(),
number of future observations per sampling occasion
(),
and number of future sampling occasions
(),
Equation (6) can be solved for
.
The function predIntNormSimultaneousK uses the R function
nlminb to solve Equation (6) for
.
When pi.type="lower", the same value of
is used as when pi.type="upper", but Equation (4) is used
to construct the prediction interval.
The Derivation of K for the California Rule
(rule="CA")
For the California rule (rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.
The formula for is the same as for the -of- rule, except that Equation (6) becomes the following (Davis, 1998b):
The Derivation of K for the Modified California Rule
(rule="Modified.CA")
For the Modified California rule (rule="Modified.CA"), with
probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.
The formula for is the same as for the -of- rule, except that Equation (6) becomes the following (Davis, 1998b):
The Derivation of K for Future Means
For each of the above rules, if we are interested in using averages
instead of single observations, with
(i.e.,
n.mean),
the first term in the integral in Equations (6)-(8) that involves the
cdf of the non-central
Student’s t-distribution becomes:
Help topic: predIntNormSimultaneousTestPower.
What is a Simultaneous Prediction Interval?
A prediction interval for some population is an interval on the real
line constructed so that it will contain
future observations from that population with some specified probability
,
where
and
is some pre-specified positive integer. The quantity
is called the confidence coefficient or confidence level associated with
the prediction interval. The function predIntNorm
computes a standard prediction interval based on a sample from a normal
distribution.
The function predIntNormSimultaneous
computes a simultaneous prediction interval that will contain a certain
number of future observations with probability
for each of
future sampling “occasions”, where
is some pre-specified positive integer. The quantity
may refer to
distinct future sampling occasions in time, or it may for example refer
to sampling at
distinct locations on one future sampling occasion, assuming that the
population standard deviation is the same at all of the
distinct locations.
The function predIntNormSimultaneous
computes a simultaneous prediction interval based on one of three
possible rules:
rule="k.of.m"), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If obserations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: When
and
,
the results of predIntNormSimultaneous
are equivalent to the results of predIntNorm.rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.rule="Modified.CA"),
with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.Simultaneous prediction intervals can be extended to using averages
(means) in place of single observations (USEPA, 2009, Chapter 19). That
is, you can create a simultaneous prediction interval that will contain
a specified number of averages (based on which rule you choose) on each
of
future sampling occassions, where each each average is based on
individual observations. For the function predIntNormSimultaneous,
the argument n.mean corresponds to
.
The Form of a Prediction Interval
Let
denote a vector of
observations from a normal
distribution with parameters
mean=
and
sd=.
Also, let
denote the sample size associated with the future averages (i.e.,
n.mean=).
When
,
each average is really just a single observation, so in the rest of this
section the term “averages” will replace the phrase “observations or
averages”.
For a normal distribution, the form of a two-sided prediction interval is:
where denotes the sample mean:
denotes the sample standard deviation:
and
denotes a constant that depends on the sample size
,
the confidence level, the number of future sampling occassions
,
and the sample size associated with the future averages,
.
Do not confuse the constant
(uppercase K) with the number of future averages
(lowercase k) in the
-of-
rule. The symbol
is used here to be consistent with the notation used for tolerance
intervals (see tolIntNorm).
Similarly, the form of a one-sided lower prediction interval is:
and the form of a one-sided upper prediction interval is:
Note: For simultaneous prediction intervals, only
lower (pi.type="lower") and upper
(pi.type="upper") prediction intervals are available.
The derivation of the constant
is explained in the section on predIntNormSimultaneousK.
Computing Power
The “power” of the prediction interval is defined as the probability
that at least one set of future observations violates the given rule
based on a simultaneous prediction interval for the next
future sampling occasions, where the population mean for the future
observations is allowed to differ from the population mean for the
observations used to construct the prediction interval.
The quantity
(upper case delta) denotes the difference between the mean of the
population that was sampled to construct the prediction interval, and
the mean of the population that will be sampled to produce the future
observations. The quantity
(sigma) denotes the population standard deviation of both of these
populations. The argument delta.over.sigma corresponds to
the quantity
.
Power Based on the k-of-m Rule
(rule="k.of.m")
For the
-of-
rule (rule="k.of.m") with
(i.e., n.mean=1), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If observations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: When
and
,
this kind of simultaneous prediction interval becomes the same as a
standard prediction interval for the next
observations (see predIntNorm).
Davis and McNichols (1987) show that for a one-sided upper prediction
interval (pi.type="upper"), the probability
that at least
of the next
future observations will be contained in the interval given in Equation
(5) above, for each of
future sampling occasions, is given by:
where
denotes the cdf of the non-central
Student’s t-distribution with parameters
df=
and
ncp=
evaluated at
;
denotes the cdf of the standard normal
distribution evaluated at
;
denotes the cdf of the beta
distribution with parameters
shape1=
and
shape2=;
and
denotes the value of the beta
function with parameters
a=
and
b=.
The quantity (upper case delta) denotes the difference between the mean of the population that was sampled to construct the prediction interval, and the mean of the population that will be sampled to produce the future observations. The quantity (sigma) denotes the population standard deviation of both of these populations. Usually you assume unless you are interested in computing the power of the rule to detect a change in means between the populations, as we are here.
If we are interested in using averages instead of single
observations, with
(i.e.,
n.mean),
the first term in the integral in Equation (6) that involves the cdf of
the non-central
Student’s t-distribution becomes:
For a given confidence level , the power of the rule to detect a change in means is simply given by:
where
is defined in Equation (6) above using the value of
that corresponds to
.
Thus, when the argument delta.over.sigma=0, the value of
is
and the power is simply
.
As delta.over.sigma increases above 0, the power
increases.
When pi.type="lower", the same value of K
is used as when pi.type="upper", but Equation (4) is used
to construct the prediction interval. Thus, the power increases as
delta.over.sigma decreases below 0.
Power Based on the California Rule
(rule="CA")
For the California rule (rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.
The derivation of the power is the same as for the -of- rule, except that Equation (6) becomes the following (Davis, 1998b):
Power Based on the Modified California Rule
(rule="Modified.CA")
For the Modified California rule (rule="Modified.CA"), with
probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.
The derivation of the power is the same as for the -of- rule, except that Equation (6) becomes the following (Davis, 1998b):
Help topic: predIntNparSimultaneous.
What is a Nonparametric Simultaneous Prediction
Interval?
A nonparametric prediction interval for some population is an interval
on the real line constructed so that it will contain at least
of
future observations from that population with some specified probability
,
where
and
and
are some pre-specified positive integers and
.
The quantity
is called the confidence coefficient or confidence level associated with
the prediction interval. The function predIntNpar
computes a standard nonparametric prediction interval.
The function predIntNparSimultaneous computes a
nonparametric simultaneous prediction interval that will contain a
certain number of future observations with probability
for each of
future sampling “occasions”, where
is some pre-specified positive integer. The quantity
may refer to
distinct future sampling occasions in time, or it may for example refer
to sampling at
distinct locations on one future sampling occasion, assuming that the
population standard deviation is the same at all of the
distinct locations.
The function predIntNparSimultaneous computes a
nonparametric simultaneous prediction interval based on one of three
possible rules:
rule="k.of.m"), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If obserations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: For this rule, when
,
the results of predIntNparSimultaneous are equivalent to
the results of predIntNpar.rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.rule="Modified.CA"),
with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.Nonparametric simultaneous prediction intervals can be extended to
using medians in place of single observations (USEPA, 2009, Chapter 19).
That is, you can create a nonparametric simultaneous prediction interval
that will contain a specified number of medians (based on which rule you
choose) on each of
future sampling occassions, where each each median is based on
individual observations. For the function
predIntNparSimultaneous, the argument n.median
corresponds to
.
The Form of a Nonparametric Prediction
Interval
Let
denote a vector of
independent observations from some continuous distribution, and let
denote the the
’th
order statistics in
.
A two-sided nonparametric prediction interval is constructed as:
where and are positive integers between 1 and , and . That is, denotes the rank of the lower prediction limit, and denotes the rank of the upper prediction limit. To make it easier to write some equations later on, we can also write the prediction interval (1) in a slightly different way as:
where
so that
is a positive integer between 1 and
,
and
.
In terms of the arguments to the function
predIntNparSimultaneous, the argument lpl.rank
corresponds to
,
and the argument n.plus.one.minus.upl.rank corresponds to
.
If we allow and and define lower and upper bounds as:
then Equation (2) above can also represent a one-sided lower or one-sided upper prediction interval as well. That is, a one-sided lower nonparametric prediction interval is constructed as:
and a one-sided upper nonparametric prediction interval is constructed as:
Usually, or and .
Note: For nonparametric simultaneous prediction
intervals, only lower (pi.type="lower") and upper
(pi.type="upper") prediction intervals are available.
Constructing Nonparametric Simultaneous Prediction Intervals
for Future Observations
First we will show how to construct a nonparametric simultaneous
prediction interval based on future observations (i.e.,
,
n.median=1), and then extend the formulas to future
medians.
Simultaneous Prediction Intervals for the k-of-m Rule
(rule="k.of.m")
For the
-of-
rule (rule="k.of.m") with
(i.e., n.median=1), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If observations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: When
,
this kind of simultaneous prediction interval becomes the same as a
standard nonparametric prediction interval (see predIntNpar).
Chou and Owen (1986) developed the theory for nonparametric simultaneous prediction limits for various rules, including the 1-of- rule. Their theory, however, does not cover the California or Modified California rules, and uses an -fold summation involving a minimum of terms. Davis and McNichols (1994b; 1999) extended the results of Chou and Owen (1986) to include the California and Modified California rule, and developed algorithms that involve summing far fewer terms.
Davis and McNichols (1999) give formulas for the probabilities associated with the one-sided upper simultaneous prediction interval shown in Equation (7). For the -of- rule, the probability that at least of the next future observations will be contained in the interval given in Equation (7) for each of future sampling occasions is given by:
where
denotes a random variable with a beta
distribution with parameters
and
,
and
denotes the pdf of this distribution. Note that
denotes the rank of the order statistic used as the upper prediction
limit (i.e.,
n.plus.one.minus.upl.rank=),
and that
is usually equal to
.
Also note that the summation term in Equation (8) corresponds to the
cumulative distribution function of a Negative
Binomial distribution with parameters
size=
and
prob=
evaluated at
q=.
When pi.type="lower",
denotes a random variable with a beta
distribution with parameters
and
.
Note that
denotes the rank of the order statistic used as the lower prediction
limit (i.e.,
lpl.rank=),
and that
is usually equal to
.
Simultaneous Prediction Intervals for the California Rule
(rule="CA")
For the California rule (rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.
In this case, the probability is given by:
Simultaneous Prediction Intervals for the Modified California
Rule (rule="Modified.CA")
For the Modified California rule (rule="Modified.CA"), with
probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.
In this case, the probability is given by:
where and .
Davis and McNichols (1999) provide algorithms for computing the
probabilities based on expanding polynomials and the formula for the
expected value of a beta random variable. In the discussion section of
Davis and McNichols (1999), however, Vangel points out that numerical
integration is adequate, and this is how these probabilities are
computed in the function predIntNparSimultaneous.
Constructing Nonparametric Simultaneous Prediction Intervals
for Future Medians
USEPA (2009, Chapter 19; Cameron, 2011) extends nonparametric
simultaneous prediction intervals to testing future medians for the case
of the 1-of-1 and 1-of-2 plans for medians of order 3. In general, each
of the rules
(-of-,
California, and Modified California) can be easily extended to the case
of using medians as long as the medians are based on an odd (as opposed
to even) sample size.
For each of the above rules, if we are interested in using medians
instead of single observations (i.e.,
;
n.median),
and we force
to be odd, then a median will be less than a prediction limit once
observations are less than the prediction limit. Thus, Equations (8) -
(10) are modified by replacing
with the term:
where
Help topic: predIntNparSimultaneousTestPower.
What is a Nonparametric Simultaneous Prediction
Interval?
A nonparametric prediction interval for some population is an interval
on the real line constructed so that it will contain at least
of
future observations from that population with some specified probability
,
where
and
and
are some pre-specified positive integers and
.
The quantity
is called the confidence coefficient or confidence level associated with
the prediction interval. The function predIntNpar
computes a standard nonparametric prediction interval.
The function predIntNparSimultaneous
computes a nonparametric simultaneous prediction interval that will
contain a certain number of future observations with probability
for each of
future sampling “occasions”, where
is some pre-specified positive integer. The quantity
may refer to
distinct future sampling occasions in time, or it may for example refer
to sampling at
distinct locations on one future sampling occasion, assuming that the
population standard deviation is the same at all of the
distinct locations.
The function predIntNparSimultaneous
computes a nonparametric simultaneous prediction interval based on one
of three possible rules:
rule="k.of.m"), at least
of the next
future observations will fall in the prediction interval with
probability
on each of the
future sampling occasions. If obserations are being taken sequentially,
for a particular sampling occasion, up to
observations may be taken, but once
of the observations fall within the prediction interval, sampling can
stop. Note: For this rule, when
,
the results of predIntNparSimultaneous
are equivalent to the results of predIntNpar.rule="CA"), with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else all of the next
observations will fall in the prediction interval. That is, if the first
observation falls in the prediction interval then sampling can stop.
Otherwise,
more observations must be taken.rule="Modified.CA"),
with probability
,
for each of the
future sampling occasions, either the first observation will fall in the
prediction interval, or else at least 2 out of the next 3 observations
will fall in the prediction interval. That is, if the first observation
falls in the prediction interval then sampling can stop. Otherwise, up
to 3 more observations must be taken.Nonparametric simultaneous prediction intervals can be extended to
using medians in place of single observations (USEPA, 2009, Chapter 19).
That is, you can create a nonparametric simultaneous prediction interval
that will contain a specified number of medians (based on which rule you
choose) on each of
future sampling occassions, where each each median is based on
individual observations. For the function predIntNparSimultaneous,
the argument n.median corresponds to
.
The Form of a Nonparametric Prediction
Interval
Let
denote a vector of
independent observations from some continuous distribution, and let
denote the the
’th
order statistics in
.
A two-sided nonparametric prediction interval is constructed as:
where and are positive integers between 1 and , and . That is, denotes the rank of the lower prediction limit, and denotes the rank of the upper prediction limit. To make it easier to write some equations later on, we can also write the prediction interval (1) in a slightly different way as:
where
so that
is a positive integer between 1 and
,
and
.
In terms of the arguments to the function
predIntNparSimultaneous, the argument lpl.rank
corresponds to
,
and the argument n.plus.one.minus.upl.rank corresponds to
.
If we allow and and define lower and upper bounds as:
then Equation (2) above can also represent a one-sided lower or one-sided upper prediction interval as well. That is, a one-sided lower nonparametric prediction interval is constructed as:
and a one-sided upper nonparametric prediction interval is constructed as:
Usually, or and .
Note: For nonparametric simultaneous prediction
intervals, only lower (pi.type="lower") and upper
(pi.type="upper") prediction intervals are available.
Computing Power
The “power” of the prediction interval is defined as the probability
that at least one set of future observations violates the given rule
based on a simultaneous prediction interval for the next
future sampling occasions, where the population for the future
observations is allowed to differ from the population for the
observations used to construct the prediction interval.
For the function predIntNparSimultaneousTestPower, power
is computed assuming both the background and future the observations
come from normal distributions with the same standard deviation, but the
means of the distributions are allowed to differ. The quantity
(upper case delta) denotes the difference between the mean of the
population that was sampled to construct the prediction interval, and
the mean of the population that will be sampled to produce the future
observations. The quantity
(sigma) denotes the population standard deviation of both of these
populations. The argument delta.over.sigma corresponds to
the quantity
.
Approximate Power (method="approx")
Based on Gansecki (2009), the power of a nonparametric simultaneous
prediction interval when the underlying observations come from a nomral
distribution can be approximated by the power of a normal simultaneous
prediction interval (see predIntNormSimultaneousTestPower)
where the multiplier
is replaced with the expected value of the normal order statistic that
corresponds to the rank of the order statistic used for the upper or
lower bound of the prediction interval. Gansecki (2009) uses the
approximation:
where
denotes the cumulative distribution function of the standard normal
distribution and
denotes the rank of the order statistic used as the prediction limit. By
default, the value of the argumentevNormOrdStats.method="royston", so the function
predIntNparSimultaneousTestPower uses the exact value of
the expected value of the normal order statistic in the call to evNormOrdStatsScalar.
You can change the method of computing the expected value of the normal
order statistic by changing the value of the argument
evNormOrdStats.method.
Power Based on Monte Carlo Simulation
(method="simulate")
When method="simulate", the power of the nonparametric
simultaneous prediction interval is estimated based on a Monte Carlo
simulation. The argument NMC determines the number of Monte
Carlo trials. If ci=TRUE, a confidence interval for the
power is created based on the NMC Monte Carlo estimates of
power.
Help topic: predIntPois.
A prediction interval for some population is an interval on the real line constructed so that it will contain future observations or averages from that population with some specified probability , where 0 < < 1 and is some pre-specified positive integer. The quantity is call the confidence coefficient or confidence level associated with the prediction interval.
In the case of a Poisson distribution, we have modified the usual meaning of a prediction interval and instead construct an interval that will contain future observations or future sums with a certain confidence level.
A prediction interval is a random interval; that is, the lower and/or upper bounds are random variables computed based on sample statistics in the baseline sample. Prior to taking one specific baseline sample, the probability that the prediction interval will contain the next averages is . Once a specific baseline sample is taken and the prediction interval based on that sample is computed, the probability that that prediction interval will contain the next averages is not necessarily , but it should be close.
If an experiment is repeated times, and for each experiment:
then the number of prediction intervals that actually contain the
future observation generated in step 2 above is a binomial random
variable with parameters
size=
and
prob=
(see Binomial).
If, on the other hand, only one baseline sample is taken and only one
prediction interval for
future observation is computed, then the number of future observations
out of a total of
future observations that will be contained in that one prediction
interval is a binomial random variable with parameters
size=
and
prob=,
where
depends on the true population parameters and the computed bounds of the
prediction interval.
Because of the discrete nature of the Poisson
distribution, even if the true mean of the distribution
were known exactly, the actual confidence level associated with a
prediction limit will usually not be exactly equal to
.
For example, for the Poisson distribution with parameter
lambda=2, the interval
contains 94.7% of this distribution and the interval
contains 98.3% of this distribution. Thus, no interval can contain
exactly 95% of this distribution, so it is impossible to construct an
exact 95% prediction interval for the next
observation for a Poisson distribution with parameter
lambda=2.
The Form of a Poisson Prediction Interval
Let
denote a vector of
observations from a Poisson
distribution with parameter
lambda=.
Also, let
denote the sum of these
random variables, i.e.,
Finally, let
denote the sample size associated with the
future sums (i.e.,
n.sum=).
When
,
each sum is really just a single observation, so in the rest of this
section the term “sums” replaces the phrase “observations or sums”.
Let
denote a vector of
future observations from a Poisson distribution with parameter
lambda=,
and set
equal to the sum of these
random variables, i.e.,
Then
has a Poisson distribution with parameter
lambda=
(Johnson et al., 1992, p.160). We are interested in constructing a
prediction limit for the next value of
,
or else the next
sums of
Poisson random variables, based on the observed value of
and assuming
.
For a Poisson distribution, the form of a two-sided prediction interval is:
where
and
is a constant that depends on the sample size
,
the confidence level
,
the number of future sums
,
and the sample size associated with the future sums
.
Do not confuse the constant
(uppercase
)
with the number of future sums
(lowercase
).
The symbol
is used here to be consistent with the notation used for prediction
intervals for the normal distribution (see predIntNorm).
Similarly, the form of a one-sided lower prediction interval is:
and the form of a one-sided upper prediction interval is:
The derivation of the constant is explained below.
Conditional Distribution
(method="conditional")
Nelson (1970) derives a prediction interval for the case
based on the conditional distribution of
given
.
He notes that the conditional distribution of
given the quantity
is binomial
with parameters
size=
and
prob=
(Johnson et al., 1992, p.161). When
,
the prediction limits are computed as those most extreme values of
that still yield a non-significant test of the hypothesis
,
which for the conditional distribution of
is equivalent to the hypothesis
:
prob=[m /(m + n)].
Using the relationship between the binomial
and F-distribution
(see the explanation of exact confidence intervals in the help file for
ebinom),
Nelson (1982, p. 203) states that exact two-sided
prediction limits
are the closest integer solutions to the following equations:
where denotes the ’th quantile of the F-distribution with and degrees of freedom.
If ci.type="lower",
is replaced with
in Equation (8) above for
,
and
is set to
.
If ci.type="upper",
is replaced with
in Equation (9) above for
,
and
is set to 0.
NOTE: This method is not extended to the case .
Conditional Distribution Approximation Based on Normal
Distribution
(method="conditional.approx.normal")
Cox and Hinkley (1974, p.245) derive an approximate prediction interval
for the case
.
Like Nelson (1970), they note that the conditional distribution of
given the quantity
is binomial
with parameters
size=
and
prob=,
and that the hypothesis
is equivalent to the hypothesis
:
prob=[m /(m + n)].
Cox and Hinkley (1974, p.245) suggest using the normal approximation to the binomial distribution (in this case, without the continuity correction; see Zar, 2010, pp.534-536 for information on the continuity correction associated with the normal approximation to the binomial distribution). Under the null hypothesis , the quantity
is approximately distributed as a standard normal random variable.
The Case When k = 1
When
and pi.type="two-sided", the prediction limits are computed
by solving the equation
where denotes the ’th quantile of the standard normal distribution. In this case, Gibbons (1987b) notes that the quantity in Equation (3) above is given by:
where .
When pi.type="lower" or pi.type="upper",
is computed exactly as above, except
is set to
.
The Case When k > 1
When
,
Gibbons (1987b) suggests using the Bonferroni inequality. That is, the
value of
is computed exactly as for the case
described above, except that the Bonferroni value of
is used in place of the usual value of
:
When pi.type="two-side",
.
When pi.type="lower" or pi.type="upper",
.
Conditional Distribution Approximation Based on Student’s
t-Distribution
(method="conditional.approx.t")
When method="conditional.approx.t", the exact same
procedure is used as whenmethod="conditional.approx.normal", except that the
quantity in Equation (10) is assumed to follow a Student’s
t-distribution with
degrees of freedom. Thus, all occurrences of
are replaced with
,
where
denotes the
’th
quantile of Student’s
t-distribution with
degrees of freedom.
Normal Approximation
(method="normal.approx")
The normal approximation for Poisson prediction limits was given by
Nelson (1970; 1982, p.203) and is based on the fact that the mean and
variance of a Poisson distribution are the same (Johnson et al, 1992,
p.157), and for “large” values of
and
,
both
and
are approximately normally distributed.
The Case When k = 1
The quantity
is approximately normally distributed with expectation and variance
given by:
so the quantity
is approximately distributed as a standard normal random variable.
The function predIntPois, however, assumes this quantity is
distributed as approximately a Student’s
t-distribution with
degrees of freedom.
Thus, following the idea of prediction intervals for a normal
distribution (see predIntNorm),
when pi.type="two-sided", the constant
for a
prediction interval for the next
sum of
observations is computed as:
where denotes the ’th quantile of a Student’s t-distribution with degrees of freedom.
Similarly, when pi.type="lower" or
pi.type="upper", the constant
is computed as:
The Case When k > 1
When
,
the value of
is computed exactly as for the case
described above, except that the Bonferroni value of
is used in place of the usual value of
:
When pi.type="two-sided",
When pi.type="lower" or
pi.type="upper",
Hahn and Nelson (1973, p.182) discuss another method of computing when , but this method is not implemented here.