What is Confidence interval ?
“Confidence intervals tells about the certainty of estimations by providing bounds around it.”
Confidence interval provides bounds over the statistical parameters of population like mean, std-dev etc. This also means that C.I should not be interpreted as prediction intervals which are used in case of single observation.
In the field of machine learning confidence intervals are mostly used to provide a concrete estimation of skill of a model.
Before reading further it is recommended to know CLT, if you don’t know what CLT is click here.
Let’s understand it by giving a example, Consider a scenario, Given a list of weights of all people kind of population and we calculate the mean by simple averaging and then we calculate sample mean .
we know that ≈ given a large sample size and large no of samples, but we still cannot write , so how good our estimation of is depends upon the range in which it can vary and if we can provide a boundary to the range with a probabilistic score that value of lies in this range is core idea of confidence interval.
This also implies smaller the confidence interval with high probabilistic score, defines the precision of our estimate.
Calculating Confidence interval for a Standard Gaussian Random Variable
There are couple of reason why i choose a Gaussian random variable,
- we know how the range is distributed in case of gaussian random variable.
- Where the mean of a gaussian distribution lies.
if the random variable follows then can we say that the confidence interval is[μ-2σ,μ+2σ] with 95% confidence as we already know the range division of values in case of gaussian distribution.
This technique can easily be extended for a distribution with finite mean and variance, as then it will follow central limit theorem.
so in case of distribution with finite mean and variance we can easily compute the 95% confidence interval for mean by applying CLT,[μ- (2σ²/n),μ+ (2σ²/n)] with 95% confidence. This is only valid when we have std-dev given.
Estimating confidence interval without std-dev provided is done by using t-dist, covering t-dist is beyond the scope of this article but if readers are interested then we will be very happy in explaining t-dist. The idea of t-dist was itself created to compute C.I only.