Basic Adjustment Statistics
This document discusses essential statistical concepts which apply when adjusting and analyzing surveying networks of any size. The example of a simple GeoLab height adjustment is compared to the standard calculation of the average value and standard deviation of a set of repeated measurements.
We first briefly review the calculation of the average value and standard deviation. We then look at that calculation in the context of a GeoLabPX5 least squares adjustment of essentially the same measurements, and discuss statistical concepts and procedures along the way. 
Simple Average and Standard Deviation
We will work with a vector, m, of repeated measurements m[i], i = 1 to n, where n is the number of measurements, which must be greater than 1 in this case, for an adjustment to be possible. In our example, n = 5.
Note that our discussion of statistics applies to even the most complicated networks of measurements. We use a simple example measurement set to ensure that the important statistical concepts we cover are not clouded by large and complicated sets of data. It is hoped that this document will help us keep these concepts in mind, even when adjusting and analyzing large data sets. 
The "average" value, a, is given by the sum of all the measurements m[i], i = 1 to n, divided by the number of measurements, n, as shown in the diagram. The formula for the "standard deviation" is also shown in the diagram.
Least Squares Adjustment Equations
Let's have a look at the least squares adjustment equations on which the above calculations are based. In our discussion, we use the following notation: The transpose of matrix A is written as A(T). The inverse of a matrix P is written as P(1). The covariance matrix of a vector m is written as Cm, and the corresponding weight matrix is written as Pm (where Pm = Cm(1)). Multiplication is indicated by an asterisk (*) for both regular numbers and matrices, with context indicating which is required. The elements of a vector v are written as v[i], i = 1 to n, where n is the number of elements in the vector.
In our example, the observation equation for each measurement m[i] is: a  m[i] = r[i], i = 1 to n. Or, in matrix form (where A is the design matrix), A * a  m = r (where A(T) = [1 1 1 1 1]), the vector of measurements m = [1.506, 1.504, 1.505, 1.503, 1.505], and r is the corresponding vector of residuals, r[i], i = 1 to n, which we'll discuss below.
The standard least squares adjustment equation for the estimated (adjusted) "average" value is: a = [A(T) * A](1) * A(T) * m. With A(T) = [1 1 1 1 1], in our example, A(T) * A results in a single number with the value n, the number of measurements. The inverse of A(T) * A is therefore 1 / n. The other portion of this equation, namely A(T) * m, is simply the sum of all the measurements, m[1] + m[2] + m[3] + m[4] + m[5].
So, we arrive at the equation for the least squares estimate of the average value as follows: a = (m[1] + m[2] + m[3] + m[4] + m[5]) / 5, which, of course, is identical to the standard formula for calculating the average value. Using our sample measurements, our average value is therefore (1.506 + 1.504 + 1.505 + 1.503 + 1.505) / 5, or 7.523 / 5, resulting in 1.5046.
In our example, the observation equation for each measurement m[i] is: a  m[i] = r[i], i = 1 to n. Or, in matrix form (where A is the design matrix), A * a  m = r (where A(T) = [1 1 1 1 1]), the vector of measurements m = [1.506, 1.504, 1.505, 1.503, 1.505], and r is the corresponding vector of residuals, r[i], i = 1 to n, which we'll discuss below.
The standard least squares adjustment equation for the estimated (adjusted) "average" value is: a = [A(T) * A](1) * A(T) * m. With A(T) = [1 1 1 1 1], in our example, A(T) * A results in a single number with the value n, the number of measurements. The inverse of A(T) * A is therefore 1 / n. The other portion of this equation, namely A(T) * m, is simply the sum of all the measurements, m[1] + m[2] + m[3] + m[4] + m[5].
So, we arrive at the equation for the least squares estimate of the average value as follows: a = (m[1] + m[2] + m[3] + m[4] + m[5]) / 5, which, of course, is identical to the standard formula for calculating the average value. Using our sample measurements, our average value is therefore (1.506 + 1.504 + 1.505 + 1.503 + 1.505) / 5, or 7.523 / 5, resulting in 1.5046.
The Misclosures
The adjustment equations we are using in this document are for linear least squares adjustments, because the example we are using is linear. Our general adjustments of real networks require nonlinear observation equations, which are linearized relative to the current values of the adjustment parameters. That's why we need initial coordinates (and initial auxiliary parameter values) to start the adjustment process. The linearized versions of the observation equations are used in an iterative process in which the latest estimated values of the parameters are used to calculate the corrections to the values of the parameters. This process is continued until the corrections are all less than a specified convergence criterion.
The misclosures are calculated with the same matrix equation (given below) used for calculating the residuals. The only difference is that the misclosures are calculated using the initial (or approximate) values of the parameters, instead of the final adjusted values. Large misclosures can therefore give us the first indication where a problem may lie, whether it's with the measurements or with the current values of the parameters.
To help discuss statistical concepts of least squares adjustment, in the context of adjusting a network with GeoLab, we have created the following files:
The misclosures are calculated with the same matrix equation (given below) used for calculating the residuals. The only difference is that the misclosures are calculated using the initial (or approximate) values of the parameters, instead of the final adjusted values. Large misclosures can therefore give us the first indication where a problem may lie, whether it's with the measurements or with the current values of the parameters.
To help discuss statistical concepts of least squares adjustment, in the context of adjusting a network with GeoLab, we have created the following files:


Please download these two simple text files. If you have GeoLab installed and licensed, you can create a new project using the downloaded IOB file as your main IOB file. Otherwise, just open them for viewing,
When you look through the LST file, you'll see the misclosures, which are initially large relative to the measurement standard deviations. This is due to the inaccurate initial value (1.500) of the station's height given in the station's NEO record. When the initial parameter values are more accurate, any large misclosures are normally due to problems with the measurements (e.g. blunders or incorrect station identification, etc).
When you look through the LST file, you'll see the misclosures, which are initially large relative to the measurement standard deviations. This is due to the inaccurate initial value (1.500) of the station's height given in the station's NEO record. When the initial parameter values are more accurate, any large misclosures are normally due to problems with the measurements (e.g. blunders or incorrect station identification, etc).
The Residuals
The above equation, A * a  m = r is used to calculate the residuals. For each measurement m[i], i = 1 to n in our example, the corresponding residual is given by r[i] = a  m[i], the average value minus the measured value, i = 1 to n.
Therefore, with a = 1.5046, the residuals are r = [0.0014, 0.0006, 0.0004, 0.0016, 0.0004]. You can check that these are the values of the residuals in the GeoLab output listing (LST file).
We can see that the residuals represent the central idea in the statistics of a least squares adjustment, which is to minimize the sum of squares of the weighted residuals. It's good to keep in mind that the residuals are corrections added to the measurements to make them fit the mathematical models perfectly  this is done so that the weighted sum of squares of the residuals is a minimum.
In other words, we minimize the weighted sum of squares of the differences between values calculated from the adjusted values and the measured values. This is a fairly simple idea, from which other statistical concepts are derived in a least squares adjustment. Simple, but powerful.
Note that we calculate the standardized residuals by dividing the residual by its standard deviation. The resulting standardized residuals have a probability density function with a mean of 0.0 and a standard deviation of 1.0, which allows us, for example, to use the same critical value for all measurements as our test for outliers.
Have another look at the residuals section in the LST file we provide above. Please especially note the following in the header of the residuals list pages: "Residuals (critical value = 1.948):". What is the "critical value"? Based on the probability density function of the residuals, when the absolute value of a standardized residual is greater than the critical value, the corresponding measurement is flagged for rejection. The suggestion is for us to evaluate the actual quality of the measurement. It is not telling us to remove the measurement from the adjustment without due consideration. One should always look at the size of the residual itself when deciding whether a measurement should be considered suspect  if the residual is small, then we can be less concerned about the standardized residual being slightly greater than the critical value.
In adjustments more complicated than our simple example, the measurements are normally assigned an estimated covariance matrix, with the diagonal elements of this covariance matrix being the estimated variances of the measurements. For leveling and total station measurements, our estimated covariance matrix of the measurements is normally a diagonal matrix (with all offdiagonal elements equal to zero), which means that those measurements are not correlated. However, GNSS measurements of sets coordinate differences, e.g. GNSS coordinate difference measurement vectors, normally have a full covariance matrix. Note that all of the concepts are the same whether our measurements are correlated or not. The same adjustment equations are used, and the same statistical analysis is made.
In our simple example adjustment, the initial covariance matrix of the measurements is the identity matrix (a diagonal matrix with each diagonal element having the value 1). Of course when a matrix A is multiplied by an identity matrix, the result is simply the matrix A, so using an identity matrix as the initial measurement weight matrix can be seen as a convenience.
If all the measurements in our network have a diagonal covariance matrix, i.e. they are all uncorrelated,, then it is valid to use the identity matrix as the initial weight matrix. Otherwise it is not, because the structure of correlation among the measurements can have a large effect on the values of the adjusted parameters. Please see this blog article for more on respecting measurements.
In the general adjustment case, when we have an initial, estimated covariance matrix C(m) for our measurements, the more general adjustment equation for the adjusted parameters is x = [A(T) * P * A](1) * A(T) * P * m, where P is the inverse of the covariance matrix of the measurements, i.e. P = Cm(1).
So a residual is simply the difference between the measured value and the value of that observation calculated from the estimated values of the parameters in the adjustment. We can also calculate the covariance matrix of the estimated residuals using the standard least squares adjustment equation for that purpose. GeoLab rigorously calculates the covariance matrix of the residuals, so you can use the GeoLab output listing to look at the statistics of the residuals.
Therefore, with a = 1.5046, the residuals are r = [0.0014, 0.0006, 0.0004, 0.0016, 0.0004]. You can check that these are the values of the residuals in the GeoLab output listing (LST file).
We can see that the residuals represent the central idea in the statistics of a least squares adjustment, which is to minimize the sum of squares of the weighted residuals. It's good to keep in mind that the residuals are corrections added to the measurements to make them fit the mathematical models perfectly  this is done so that the weighted sum of squares of the residuals is a minimum.
In other words, we minimize the weighted sum of squares of the differences between values calculated from the adjusted values and the measured values. This is a fairly simple idea, from which other statistical concepts are derived in a least squares adjustment. Simple, but powerful.
Note that we calculate the standardized residuals by dividing the residual by its standard deviation. The resulting standardized residuals have a probability density function with a mean of 0.0 and a standard deviation of 1.0, which allows us, for example, to use the same critical value for all measurements as our test for outliers.
Have another look at the residuals section in the LST file we provide above. Please especially note the following in the header of the residuals list pages: "Residuals (critical value = 1.948):". What is the "critical value"? Based on the probability density function of the residuals, when the absolute value of a standardized residual is greater than the critical value, the corresponding measurement is flagged for rejection. The suggestion is for us to evaluate the actual quality of the measurement. It is not telling us to remove the measurement from the adjustment without due consideration. One should always look at the size of the residual itself when deciding whether a measurement should be considered suspect  if the residual is small, then we can be less concerned about the standardized residual being slightly greater than the critical value.
In adjustments more complicated than our simple example, the measurements are normally assigned an estimated covariance matrix, with the diagonal elements of this covariance matrix being the estimated variances of the measurements. For leveling and total station measurements, our estimated covariance matrix of the measurements is normally a diagonal matrix (with all offdiagonal elements equal to zero), which means that those measurements are not correlated. However, GNSS measurements of sets coordinate differences, e.g. GNSS coordinate difference measurement vectors, normally have a full covariance matrix. Note that all of the concepts are the same whether our measurements are correlated or not. The same adjustment equations are used, and the same statistical analysis is made.
In our simple example adjustment, the initial covariance matrix of the measurements is the identity matrix (a diagonal matrix with each diagonal element having the value 1). Of course when a matrix A is multiplied by an identity matrix, the result is simply the matrix A, so using an identity matrix as the initial measurement weight matrix can be seen as a convenience.
If all the measurements in our network have a diagonal covariance matrix, i.e. they are all uncorrelated,, then it is valid to use the identity matrix as the initial weight matrix. Otherwise it is not, because the structure of correlation among the measurements can have a large effect on the values of the adjusted parameters. Please see this blog article for more on respecting measurements.
In the general adjustment case, when we have an initial, estimated covariance matrix C(m) for our measurements, the more general adjustment equation for the adjusted parameters is x = [A(T) * P * A](1) * A(T) * P * m, where P is the inverse of the covariance matrix of the measurements, i.e. P = Cm(1).
So a residual is simply the difference between the measured value and the value of that observation calculated from the estimated values of the parameters in the adjustment. We can also calculate the covariance matrix of the estimated residuals using the standard least squares adjustment equation for that purpose. GeoLab rigorously calculates the covariance matrix of the residuals, so you can use the GeoLab output listing to look at the statistics of the residuals.
The Estimated Variance Factor
Now let's consider the important estimated (a posteriori) variance factor. From least squares adjustment theory, we know that the calculation of the estimated variance factor for a set of measurements is given by: (r(T) * P * r) / (n  u), where n is the number of measurements, and u is the number of parameters in the adjustment. For use below, we let s0 be the square root of the variance factor, so that the variance factor = s0 * s0.
For our example, P = I (the identity matrix), so this reduces to (r(T) * r) / (n  1), or the sum of r[i] * r[i] / (n  1), i = 1 to n. We can see that this is the same as the standard calculation of the variance for a set of repeated measurements.
Although the minimum number of measurements is 2 for our adjustment, the actual number needed, to result in the required accuracy of the estimated average value, can be determined by performing a simulation (or preanalysis). For now we can understand that the accuracy of the estimated values of the parameters depends on the number of measurements made as well as the strength of the observation equations.
When we set out to do a least squares adjustment, our main objective is to determine the values of the parameters by making measurements that will result in the required accuracy of those parameters. However, it is important to realize that, when we perform a least squares adjustment, we are also estimating the covariance matrix of the measurements that was actually attained in the particular network we're working with, on which the covariance matrices of the estimated parameters, the residuals, and other derived quantities such as confidence regions, are based.
It is clear then, that the estimation of the covariance matrix of the measurements, and the strength of the design matrix, provide the foundation of all statistics in the adjustment. It's true that our experience in measuring networks with the same equipment and procedures gives us a good idea of the measurement accuracy, but every time we measure a network the conditions are different and so is the actual measurement accuracy.
For our example, P = I (the identity matrix), so this reduces to (r(T) * r) / (n  1), or the sum of r[i] * r[i] / (n  1), i = 1 to n. We can see that this is the same as the standard calculation of the variance for a set of repeated measurements.
Although the minimum number of measurements is 2 for our adjustment, the actual number needed, to result in the required accuracy of the estimated average value, can be determined by performing a simulation (or preanalysis). For now we can understand that the accuracy of the estimated values of the parameters depends on the number of measurements made as well as the strength of the observation equations.
When we set out to do a least squares adjustment, our main objective is to determine the values of the parameters by making measurements that will result in the required accuracy of those parameters. However, it is important to realize that, when we perform a least squares adjustment, we are also estimating the covariance matrix of the measurements that was actually attained in the particular network we're working with, on which the covariance matrices of the estimated parameters, the residuals, and other derived quantities such as confidence regions, are based.
It is clear then, that the estimation of the covariance matrix of the measurements, and the strength of the design matrix, provide the foundation of all statistics in the adjustment. It's true that our experience in measuring networks with the same equipment and procedures gives us a good idea of the measurement accuracy, but every time we measure a network the conditions are different and so is the actual measurement accuracy.
Statistical Analysis Notes
The above discussion indicates that perhaps our most important task in producing quality adjustments, is the analysis of the residuals and their covariance matrix.
The analysis of residual outliers (i.e. standardized residuals that are larger in absolute value than the critical value) must be carried out before we can rely on subsequent statistical tests, such as the chi square test on the variance factor, and assessment, such as the confidence regions of the parameters and parameter differences.
You can view the GeoLab output listing file provided to view the statistics mentioned.
The analysis of residual outliers (i.e. standardized residuals that are larger in absolute value than the critical value) must be carried out before we can rely on subsequent statistical tests, such as the chi square test on the variance factor, and assessment, such as the confidence regions of the parameters and parameter differences.
You can view the GeoLab output listing file provided to view the statistics mentioned.