Shot group statistics

 Jeroen Hogema  

E-mail:

February 2005

Minor update April 2019

 

To articles overview

file..//ballist.htm

 

 

CONTENTS

 

1        Introduction   1

2        One-shot probability distributions  2

3        Shot group SD and shot group diameter  6

4        Estimating the shot group SD   9

4.1     From shot group diameters 9

4.2     From individual shot locations 10

4.3     Comparing both estimation methods 13

4.4     How many groups of how many shots?  15

5        Applications  18

5.1     Scores in competition shooting  18

5.2     Matching Ammunition to the Rifle  20

5.2.1  Integer score  22

5.2.2  Decimal scoring  24

5.3     Using shot group diameters in the pellet selection process 25

 


 

 

 

 

 

 

1       Introduction

The context of what is written in this document is competition shooting: the fine art of using a rifle or pistol to shoot holes in a paper target. On the target, circular rings are printed to assign a score to each hole that is shot. To win, you need to have the highest total score. To get a high total score, you need to minimise the dispersion of the group of holes with respect to the target centre.

 

Under certain assumptions, the shot groups on the paper target can be characterised statistically by a single parameter. This parameter, named the shot group standard deviation in this document, is a measure for the shot group dispersion.

 

Some statistics concerning the location of a single shot are discussed in Chapter 2. Next, Chapter 3 discusses the relationship between this shot group standard deviation and the statistical distribution of the shot group diameter (for a given number of shots).

Then methods for estimating the shot group standard deviation from measurements (individual shot locations or group diameters) are presented in Chapter 4.

Finally, Chapter 5 gives some applications. The relationship between the shot group SD and the score obtained in competition shooting is discussed for the case of air rifle. A next issue is matching ammunition to the rifle.

 

Part of the work in this document is analytical, but most of the results were obtained by a Monte-Carlo type of analysis. For many combinations of shooter quality and ammo&rifle quality, millions of virtual shots were fired in a set of computer programmes to find out the effects on the grouping and the score, etc.

 

This document combines some of my earlier web pages. Some people have contacted me asking for more details about the statistical methods. So I have added more details and more equations. That will not make be very interesting for those who are only reading this looking for practical consequences. Sorry about that. Maybe one day I will find the time to separate this document into two versions: one for people who want to know all the statistical details and one for normal people. For now, the latter category can go straight to Section 4.4 and Chapter 5.

 

 

2       One-shot probability distributions 

The horizontal deviation of the centre of a shot hole and the centre of the target is called x; the vertical deviation of the centre of a shot hole and the centre of the target is called y.

 

One way to start is from the bivariate normal distribution density function (Clarke, & Disney, 1972). Two random variables X and Y are said to have this distribution if they have ranges -¥<x<¥, -¥<y<¥ and a joint density function of the form

 

where

 are constants with, and

 

According to Clarke and Disney (1972), calculus computations show that

1.      this is indeed a density function

2.      the marginal densities fX(x) and fY(y) are normal, and

3.      X and Y are independent if, and only if,

 

For:

 

this simplifies to:

 

The probability density function is plotted as a function of x and y in Figure 1 for a standard deviation of 1.0 and 1.5. For increasing standard deviation, the peak in the centre becomes lower and the probability density further away from the centre becomes higher.

 

 

Figure 1 Probability density function for SD 1.0 (upper) and SD 1.5 (lower) (means are 0, x and y are independent).

D:\JEROENX\SKEETN\monte\pdf_3d.m

 

Changing to polar co-ordinates:

 

 

For a given range r (from r1 to r2), the probability of a hit can be determined by integrating f:

 

N.B.

 

The probability density function f (r) thus becomes, for r>0:

                                                                  [Eq.1]

 

This function is known as the Rayleigh distribution function.

 

The mean mR and standard deviation  of a Rayleigh process are (Evans, Hastings, & Peacock, 1993):

                                                                                         [Eq.2]

                                                                                [Eq.3]

 

In Figure 2, this function is shown for various levels of the standard deviation. The figure shows that a peak occurs at an r equal to the standard deviation . The lower the standard deviation, the more pronounced the peak is.

 

D:\JEROENX\SKEETN\monte\plot_pdf.m

 

Figure 2 Probability density function of r (distance between shot centre and target centre) for several standard deviations.

 

From this density function, the cumulative probability distribution function F(r) can be solved:

                                                                  [Eq.4]

This function is illustrated in Figure 3.

Figure 3 Cumulative probability distribution function of r (distance between shot centre and target centre) for several standard deviations.

In Figure 4, the inverse probability distribution function is plotted as a function of x and y for standard deviations of 1.0 and 1.5, respectively. The inverse was used for visualisation purposes.

 

This is basically the same graph as Figure 3. For any point in the x-y plane, this plot shows the probability that a shot will be worse than in point under consideration. For instance, in a corner of the plot (x=y=3), this probability is about 0, meaning that it is unlikely that a next shot will be further away from the target centre than this point (x=y=3). On the other hand, precisely in the centre of the target (x=y=0), the probability is 1, meaning that it is impossible that a shot will be better than at this point.

Figure 4 Inverse probability distribution function SD 1.0 (upper) and SD 1.5 (lower) (means are 0, x and y are independent).

D:\JEROENX\SKEETN\monte\cum_pdf_3d_a.m

 

With the assumptions listed above, it is easy to simulate individual shots. Any random number generator that follows a normal distribution can be used: two independent random numbers can represent the x and y coordinate of the shot centre, respectively. Only one parameter is involved: the shot group standard deviation (SD).

 

 

 

3       Shot group SD and shot group diameter

D:\JEROENX\SKEETN\monte2\sd_vs_mean_SGD.m

Once a single shot can be simulated, so can groups of shots. Thus it becomes possible to analyse the resulting shot group diameter. By doing this many times, the distribution of the shot group diameter can be determined for e.g. a 5-shot or a 10-shot group. Here, the shot group diameter is defined as the distance between the centres of the furthest apart two shots. Examples of such distributions are shown in Figure 5 for two levels of the shot group SD.

 

Figure 5 Distribution of shot group diameter: histograms of simulated data, and fitted normal probability distributions (5 shots in a group, 10,000 groups per distribution, shot group SD 2 and 6 mm).

The following observations can be made from Figure 5.

·             The normal distribution appears to be a reasonable approximation of the distribution of the shot group diameter. However, the distributions do differ somewhat from the normal probability density function, as seen from the figure and confirmed by results from Lilliefors tests for goodness of fit.

·             When the shot group SD is increased, the mean of the shot group diameter increases. Further analysis showed that the relationship between these two variables is linear, and only depends on the number of shots in the group (see Figure 6).

·             When the shot group SD is increased, the SD of the shot group diameter increases as well. Again the relationship is linear (not shown here).

 

Figure 6 Mean shot group diameter as a function of the shot group SD (simulated data and fitted linear function).

 

Thus, the SD and mean of the shot group diameter turn out to be linearly related, see Figure 7. The slope depends on the number of shots in the group; the constants are 0.2691 and 0.1947 for the 5 and 10 shots groups, respectively.

 

Figure 7 Relationship between the SD and the mean of the shot group diameter (simulated data and fitted linear function).

 

4       Estimating the shot group SD

Starting with a given shot group SD, one can simulate shots and hence shot groups. For practical applications, it will be necessary to reverse this path: given real-world shot data, how can one estimate the underlying shot group SD?

 

As shown in Chapter 3, the mean shot group diameter increases as a linear function with increasing shot group SD. This implies that shot group diameters can be used to estimate the shot group SD. This is discussed further in Section 4.1.

When it is possible to measure individual shot locations (either by hand when the holes are far enough apart or electronically) this more detailed information from a shot group can be used as well. This method is described in Section 4.2. Both methods are compared in Section 4.3.

 

4.1   From shot group diameters

To obtain an unbiased estimator, the following procedure was used.

 

Some results (median, 85th and 15th percentiles) are shown in Figure 8. Obviously, the shot group diameter increases when the SD increases or when the number of shots increases. This is in line with the findings from Chapter 3.  Furthermore, the as the number of shots in the group increases, the range of the shot group diameter distribution (from the 15th to the 85th percentile) decreases.

Figure 8 Median (box) and 15th and 85th percentiles (whiskers) of the shot group diameter as a function of shot group SD and number of shots.

 

Using the resulting data set, the Eq. [5] was used to obtain an unbiased estimator.

 

            SDe = an*(sgd-cal)                                 [Eq.5]

where

SDe= SD estimation (mm)

sgd = shot group diameter, outer edge to outer edge (mm)

cal = the calibre (mm)

an = a constant (-) in the procedure (dependent on n, the number of shot in the group)

 

Next, the error of the estimation for each observation was defined as:

            e =  SD - SDe                                                                   [Eq.6]

 

The constant an from [Eq.5] was adjusted using a numerical procedure to a value where the mean error over all observations was 0. The resulting constants are shown in Table 1.

 

Table 1 Estimation parameter an as a function of the number of shots in a group

 

group

an

2

0.559228653

3

0.413495343

4

0.359363108

5

0.325779489

6

0.305424427

7

0.291006222

8

0.279012321

9

0.26960721

10

0.263336239

11

0.25671031

12

0.250719049

 

4.2   From individual shot locations

In some situations, it is possible to measure the locations of the individual shots in a group:

·             on a paper target when the group is widely enough to distinguish the individual holes,

·             when using electronic targets, and

·             in simulations.

 

In simulations, the location of each shot is known with respect to the centre of the distribution. Thus, the distance of each shot centre to the distribution centre can be calculated:

                                               [Eq. 7]

 

When analysing a shot group with n shots, there are n observations of r. Based on these observations, the following (Maximum Likelihood) Estimation for  can be used (Evans, Hastings, & Peacock, 1993):

                                               [Eq. 8]

 

To test this method, simulations were carried out for various fixed levels of . The estimation error was calculated from [Eq.6]  and analysed in terms of the mean and standard deviation. Results for 10-shot groups are shown in Figure 9.

 

D:\JEROENX\SKEETN\monte2\sigma_estimations0.m

 

std_factor_MLE =    0.1572

bias_factor_MLE =   -0.0126

As the graph shows, the mean error is slightly positive. This means that the Maximum Likelihood Estimation is somewhat biased;  the shot group SD is underestimated by approximately 1.2%. The standard deviation of the estimation error increases with the shot group SD: the estimation error SD is 15.7% of the shot group SD.

 

 

Figure 9 Mean and standard deviation of the estimation error for 10-shot groups as a function of the shot group SD (Maximum Likelihood Estimator).

 

This is in the rather theoretical situation, where the distances are measured with respect to the origin of the axes, on which the shot group distribution is aligned. In practice, this is in the situation where you have first have accurately zeroed the rifle and then measure the shots with respect to the centre of the target, as illustrated in Figure 10A.

In reality, when testing new pellets, bullets or loads, it is realistic that you only have the shot group, without having zeroed exactly. Thus, in terms of measuring your results, it is more convenient to express the distances of the shots with respect to the group’s centre, whose coordinates are found by averaging the x and y coordinates of all shots in the group. This is illustrated in Figure 10B. This approach will yield distances equal to, or smaller than the theoretical method. Thus, the estimations of  will turn out smaller.

This is confirmed by further simulations in which the estimator is based on the distances with respect to the group centre (see Figure 11). The bias is now about 6.5% (i.e., the shot group SD is underestimated). As before, the error SD is 15.7% of the shot group SD.

 

bias_factor_MLE_reeel = 0.0646

std_factor_MLE_reeel = 0.1570

 

Estimation based on the mean radius:

 

 

 

Figure 10 A random shot group (SD=2 mm): distances with respect to the target centre (A - upper) and with respect to the group’s centre (B- lower).

 

 

Figure 11 Mean and standard deviation of the estimation error for 10-shot groups as a function of the shot group SD (Maximum Likelihood Estimator; shot locations with respect to centre of shot group).

Since the bias is a constant fraction of the variable to be estimated, the estimation can be corrected for this bias. When the error is a constant fraction K of the shot group SD to be estimated:

 

then K can corrected for to obtain an unbiased estimation

 

K varies with the number of shots; as shown above, for 10-shot groups, K is 0.157.

 

K as function of n; estimated in

D:\JEROENX\SKEETN\monte2\sigma_estimations4.m

result in biastabel.txt

 

4.3   Comparing both estimation methods

In the previous two sections, two methods for estimating the shout group SD from empirical data have been discussed. Now both methods will be compared. In this comparison, the Maximum Likelihood Estimator was based on distances with respect to the group centre, and corrected for bias.

 

Figure 12 Standard deviation of the estimation error as a function of shot group SD, using groups of 10 shots: Maximum Likelihood estimator (MLE, compensated for bias, 1 group), and shot group diameter method (SGD, 1 and 2 groups of 10 shots).

 

Some results are shown in Figure 12. When using one group of 10 shots, the Maximum Likelihood Estimator gave better results than the shot group diameter method (ratios in Figure 12 are 17% and 20%, respectively). When using the shot group diameter method with 2 groups of 10 shots, the estimator (ratio 14%) is better than the Maximum Likelihood Estimator based on one group.

 

D:\JEROENX\SKEETN\monte2\sigma_estimations2.m

mean_std_MLE = 0.1677

mean_std_SGD1 = 0.1953

mean_std_SGD2 = 0.1381

 

D:\JEROENX\SKEETN\monte2\sigma_estimations3.m

Figure 13 Relative standard deviation of the estimation error as a function the numbers of shots in a group: Maximum Likelihood method (corrected for bias) versus group diameter method.

 

Both methods are compared in Figure 13, showing the relative standard deviation (relative to the shot group SD to be estimated, i.e. the slope in Figure 12) as a function of the number of shots in the group. For a group of two shots, both methods have equal quality, but for groups with more shots the Maximum Likelihood method is (somewhat) better than the diameter method.

 

 

4.4   How many groups of how many shots?

D:\JEROENX\SKEETN\monte2\sigma_estimations1.m

Sections 4.1 and 4.2 yield the means to estimate the shot group SD from measurements. The next question is if there is an optimal way of using these methods, given a certain number of shots you want to spend. For instance, suppose you want to use 12 shots to get an estimate of the SD. What is the best way to do this:

 

Similar, for a total of 6 shots, 1 group of 6 shots can be fired, 2 groups of 3 shots, or 3 groups of 2 shots. In general, a total of p shots can be fired in several combinations of m groups with n shots each, where p = m*n (m>=1; n>=2)

 

To find this out, another Monte Carlo analysis was run. For each of the m*n combinations listed above (1 group of 12 shots, 2 groups of 6 shots, etc.), the required shots were ‘fired’ and the SD was estimated using the shot group diameter method [Eq.5]. When m groups were fired, the resulting m SD estimation values were averaged to obtain a single overall SD estimate. Finally, the estimation error was determined using [Eq.6]. The mean of the estimation error is expected to be (about) 0. The standard deviation of the estimation error is a measure for the quality of the estimator: the lower this SD, the better the estimator.

The mean and SD of the estimation error are depicted in Figure 14 and in Figure 15 as a function of the shot group SD, for 12 vs. 6 shot groups, and for 1*12 vs. 6*2 shot groups, respectively. Figure 14 shows (in line with Section 4.3 findings, see Figure 13) that the estimation improves when more shots are used. Figure 15 shows that an estimation based on one group of 12 shots gives better results than an estimation based on 6 groups of 2 shots.

 

Figure 14 Mean and standard deviation of the estimation error as a function of shot group SD and method: 6 shots vs. 12 shots.

 

Figure 15 Mean and standard deviation of the estimation error as a function of shot group SD and method: 1 group of 12 shots vs. 6 groups of 2 shots.

 

 

Further results are shown for a total of 12 shots in Figure 16. The worst estimator is obtained when 6 groups of 2 shots are used. The other four conditions are close together; the best estimator is obtained when 2 groups of 6 shots are used.

 

The results for a total of 10 shots are shown in Figure 17.

 

 

Figure 16 Standard deviation of the estimation error as a function of the shot group SD and the number of targets and shots (total 12 shots).

 

Figure 17 Standard deviation of the estimation error as a function of the shot group SD and the number of targets and shots (total 10 shots).

 

Figure 18 Standard deviation of the estimation error as a function of the shot group SD and the number of targets and shots (total 8 shots).

In conclusion:

 

 

5       Applications

5.1   Scores in competition shooting

The ISSF rules define the dimensions of the scoring rings of the shooting target. By using the known dimensions of each scoring ring, taking into account the radius of the bullet, and by integrating the probability density function over the relevant range of r, the probability of a shot hitting in a specific ring can be calculated. By doing that for each scoring ring, the total score for e.g. a 60 shot match can be calculated as a function of the standard deviation.

 

 

where

 

Since F(r) is a function of the shot group SD, so is the mean score. This is shown in Figure 19 for the air rifle discipline.

Figure 19 Mean score as a function of shot group standard deviation (integer scoring and decimal scoring, 60 shots, air rifle).

D:\JEROENX\SKEETN\monte\ammo_sgd_main.m, Figure 1

 

There is mainly a linear relationship: for each mm increase of the SD, the mean score drops by 30 points. However, for a SD below 1 mm, there is no effect of the SD on the integer score. This can easily be understood: considering that (1) just touching the ‘10’ is enough to get a ’10’ on your scoring card, (2) the ‘10’ has a 0.5 mm diameter and (3) the pellet has a 4.5 mm diameter, you can see that starting from a 10.9 there is some margin for error before the ‘10’ becomes a ‘9’. When the SD increases, starting at 0, at first you are still within the margin that gives you a ‘10’ even for a non-perfect shot. Once you are out of this margin (above about SD=1 mm), the mean score drops with increasing SD.

 

For a given the standard deviation (and, using Figure 19, the corresponding score), the distribution of points can be derived from the cumulative probability distribution function. The probability function [Eq.2] can be used to find the proportion of shots hitting in a given radius range. Some results are shown in Figure 20 for the air rifle discipline.

 

shoot2d.pas; uitvoer naar shoot2d.txt, 3 kolommen: SD(mm), avg_0, avg_10

 

 


        Figure 20 Scoring distribution as a function of the total score (air rifle).

 

For example, when you are good enough to shoot 570, you can see that is still ‘normal’ to have an occasional 8…

 

 

 

5.2   Matching Ammunition to the Rifle

The reason why it may be a good idea to search ammunition that matches your rifle is shown in Figure 21. If you pick a random can of pellets, you may end up in a situation where you lose points simply because of an unlucky combination of ammo and rifle.

 

   

 

Figure 21 Two five-shot air rifle groups, fired from a rest: good ammo, bad ammo and a detail of the ISSF target shown on the same scale.

 

This section discusses the effects on the result (i.e. shooting score) of error introduced by non-perfect ammo in combination with a non-perfect shooter. All examples are based on ISSF air rifle shooting, i.e. using:

 

Two ways of counting the score are distinguished:

 

The same assumptions as before were used. Once again:

 

One new assumption is that the error introduced by the shooter and the error introduced by the ammo are independent from each other.

 

In Chapter 3, it has been shown that there is a linear relationship between the average shot group diameter and the shot group SD. Furthermore, the average shot group diameter increases with the number of shots fired. This is shown again in Figure 22, now for groups of 3 and of 10 shots.

 

Figure 22 Average shot group diameter as a function of the shot group standard deviation (SD).

 

A second element needed here is how the shot group SD relates to the shooting result, i.e. the score. This is shown in Figure 19 for a 60-shot match.

 

Now, distinguish 2 sources of dispersion:

 

It is assumed that these sources contribute independently to the total shot group dispersion. The question is how the total dispersion relates to these two sources of dispersion. As statistical textbooks will tell you, the total variance is the sum of the two variances. Since the standard deviation is the square root of the variance, this means:

 

                                                      [Eq.5]

 

In the examples presented below, the following line of reasoning is followed.

 

 

Table 1 SD for three pellet quality levels

Pellet quality

shot group size

shot group diameter (mm)

SD (mm)

Perfect

inf. shots

4.5

0.00

Bad

3 shots

9.0

 1.87

Fair

3 shots

6.75

 0.94

Good

10 shots

 5.0

 0.13

 

5.2.1    Integer score

For non-perfect pellets, the total SD depends on the SD of the ammo and the SD of the shooter according to [Eq.5]. Figure 23 shows the effect of the SD of the shooter and the ammo quality on the normal score. For perfect ammunition, SDAMMO =0 and according to [Eq.5], the SDTOT equals the SDSHOOTER. Thus, in this situation, the relation between the SD of the shooter and the score is the same as in Figure 19. Now switching to less perfect ammo, the score is lowered for all levels of SDSHOOTER.

The same data are shown in Figure 24, now using the score the shooter would obtain with perfect ammunition as the horizontal axis.

 

 

Figure 23 Effect of SD of the shooter and the ammo quality on the average normal (integer) shooting score.

Figure 24 Loss of points due to non-perfect ammo as a function of ammo quality and shooter’s ability (normal, integer score)

 

The figures shows the following results.

5.2.2    Decimal scoring

The same approach was used for the situation with the decimal (finale) scoring. Here, the maximum score over a series of 60 shots is 60*10.9 = 654. The results are shown in Figure 25 and Figure 26.

 

Figure 25 Effect of SD of the shooter and the ammo quality on the average decimal shooting score.

Figure 26 Loss of points due to non-perfect ammo as a function of ammo quality and shooter’s ability (decimal score)

The figures show the following results.

 

 

5.3   Using shot group diameters in the pellet selection process

The information from the previous sections can be used to select pellets that are ‘good enough’ for a given performance level of the shooter, i.e., good enough to ensure that the shooter does not lose points due to the (mis-)match between rifle and ammo. Basically, the approach that was used is as in this example:

 


This method was applied for a range of ideal scores and of sd_ammo values to yield Figure 27.

 

D:\JEROENX\SKEETN\monte\ammo_sgd_main.m

Figure 27 Mean loss of points (out of 60 shots) as a function of the ideal score of the shooter and of the standard deviation (SD) of the ammo.

The final step is to make this applicable in practice is to relate the mean loss of points to the mean shot group diameter as you measure it in an ammo test, i.e. only due to sd_ammo. This relationship was derived in Section 4.1. One extra parameter involved in this translation is the number of shots in the group.

 

Combing all the elements described above yields Figure 28. Given your ideal score and given a shot group diameter from a group of 10 shots, you can see how many points you will lose on average due to your ammo being less than perfect.


 

 

Figure 28 Mean loss of points out of 60 shots (integer scoring) as a function of the mean shot group diameter (10 shots) and of the ideal score of the shooter.

 

For example, if your ideal score is 570, and you are using pellets that give a shot group diameter of 6.5 mm (2 mm more than the pellet diameter) in 10-shot groups during ammo tests, you will (on average) lose 2 points due to your poor ammo quality.

 

 

 

References

 

Clarke, A.B., & Disney, R.L. (1972). Probability and random processes: a first course with applications (2nd ed.). New York: John Wiley & Sons.

 

Evans, M., Hastings, N., & Peacock, B. (1993). Statistical Distributions. Second Edition, New York: Wiley.

 

 

 

To articles overview