To determine the misdetect of the test, we will do a 10,000 trial Monte Carlo experiment which for each trial we will use for our first data set, a data set generated according to the model and the remaining data sets generated according to the Null hypothesis. For each trial, we will compute the test statistic, which is the closest interpoint distance between the sets. We will find what fraction of trials had closest interpoint distances greater than the critical value .005513 . Recall that when the closest interpoint distance is greater than the critical value, we do not reject the Null hypothesis. Rather we reject the Alternative hypothesis in favor of the Null hypothesis. And in this case we have a misdetection.

In a 10,000 Monte Carlo trial experiment involving point sets where one point set was moved closer to the other point set, 4319 trials had a closest point distance greater than or equal to .005513. Hence the misdetect rate is .4319. The power of the test is one minus the misdetect rate. So the power of the test is .5681. If the Alternative hypothesis is true, only 56.81% of the time will this test reject the Null hypothesis. This test is not a good test.

There are ten points in the first set and ten points in the second set. Hence there are a total of 100 interpoint distances associated with the point sets. We next explore as test statistics the arithmetic mean, the harmonic mean, the geometric mean, and the maximum interpoint distance of these 100 interpoint distances.

The table below shows the misdetect rate when we perform a hypothesis test at the 1% significance level. The results from this should be interesting to us because the inner part of the WRR methdolology essentially makes a computation that is like the harmonic mean. Interestingly enough, we see that among the test statistics we examined the harmonic mean does best.

Table showing the misdetect rate for different test statistics for a 1% significance level test

< Previous >         < Next >