The probability of obtaining the value of the realized test statistic, or a more extreme value, is the proportion of test statistic values equal to or more extreme than the realized value.
Second, as sample sizes increase, the number of permutations can become very large and, consequently, only a random sample of all possible test statistic values is generated and examined