Frequentism and explanation

This is really very obvious, and no doubt in the literature, but somehow hasn't occurred to me until now. Suppose that a fair coin is tossed an infinite number times. Suppose, further, than in the first hundred tosses it lands heads about half the time. It's no mystery why it lands heads about half the time in the first hundred tosses: it's because the probability of heads is 1/2 (plus properties of the binomial distribution). But suppose frequentism is true. Then the reason the probability of heads is 1/2 is that the infinite sequence has a limiting proportion of heads of 1/2. Now consider these three statements:

• A: approximately half of the first 100 tosses are heads
• B: the limiting proportion of heads is 1/2
• C: the limiting proportion of heads starting with the 101st toss is 1/2.

Then, C is statistically independent of A, as A depends on the first 100 tosses and C depends on the other tosses. Clearly, C has no explanatory power with respect to A. But B is logically equivalent to C (the first 100 tosses disappear in the limit). How can B explain A, then?

There are some gaps in the argument--explanation is hyperintensional, for instance. But I think the argument has a lot of intuitive force.

A theistic frequentist theory of chances

An event E has divine-intention probability p if and only if E is part of a system S of events such that there is a probability function P applicable to events in S and God intends S to be such as to be described well by P. For instance, suppose over the history of the world a million coins are tossed and of these 50.1% land heads and other statistical properties of the coin tosses are described well by the assumption that the coin tosses are independent events with each outcome of equal probability. Then if God intended that the coin tosses be such as to be thus described, each coin toss has a divine-intention probability 1/2 of being heads.

In the case of finite sequences, standard frequentism suffers from the problem that it gives intuitively incorrect answers. In the above scenario, it would say that the probability of heads is 0.501. But surely it is possible for a million coins to be tossed, each with probability 1/2 of heads, and yet to get 50.1% of the coins landing heads.

Lewis's regularity theory of laws can get the same result as the theistic version, for in the case of probabilistic laws one trades accuracy against simplicity. But regularity theories of laws get the order of explanation mixed up: chances are supposed to be explanatory rather than descriptive (of course Humeans will say that this is a false dilemma, but they are wrong).

Theistic frequentism and evolution

Readers here may be interested in this post of mine on Prosblogion.

Random numbers and their sequences

Bear with a simple and standard bit of mathematics: the mathematics may give us lessons about God and evolution, frequentism, single-case chances and Humean views of causation.

Consider the following standard one-to-one and onto map between the interval [0,1] and the space [0,1]^ω of infinite sequences of numbers from that interval. The map starts with a single decimal number x=0.d₁d₂d₃... in [0,1][note 1] and generates an infinite sequence ψ(x)=(ψ₁(x),ψ₂(x),...) by taking every second digit of x after the decimal point and letting that define ψ₁(x), then discarding these digits, taking every second digit of what remains and letting that define ψ₂(x), and so on. Thus, ψ₁(x)=0.d₁d₃d₅..., ψ₂(x)=0.d₂d₆d₁₀..., ψ₃(x)=0.d₄d₁₂d₂₀..., and so on.

Interestingly, ψ not only shows that [0,1] and [0,1]^ω have the same cardinality, but if we equip [0,1] with a uniform probability measure and [0,1]^ω with an infinite product of uniform probability measures, i.e., let [0,1]^ω be the probability space modeling infinite independent choices of uniformly distributed numbers in [0,1], then it turns out that ψ is a probability-preserving isomorphism. Hence, the two probability spaces are probabilistically isomorphic. There is, thus, "nothing more" to choosing an infinite sequence of uniformly distributed numbers in [0,1] than there is to choosing a single such number.

And of course what goes for [0,1] and [0,1]^ω also goes for finite sequences: the probability-preserving isomorphism between [0,1] and [0,1]ⁿ is even easier to construct.

There are some potential philosophical consequences of this isomorphism: it shows that there is no principled difference between single-case and sequences, when we're willing to deal with continuous outcomes (there is when we have a finite outcome space).

Lesson 1: Anybody who believes in the utter impossibility of single-case chances or probabilities, including for continuous-valued events like decay times or darts thrown at boards, should believe in the utter impossibility of chances or probabilities in the case of infinite sequences as well.

Thus, Lesson 2: Frequentism is dubious.

Lesson 3: If probabilistic causation with continuous-valued outcomes is possible, single-case probabilistic causation should be possible, and in particular single-case causation should be possible. For there is in principle no difference between single-case and sequential probabilities.

Thus, Lesson 4: Humeanism about causation is dubious.

Lesson 5: Given that it is plausible that if God intentionally and specifically chooses just a single real number in [0,1] with full precision, that real number isn't genuinely random in the sense scientists like biologists or quantum physicists mean, neither will an infinite sequence of divine choices embody randomness. Hence, reconciliations between random evolution and exhaustive divine planning of every particular event fail.

A new sceptical argument

Let's say that an infinite sequence of real-numbered observations is generated by independent runs of a random process. Suppose that we can represent the runs of the random process as independent and identically distributed random variables X₁,X₂,.... Recall that a random variable is a function f from some probability space Ω to the reals R with the measurability property that f⁻¹[B] is a measurable subset of Ω for every Borel-measurable subset B of the reals R (and it's enough to check this for B an interval, since the intervals generate the Borel sets).

It turns out that under these assumptions we can almost surely recover the distribution of the random variables X_i from the observed sequence. For almost surely the frequency of the observations fitting into any given interval with rational numbers as ends will converge to the probability that X_i is in that interval. And since there are only countably many such intervals, almost surely for every such interval I we can read off the probability P(X_i in I) from the observed frequencies. And then by the uniqueness condition in the Caratheodory extension theorem, we can recover the probability of X_i being in A for any Borel subset, not just a rational-ended interval.

So far this sounds like a kind of vindication of infinitary frequentism. It is a helpful, optimistic result.

But notice a crucial assumption the recovery of the distribution of the X_i made: that the X_i are measurable when considered as functions to the Borel-measure space R. But there are infinitely many other σ-algebras on R besides the Borel one. When recovering the distribution from the observations, what justifies the assumption that the X_i are measurable as a function to R considered as coming with the Borel σ-algebra?

We might have some hope that the recovery process will be likely to fail if the X_i aren't measurable in this way, so that if the recovery process succeeds, we have good reason to think that the X_i have this measurability condition. But I suspect that some analogue of these results will show that this won't work.

So I guess we just need to assume measurability with respect to Borel sets. But why? Because God loves the Borel sets? It's not so crazy. I love the Borel sets, and God made me in his image, after all. :-)

Defending infinitary frequentism from some arguments

Frequentism defines probabilities in terms of long-term frequencies of outcomes. This doesn't work very well with finite frequencies—for one, it's going to conflict with physics in that finite sequence frequentism can only yield rational numbers as probabilities while quantum physics is quite happy to yield irrational numbers. As a result frequentism is often extended to suppose a hypothetical infinite sequence of data for defining frequencies. Alan Hajek has a paper that gives fifteen arguments against such a frequentism. Fourteen of them are strong arguments against standard hypothetical frequentism (I am unmoved by argument 15 involving infinitesimals, since I doubt that infinitesimal probabilities are much use to us).

But it turns out that one can formulate a frequentism that escapes or partly escapes some of Hajek's arguments.

A representative of these six arguments is the observation going back to De Finetti that probabilities defined via frequencies fail to satisfy the Kolmogorov Axioms (arguments 13 and 14). But my modified frequentist probabilities satisfy the Kolmogorov Axioms.

For simplicity, our data will be real valued, but the extension to Rⁿ is easy. Let s=(s_n) be our sequence of real numbers in R. For any subset A of R, let F_n(s,A) be the proportion of s₁,...,s_n that is in A. Let L(s,A) be the limit of F_n(s,A) if that limit exists, and otherwise L(s,A) is undefined.

Say that a (classical) probability measure m on the Borel subsets of R fits s provided that for all subintervals I of R, L(s,I) is defined and L(s,I)=m(I).

If there is a probability measure m that fits s, let a frequentist probability measure P_s defined by s be the completion of m (basically, the completion of a measure sets the measure of all subsets of null sets to be measurable and have measure zero).

Proposition:

1. If s defines a frequentist probability measure, it defines a unique frequentist probability measure.
2. Suppose that P is a probability measure and X=(X_n) is a sequence of independent identically distributed random variables. Let P₁ be the measure on R defined by P₁(A)=P(X₁ in A). Then with probability one, X defines a frequentist probability measure on R which coincides with P₁.

Because P_s is an ordinary Kolmogorovian probability measure, Hajek's arguments 13 and 14 do not apply. Argument 15 is anyway not very convincing, but is weakened since our version of frequentism handles the case of a dart thrown at [0,1] about as well as one can expect classical probabilities to handle it. (There is also a tension between arguments 13-14 and 15, in that probabilities involving infinitesimals are unlikely to be Kolmogorovian.) It is plausible that our frequentist probability measure will provide frequencies only when there is a probability, which makes argument 8 not apply, and non-uniqueness worries from argument 4 are ruled out by (1). I think the frequentist can bite the bullet on arguments 5 and 6, whether with standard frequentism or our modified version, given that the problem occurs only with probability zero.

Remark: The big philosophical problem with this is the reliance on intervals.

Quick sketch of proof:

Claim (1) is easy, because two Borel measures that agree on all intervals agree everywhere.

Claim (2) is proved by letting S be the collection of (a) all intervals with rational numbers as endpoints and (b) all singletons with non-zero P₁ measure, and using the Strong Law of Large Numbers to see that for each member A of S almost surely L(X,A) exists and equals P₁(A). But since S has countably members (obvious in the case of the intervals, but also easy in the case of the singletons), almost surely for every A in S we have L(X,A) existing. Moreover, almost surely, no singleton with null P₁ measure will be hit by infinitely many of the X_n, and hence L(X,A) will be defined and equal to zero for all such singletons.

Thus there is a set W of P-measure one such that on W, we have L(x,A) existing and equal to P₁(A) for every A that is either an interval with rational number endpoints or a singleton. Approximating any other interval A from below and above with monotone sequences of rational-number-ended intervals plus or minus one or two singletons, we can show that L(x,A) exists and equals P₁(A) for any other interval everywhere on W.

Another argument against frequentism

Suppose that the probability of a process Q being followed by A is given by the ratio of the number of times it is followed by A to the number of times the process occurs. Suppose that Q is run only finitely many times in the history of the universe, and that the frequency is some number f(A,Q) strictly between 0 and 1. Here is an argument against the claim that f(A,Q) is equal to the probability P(A|Q) that Q is followed by A: if the number n of occurrences of Q is large, and the occurrences are independent, the probability distribution of the observed frequency of As will be approximately a Gaussian centered on P(A|Q) with a standard deviation proportional to n^−1/2. Because a Gaussian is flat around its peak, the probability that the observed frequency of As will be exactly equal to P(A|Q) is small (and tends to zero as n tends to infinity). Hence, it is unlikely that the observed frequency for an independent process that occurs many times should equal the probability. But if frequentism is true, it is certain that the observed frequency equals the probability. The only way this could be true, assuming the frequency is strictly between 0 or 1, is if the process is not independent. Therefore, if frequentism is true for a process with a frequency strictly between 0 and 1, the process is not an independent one. But sure it is possible to have independent processes with frequencies strictly between 0 and 1. Hence, frequentism is not true.