The title is provocative, but the thesis is less provocative (and in essence well-known: Hawthorne's work on the deeply contingent a priori is relevant) once I spell out what I stipulatively mean by the terms. By evidential Bayesianism, I mean the view that evidence should only impact our credences by conditionalization. By evidentialism, I mean the view that high credence in contingent matters should not be had except by evidence (most evidentialists make a stronger claims). By weak fallibilism, I mean that sometimes a correctly functioning epistemic agent appropriately would have high credence on the basis of non-entailing evidence. These three theses cannot all be true.
For suppose that they are all true, and I am a correctly functioning epistemic agent who has appropriate high credence in a contingent matter H, and yet my total evidence E does not entail H. By evidentialism, my credence comes from the evidence. By evidential Bayesianism, if P measures my prior probabilities, then P(H|E) is high. But it is a theorem that P(H|E) is less than or equal to P(E→H), where the arrow is a material conditional. So the prior probability of E→H is high. This conditional is not necessary as E does not etnail H. Hence, I have high prior credence in a contingent matter. Prior probabilities are by definition independent of my total evidence. So evidentialism is violated.