Statistics in EXAFS analysis

My plan is to adapt the content of the statistics section of my Advanced Topics lecture from the 2008 APS XAFS school. I will make reference repeatedly to Data Reduction and Error Analysis for the Physical Sciences by Bevington and Robinson and to Science and Information Theory by Leon Brillouin. (I don't necessarily want to shill for Amazon, so I am open to other ways of referring the reader to these texts....)

The theme of the module will be a series of "What now?" questions. I will present a series of increasingly subtle EXAFS analysis results and ask a question whose answer requires an understanding of the statistical model we use and how it is applied in Ifeffit. Basically, each launch point into a discussion of statistics is a moment when one may have completed a bit of analysis and is asking a "What now?" question either in the sense of "What do I need to continue making progress?" or "How do I understand my current result?" I am hoping for a kind of synergy with Scott's module about deciding when to stop fitting.

There is going to be math in this module. It cannot be avoided. To discuss with Steve: images of equations or MathML?

Each of the level 2 heading to follow is intended to be its own web page. Additionally the examples I plan to show for the last two headings will need to be individual pages.

Open questions:

• Incorporate spoken pieces into the pages?

• An N_idp calculating applet would be helpful (i.e. drag sliders for k- and R-ranges over some real data and have N..idp,, be updated in real time)

• Something like the java app about halfway down the page at http://webmineral.com/data/Periclase.shtml would be very handy for understanding the uranyl example. That particular one is probably not the right option, so I'll need to look into other options along with Steve.

Introduction to the problem

1. Show beautiful data measured under somewhat challenging conditions → this is a relatively simple material and we have N_idp = 19

2. Show icky data measured as a standard fluorescence experiment → this is a complicated and entirely unknown structure and we have only N_idp = 8

3. In either case, the situation is the same ... we need to know certain things about the structure. Our approach is to use Feff as our theory, to come up with a model for the structure, to do a fit, and to somehow interpret that fit to tell us what we need to know. The perverse thing about a real experiment is that we tend to have lovely data for simpler problems and ickier data for more complex problems. In any case, we need to understand statistics in order to make the best possible use of the data we have.

Introduction to the basics of satistics

1. Show the EXAFS equation

2. Define χ², χ²_nu, and R-factor, discuss how these get evaluated for fits in k, R, or q space.

3. Bevington says:

   Values of [reduced chi-square] much larger than 1 result from large deviations from the assumed distribution and may indicate poor measurements, incorrect assignment of uncertainties, or an incorrect choice of probability function.
                            Page 69, Second Edition

4. This is often interpreted to mean that a non-linear, least-squares minimization should yield a χ²_nu≈1 for a good fit.

5. Show a two-shell fit of copper. Excellent fit, χ²_nu≈32.

6. Now what?

Interpreting reduced chi-square

1. Bevington's quote above is really quite deep. The χ²_nu prescription applies to our problem if

   1. The error in the fitting parameters is normally distributed
   2. We understand and can enumerate all sources of measurement error
   3. We know the theoretical lineshape of our data

2. In practice, none of these is true for an EXAFS measurement

3. Explain where ε comes from in practice, why this is a poor approximation, and what impact an underestimate of the size of ε has on the evaluation of χ², contributory problems include

   • approximations in the theory
   • sample inhomogeneity
   • sample redox in the beam
   • detector nonlinearities
   • noise in the signal chains
   • gremlins!

4. Explain where N_idp and ν come from. Nyquist criterion, cite Brillouin. Nyquist further requires that the signals be ideally packed. Explain why EXAFS data are not ideally packed, i.e. they are not simple sine waves due to the other terms in the EXAFS equation.

5. N_idp is an upper bound (and perhaps a generous upper bound) of the information content of your data. (Perhaps cite Rossner and Krappe using Bayes to determine actual information content as sometimes as small as 2/3 of N_idp)

6. So, we cannot expect χ²_nu to be ≈ 1, even when a fit seems good... Now what?

How do we use reduced chi-square in practice?

1. χ²_nu is always somewhere between big and enormous.

2. χ²_nu is impossible to interpret for a single fit.

3. χ²_nu can be used to compare different fits. A fit is improved if χ²_nu is significantly smaller.

4. Error bars are taken from the diagonal of the covarience matrix. If χ²_nu is way too big, the error bars will be way too small. The error bars reported by ifeffit have been scaled by the square root of χ²_nu

5. Thus the error bars reported by ifeffit are of the “correct” size if we assume that the fit is a “good fit”.

How do we know if a fit is "good"?

1. The current fit is an improvement over the previous fit if χ²_nu is "sufficiently" smaller.

2. You should be suspicious of a fit for which N_var is close to N_idp, i.e. a fit for which ν is small.

3. All variable parameters should have values that are physically defensible and error bars that make sense.
4. The results should be consistent with other things you know about the sample.

5. The R-factor should be small and the fit should closely overplot the data. (That was redundant. )

Now what? Well, see Scott's module....

Interpreting error bars

You have done a fit and now have some best-fit values for your parameter and their associated error bars. Now what?

1. The interpretation of an error bar depends on the meaning of the parameter.

2. A fitted σ² value of, say, 0.00567 ± 0.00654 is troubling. That result means suggests that σ² is quite ill-determined for that path and not even positive definite. Yikes!

3. On the other hand, a fitted E₀ value of, say, 0.12 ± 0.34 is just fine. E₀ can be positive or negative. A fitted value consistent with 0 suggests you chose E₀ wisely back in athena.

Say some words about reporting your data and your error bars in press.

Informal use of prior knowledge

1. Because the information content of the XAS measurement is so limited, we are forced to incorporate knowledge from other measurements into our data analysis and its interpretation.

2. Other XAS measurements — for instance, the “chemical transferability” of S₀²

3. Diffraction tells us structure, coordination number, bond lengths, etc.
4. Things like NMR, UV/Vis, and IR can tell us about the ligation environment of the absorber
5. Common sense:

   • near neighbor distance is neither 0.5Å nor 4.0Å

   • σ² cannot be negative

6. ... and anything else your (physical | chemical | biological | whatever) intuition tells you

Formal uses of prior knowledge: constraints

A fixed relation between two or more parameters of the fit.

Constraints can be applied in a way that does not use any additional fitting parameters, thus not changing ν in the evaluation of the fit.

Example: Uranyl with carboxyl and phosphoryl ligands. Constrain numbers of short and long equatorial oxygens to the numbers of C and P atoms.

Formal uses of prior knowledge: restraints

Restraints are additive in quadrature with the χ² defined above. The fit is then evaluated to minimize the sum-in-quadrature of χ² and the restraints. Restraints thus encourage the fit to stay close to your prior knowledge without imposing a strict constraint.

Discuss limits of scaling factor -- the restraint can be tuned from irrelevant to a full constraint. How do you pick the scaling factor ...? (Baysian approaches are beyond the scope of this module.)

Restraints can be applied in a way that does not use any additional fitting parameters, thus not changing ν in the evaluation of the fit.

Example 1: Restraint on S₀² as a looser sort of chemical transferability. Any metal and its oxide will work fine.

Example 2: Bond valence sum implemented as a restraint. FeO works well.

Etc.

• Discussion questions...