Effect Size per Hour: A Natural Unit for Interpreting Anomalies Experiments |
Roger D. Nelson Princeton Engineering Anomalies Research, Princeton University, Princeton, NJ 08544 Technical Note PEAR 94003 Comparing effects in different experiments is not generally possible because the effects are measured on differing scales. Yet, especially in anomalies research, such comparisons are important for both theoretical and practical purposes. This exploration seeks to identify a study unit that can render results from various types of anomalies experiments in a common scale. The units in which experiments are posed range across digital and analog measures recorded in a wide variety of defined trials and series, and apparently fundamental units such as bit rates yield disparate calculated effect sizes and potentially misleading inter-experiment comparisons. Across several databases generated in the consistent environment of the Princeton Engineering Anomalies Research (PEAR) laboratory, the number of series and the number of hours are the most promising of several measures considered, but the series definitions are somewhat arbitrary, depending as they do on operator convenience and statistical power considerations. In contrast, the amount of time during which participants attempt to produce anomalous effects can be consistently defined, and the effect size E(h) = Z/sqrt(hours) is demonstrably similar across a number of human/machine experiments, with a magnitude of about 0.2. Application to a broad range of experiments, including examples from other laboratories, confirms the viability and utility of a time-based effect size calculation. A Chi-squared test across 12 local and remote databases from PEAR's human/machine experiments indicates strong homogeneity, while addition of the remote perception database, which has a significantly larger effect size (E(h) = 0.6), immediately renders the distribution of effect sizes heterogeneous. These and other applications yield reasonable and instructive results that recommend the time-based measure as a natural unit for cross-experiment comparisons permitting an integrated view of anomalies research results. To see a graphic display that shows the striking similarity of effects across the 12 different human/machine experiments, as well as direct comparisons with results in a well-controlled remote perception experiment (PRP), a "ganzfeld" experiment (CH), and an REG experiment with selected subjects at another lab (HS), click here. |