|
This is a sample of the kinds of skeptical questions that help to
clarify issues and make the scientific aspects of the GCP more
robust.
Date: Fri, 22 Dec 2000 18:26:18 -0500 (EST)
From: rdnelson
Questions from a professor of statistics.
> My primary research area is uncertainty in artificial intelligence,
> and I have a strong interest in the cognitive sciences. Recently I
> have become very interested in issues of global consciousness,
> perhaps catalyzed by the emerging Internet, and also in the question
> of the possibility of engineered conscious artifacts. I have been
> trying to assess the state of the science in this area. For this
> reason, I was quite excited when someone recently pointed me to the
> GCP website http://teilhard.global-mind.org/.
Thanks for your note and more generally for your interest in
the issues of global consciousness. I will address some of
your specific remarks in context below, but I'd like to say
that I deeply appreciate your thoughtful and completely
constructive skepticism -- and the implicit suggestions for
how to do better what I am doing; how to make a serious
effort more productive. I also have a request. I would
like to make use of your comments on the GCP website, where
there are some discussions of statistical and scientific
issues. So I'd like your permission to include attribution
if that is OK with you. I would include your email address
or not according to your wishes.
> As I'm sure you appreciate, sorting out what we really know from hype
> and over-enthusiastic credulity can be a daunting task.
> Nevertheless, it's an extremely important task for us to attempt at
> this juncture in history. Our fiction writers over the past
> generation or so have alerted us to the potential consequences that
> could await us if a global consciousness or conscious artifacts are
> indeed emerging, and if we as a species botch badly our
> responsibility as midwife and/or parent.
>
> As a scientist and a skeptic, it seems clear to me that the jury
> remains out on whether any new form of consciousness is indeed
> emerging, and if so what the time frame is on its emergence. As
> someone trained in public policy, I cannot over-stress the importance
> of watching what is happening, looking for key indicators, and
> planning for various contingencies of things that could go wrong.
> Your project could be very useful in that regard. I'm glad I have
> become aware of it.
We certainly do need to stay tuned to signs of change in
global consciousness however defined. There is little
question of changes happening, but I think there is lots we
don't know about the sources, the directions and
implications and timing, and there is (I believe) lots we
might do in a more conscious way to participate in the
changes. It is a really big scale we are talking about, so
I am not sanguine about "controlling" or even directing it
much, but I think we are in a position to tweak and to make
suggestions that may be influential. I hope the GCP might
be useful, though I confess I have thought of it more as a
possible source of inspiration than of information which
might be useful in contingency planning.
>
> I browsed the site. I haven't had the time to study it in detail,
> but I saw some things that looked quite interesting. I found,
> though, that the graphs you displayed, for example in the "Princess
> Diana" results, were hard to interpret. It wasn't clear to me what
> exactly was being plotted. Perhaps you could include a hyperlink to
> a page giving a clear interpretation of what the plot means.
I do have some material like that, but it is not linked to
the individual presentations. Thank you for the suggestion.
The Diana graph is in an older style which is slightly more
complex, visually, than the ones I use for displaying current
results for the GCP. It shows the locus of expectation as a
line defined by the degrees of freedom for Chisquare, and
another that shows the amount of deviation required for
nominal "significance." Since that time I've adopted a
simpler graphic presentation which shows the cumulating
deviation of (Chisquare - df) from its expectation of zero.
In any case, I will look for ways to explain better what the
graphs are showing.
>
> Who are the statisticians associated with this project? Have you had
> your procedures and analysis methods analyzed and critiqued by
> statisticians (particularly ones with a skeptical mind-set) who can
> help you identify weaknesses? I'm sure you are aware of how easy
> it is to find spurious effects in data sets, especially for
> over-enthusiastic researchers lacking extremely strong background
> in statistics.
There are several people with sophisticated statistical
understanding associated with the project, and I solicited
comments widely, early in the design phase. Only one is a
professional statistician, Prof. Jessica Utts from UC Davis.
Most of the skeptical commentary has come from within the
group of colleagues who wish the GCP well. For example, Ed
May and James Spottiswoode, both physicists but also good
statisticians, have been quite skeptical, especially of my
"event-based" prediction and testing approach. James, for
example, says he might be persuaded if there is a grand
overall non-chance intercorrelation of the eggs, but he
says the event-based analysis does not impress him. We agree to
disagree. We do also have in the works two different approaches
to the intercorrelation question. One has some tentative
results, but has independence issues which I am not sure we
have dealt with adequately. The other is fallow until my
colleague York Dobyns (another physicist with highly
sophisticated statistical understanding) and I hear back
from a very busy James Spottiswoode.
>
> I personally am NOT a fan of frequentist "significance tests" of the
> variety you appear to be relying on. Standard significance tests are
> based on a theoretical premise which is typically not satisfied in
> practice and is clearly violated in this case -- that our purpose is
> to run exactly one experiment to test one specific hypothesis. In
> this case, it appears that you have done a multiplicity of tests,
> yet> are treating each test as a separate item independent of the
> others.
There is an issue of the level or meta-level of sampling.
If I understand you correctly, you are objecting to having
repeated event-based predictions, saying they are not
independent. I would like to understand why you think they
are not independent. I see the individual events in the
formal prediction list as independent samples from an
indefinitely large population of unique moments in time.
I do try to learn what sort of predictions to make, and in
that sense the array has some interdependence (this would be
the case even if I didn't try to learn how to do the
predictions well). However, the data are random, and
under any reasonable interpretation I can envision,
there can be no dependence of selected subsequences of the
random data stream on each other, no matter how much learning
or imagining I do. As far as I can see, there is no linkage
at all among the 60 formal events except their membership in
the arbitrary category we have described as a global event.
> Specifically, consider the case of Princess Diana's funeral. Your
> analysis found an effect of p < .01, which means less than a 1%
> chance of finding an effect this extreme by chance. What is not
> noted, but is clearly relevant in this case, is that this example
> appears to be one of a small number of significant results out of a
> large number of experiments that were run. In several hundred
No, there was only one experiment run in the context of our
prediction that the REG data would show departures from
expectation during the funeral ceremonies. Can you explain
why it appears to you that there was a "large number of
experiments run"? To be sure we had already the background
of the "Fieldreg" experiments referenced in the Diana
article, but the Diana experiment was only the second
"global" experiment of this sort. Further, it is presented
together with a repeated application of the paradigm to
Mother Teresa's funeral the following week, in which the results
were not significant. Two or three such "samples" from the
universe of possible outcomes is not adequate for any sort
of conclusions, even though the Diana results looked exciting
and had a "face validity" that obviously could not be justified
on statistical grounds. The GCP is a way to gather a reasonable
sample, now with an N of 61 and growing, to ask the question
in a responsible way whether there is anything to this notion
of global consciousness (or consciousness field, or whatever
convenient name to give such an enigmatic possibility.)
> experiments of this kind one would expect to see a few effects with
> p < .01. Hence, one could easily argue that the few effects you did
> find were artifacts. There are more sophisticated statistical
> procedures, some frequentist and some Bayesian, that control for this
> "fishing expedition" effect. Have you applied these approaches?
> If not, I would strongly encourage you to seek out top-notch
> statisticians to consult on your data collection and analysis
> procedures, to make certain that what you think you are discovering
> is more than just spurious correlations occurring as artifacts of
> flawed statistical analyses.
I don't accept the "fishing expedition" description. That
implies poking around in the data until you find something
interesting, and we specifically do not do that in the
formal prediction sequence. There is for each event a
single prediction made, in explicit terms that do not allow
any selection based on seeing results. The analysis for that
prediction is always used, i.e., no cases are discarded, so
the formal dataset is, I believe, quite sound as a sample
from the population we have defined. The primary "control"
we use is resampling. In cases where there is high interest
or any concern there could be something wrong, I have created
a resampling distribution by random selection of
appropriate-size data segments from the data stream
around the active segment. For example, the detailed
presentation of the Y2K results describes several instances
where resampling was used. Your comments urge me to proceed
with the intention (already in the original design) to do
that for all events as a matter of course. I need a longer
day, but the invitation is out for help on this and any
number of other projects.
Please tell me more about your ideas for alternatives that
could be implemented to help strengthen or clarify the
statistical case. Also, though I have the impression
that you have plenty to do, please let me know if
I can be helpful in any way should you choose to download
some data and have at it.
>
> Again, I remain very interested in your work, and will be watching
> your site with interest over the coming months. I hope you find my
> suggestions useful.
Wonderfully so. Thank you again. If any of my responses
are unclear or wrongheaded, please let me know. I value
your comments.
Best wishes for the Holiday season, and for the New Year!
Roger
|