PROJECT EVALUATION
CIA-STARGATE
PDF Scan: PDF
Open AI Summary
Summary:
This document is a memorandum from Edwin C. May, Ph.D., regarding the evaluation of the Stargate program. May states that he has been asked to identify the 10 primary studies or reports that establish the validity of the paranormal phenomenon known as Remote Viewing. However, he raises concerns about the "file drawer" problem, which refers to the selective publication of statistically significant studies. May suggests an alternative approach to address this problem by referencing a meta-analysis conducted by SRI in 1989. The document includes a cover sheet with the necessary information for transmission.
Text
Body: SAIC-(415) 322-7960-Created: Wednesday, June 14,199522:38- Page 1 of 2 ---------------
------------- Appr4wed-Release-?{I0-3f@9M 6 rt:fA=FZDP56-'0D7911R660200190041-2
The Cognitive Sciences Laboratory
330 Cowper Street, Suite 200, Palo Alto, CA 94301
Voice: 415.327.2007 - Fax 415.322.7960
~.
An Employee-Owned Company e-mail: may@hildegard.saicmp.com
pQemorandum
SG1I
SG1I
Date: 14 June 1995
To:
From: Edwin C. May, Ph.D.
Re: Project Evaluation
o
1 axe
evaluation. I am asked to find "the 10 primary studies or reports developed under the
subject program ...that.make the best case for establishing the validity of the paranormal
phenomena known as Remote Viewing." I am happy to do this as soon as possible.
I have a concern, however, about a problem that may be raised that could call any results
into question.
Dr. Robert Rosenthal in the Psychology Department at Harvard and others have defined a
statistical circumstance known as the "file drawer" problem. It is standard practice in
behavioral sciences (including the study of anomalous cognition-remote viewing) to use
a p=0.05 criteria as a measure of statistical significance. That is, statistical significance is
claimed when, given that the null hypothesis (i.e. no RV in this case) is TRUE, there is a
5% chance that a repeated experiment of the same number of trials would yield a
statistical deviation as large as in the original, or larger. We use that criterion in our
laboratory.
The file draw problem is this: Under the null hypothesis 5% of the studies will be
statistically significant. If only the significant studies are published (or in our case given
to AIR), it is problematical to assess the validity of the research, because it might be that
for every study published there are 19 others (in the file drawer) that did not reach
statistical significance and were not published. If this were the case, there would be no
evidence for an anomaly. Rosenthal and others provide ways of assessing the potential
magnitude of this problem.
It might be argued that if I am allowed to pick the best 10 studies, and if it turns out that
there were 200 others that were not "good," one could be seriously mislead. This is a
worst case scenario; however, critics would correctly argue that there is an undetermined
file drawer problem with this new approach, and if it is not addressed, the validity of the
evaluation can be questioned. I can assure you that Ray Hyman will raise this point.
Might I suggest an alternative that address the concerns of the time constraint, yet avoids
the file drawer problem. In 1989, SRI published (now downgraded to unclassified) a
meta-analysis of all the SRI work from 1973 through 1989. This study was part of the
me new tasking with regard to providing support for the project
P. d t
Page 1
Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2
SG1I
ApprovFax I ransmlttalRCover'Sneet041-2
To: - ORD
From: Edwin C. May, Ph.D., SAIC
Fax Phone Number: (415) 322-7960
Date: Wed, Jun 14, 1995 - 22:38
Transmitting (3) pages, Including cover sheet.
If there Is difficulty with this transmission, please call: (415) 322-7960
Note:
Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2