THE NCI BEST CASE SERIES PROCESS

| Conference Home Page | Presenter Bios | CMBM Home Page |

Presenters: Jeffrey White, M.D., Mary McCabe, R.N., and Michael Hawkins, M.D.
Moderator: Jeffrey White, M.D.
Session: F2; June 11, 1999

I. Abstract

The National Cancer Institute has created a process by which Complementary and Alternative Medicine (CAM) practitioners who treat cancer can have their therapeutic approach evaluated for further study. In this Best Case Series (BCS) process, practitioners apply to the Office of Cancer Complementary and Alternative Medicine (OCCAM) within the NCI. If selected for a Response Audit, they present five to ten of the best cases that demonstrate the efficacy of their treatment strategy. The Cancer Advisory Panel for Complementary and Alternative Medicine (CAP-CAM) then decides whether to recommend further controlled investigations. The session presented the developmental history of this process focused on the pitfalls that have caused trouble for past applicants and discussed its present status. Finally, a physician described his experience with a Best Case Series process established in Germany.

II. The Cancer-Related Issue Addressed

The purpose of this session was to educate health practitioners about how they can submit a promising cancer therapy/treatment regimen/modality to the National Cancer Institute (NCI). for further study. The NCI has created submission guidelines for a "Best Case Series" that aims to prevent many of the pitfalls they have previously encountered.

III. The Program

Historical Background Presented by Michael Hawkins, M.D.

When Dr. Hawkins was the Director of the Investigational Drug Branch at the NCI, the office handled many investigational drug applications. Data regarding alternative therapies would be reviewed by a number of individuals throughout the institute, depending on their schedules. There was no generalized approach or rationale to review these data. Dr. Hawkins, along with Dr. Sam Broder and Dr. Bob Wittes, decided to set up a mechanism by which proponents of alternative therapeutic approaches could have their approaches reviewed by the NCI. Via this mechanism, practitioners present their data to the NCI. The NCI determines which alternative approaches should be advanced to prospective clinical trials. Since most of the practitioners of alternative approaches to healing lack scientific data to back their particular therapeutic methodology, the NCI decided that provocative anecdotal clinical experiences could form a basis for pursuing these approaches. The result was the creation of the "NCI Best Case Series".

Details

As described above, the purpose of the Best Case Series (BCS) is to allow proponents of complementary and alternative medicine (CAM) treatments of cancer to present data on their treatment regimen. Because most CAM treatments lack scientific data, the practitioners are asked to present 5 to 10 of their best clinical cases that will most clearly demonstrate their treatment’s efficacy. Based on these admittedly anecdotal data, the NCI decides whether to further pursue these approaches to cancer treatment with prospective, controlled clinical trials. This BCS review mechanism has been in place since 1991.

Since the BCS was created, several common problems have been recognized. In response, a set of guidelines for the NCI Best Case Series was devised to help proponents avoid many of the pitfalls that were commonly seen in applications submitted by alternative practitioners. These pitfalls include:

  1. The diagnosis of cancer was never documented by the alternative practitioner.
  2. After surgical removal of the primary tumor, patients received alternative therapy as an adjuvant treatment. In many cases, it was not clear if there was any residual disease remaining to be treated. This made the results difficult to interpret.
  3. All cancers have a variable natural history. The graph of patients’ survival is always a bell-shaped curve. This curve has a tail that represents long term survivors.
  4. Concurrent conventional medical treatment was administered making it difficult to determine whether the or alternative therapy that resulted in therapeutic effect.
  5. Treatments often are not well described and documented. Especially problematic with herbal and "pharmaceutical" approaches. Manufacturing process may be extremely variable and it is difficult to document what is in each preparation of the products.
  6. Effects of treatment on all cancer sites in the patients. Sometimes one mass will respond while the patient is dying from the disease elsewhere.

Dr. Hawkins closed with a few last points. Sometimes, practitioners will find that the appropriate data was not collected for previously treated patients. In these cases, Dr. Hawkins recommended that proponents of the therapy prospectively collect the data on the next 20 patients seen by the practitioner. Another possibility is to conduct a small randomized trial. The trial would probably be decidedly under-powered for statistical significance. Nonetheless, it may provide the trend to help determined whether further research is warranted. In either of these scenarios, Dr. Hawkins suggested that the practitioner get in touch with Mary McCabe at the NCI to determine unequivocally the data that are required for analysis.

Past Experience with Best Case Series Presented by Mary McCabe, R.N.

Mary McCabe, Director of the Office of Clinical Promotion, spoke about the NCI’s past experience with the BCS. She views the BCS document (which was authored by Dr. Hawkins) as 1) A template for data collection, 2) a vehicle for discussion with the investigators in the CAM community and 3) a vehicle for analysis. This document has formalized the process by which practitioners can have their data/therapy evaluated.

Ms. McCabe then briefly described how submissions for the BCS normally proceeded in the past. First, they receive information from a practitioner regarding their treatment Usually, the NCI would inform the practitioner that additional information was required. In some cases the NCI would receive this additional information. In other cases the request would prove too much of a burden and the NCI would not hear back. Once they had received sufficient information, it went to reviewers (usually internal) who would look at the data from either a disease perspective (e.g. breast cancer) or an intervention perspective (e.g., imagery). These reviewers then made a determination within the program as to whether the data was of interest. If so, it would move on to a Response Audit which involved either the NCI going to the practitioner to look at the cases, films, tests, etc. or having the individual come to the NCI to present their cases. Based on the Response Audit they would make recommendations to the NCI regarding clinical trials and/or to the practitioner/investigator regarding a prospective evaluation.

Of about 25-30 submissions over a four year period, about 10 contained sufficient data that necessitated further investigation. Of these 10, five Response Audits were done in a very formal, time-intensive manner. Ms. McCabe then briefly reviewed the timelines for some of the Response Audits. From the five Response Audits, two of the interventions went to NCI supported clinical trials, for two the NCI made recommendations and the proponents developed their own funding. For another, Dr. Jeffrey White is still working with the practitioners for two of the interventions (Shark cartilage and Kelley Regimen.) (These actually add up to six).

The NCI spent a significant amount of time assisting investigators/practitioners. They explained how to put the data together and facilitated partnerships with experts around the country who could help them put Case Reports together.

Based on their experience, Ms. McCabe listed the most common problems that were encountered with the BCS submissions. Some of these overlap with Dr. Hawkins list above.

  1. Diagnosis not documented.
  2. Use of subjective endpoints (e.g., patient feels better, increased vitality).
  3. Very short follow up time, sometimes only a couple of weeks.
  4. Concurrent and previous standard chemotherapy.
  5. Missing data because these are retrospective (most common problem).

Because they were running up against the same issues, they developed a simple outline of what would need to be collected. What they want to know about are 1) the patient, 2) the intervention, and 3) the outcome. Many times they would not get clinical data but just descriptive summaries. The detailed outline (as well as the BCS document) can be acquired from Dr. Jeffrey White’s office at the NCI (see below).

In addition, case report forms will need to developed for the individual intervention. These include: 1) an eligibility form, 2) an on-study form (patient’s baseline, history), 3) a treatment summary, 4) a response evaluation form, and 5) a follow up form for long term evaluation. These are not complicated and can be generated quite simply.

In conclusion, Ms. McCabe believes that although it has not been a completely smooth process, much has been learned and the future of this process is bright.

The Present Form of the Best Case Series Presented by Jeffrey White, M.D.

Dr. Jeffrey White spoke about the future of the BCS process. Dr. White is the Director of a new office at the NCI called the Office of Cancer Complementary and Alternative Medicine (OCCAM). This office was created within NCI to deal with issues regarding CAM. The Best Case Series is a major focus. In addition, Dr. White is the liaison from the NCI to the National Center for Complementary and Alternative Medicine (NCCAM). He helped develop the prospective agenda for the NCI and helped interface with the public regarding CAM. Anyone interested in having a BCS evaluated, should get in touch with Dr. White’s office. The OCCAM has all of the necessary materials and is available to help.

The Best Case Series today is essentially as Ms. McCabe described. (It is important to realize that the proponents of the interventions initiate this process that the NCI is interested in doing BCS.) Once contacted by someone about the BCS, the office sends out a packet containing a booklet and two forms. These materials give a description of the BCS and the data collection that will be required. The proponent of the intervention then collects required information. The documentation is reviewed for completeness and the OCCAM works with the proponent until completeness is achieved. At this point, the radiology and pathology is reviewed at the NIH clinical center. These materials are summarized and presented to the Cancer Advisory Panel for Complementary and Alternative Medicine (CAP-CAM).

CAP-CAM was recently established and chartered this spring. Its role is to advise and make recommendations to the directors of the NCCAM and NIH regarding future clinical trials. They review and assess summaries of evidence for CAM cancer interventions; clinical trials submitted by practitioners. They decide whether and how these interventions should be followed up. They develop a means to communicate the study results to practitioners and, when appropriate, the public. They are also to identify future CAM cancer clinical trials initiatives. It is a 15 member panel. The chairman is Dr. Ernst Wynder (American Health Foundation) and the co-chairman is Dr. Michael Hawkins. It is comprised of oncologists, oncology nurses, CAM specialists, statisticians from the FDA, epidemiologists, and a patient advocate.

The panel will meet two to three times per year. Practitioners will be invited to present the material along with a radiologist and pathologist from the clinical center. The practitioner will then answer questions. The panel then returns its assessment of the BCS and its recommendations for further research, if there are any.

Possible outcomes of the BCS include:

  1. Observational studies – the NCI goes to the practitioner’s office to gather more complete information than the BCS could capture.
  2. Early phase clinical trial – e.g., if dosing was unclear.
  3. Prospective clinical trials – i.e., Phase III.
  4. Basic research into mechanisms of action and potential interactions.

Dr. White concluded by inviting anyone interested in presenting a BCS to CAP-CAM contact his office (see section V).

The German Experience with the Best Case Series Presented by Gerwin Kaiser, M.D.

Dr. Gerwin Kaiser is a medical oncologist in the Department of Internal Medicine, Oncology, Hematology, and Infectious Diseases at the 5th Medical Clinic in Nurenburg, Germany. The Biologic Cancer Therapy Working group in Nurenburg has reported their results from well over 100 Best Case Series from CAM practitioners.

Dr. Kaiser defined Unconventional Methods as those that make claims of efficacy without sound evidence. What is important is not the method itself but the use of the method. He defined Conventional Methods as those that have been proven to have efficacy. Experimental Methods are defined as those for which there is hope that they will be efficacious but this efficacy is unproved. Note that these definitions differ from those that would be used in the US regarding conventional medicine and CAM.

Dr. Kaiser briefly related the process by which they would decide which unconventional treatments to investigate. One important difference between their methodology and the NCI’s is that they initiate investigations, usually of those interventions that had received significant press coverage. This contrasts with the administration of the NCI BCS where the proponent of the intervention always initiates the BCS.

Dr. Kaiser summarized their experience by saying that, of the 134 cases that they reviewed, the practitioners claimed 63 constituted complete remissions while they were unable to document a single complete remission. The reasons for this disparity were several. They included:

  1. No tumor specimen.
  2. Insufficient case records.
  3. Parameters non-evaluable/-measurable.
  4. Non-established diagnostic procedures.
  5. No evidence of disease (adjuvant treatment).
  6. Incorrect use of diagnostic criteria.
  7. Concurrent treatment with standard therapy.

He concluded that these results were very disappointing and he doubted that any viable treatments could come from the unconventional realm.

IV. Limitations and Difficulties

For the difficulties associated with this process in the past, see above.

V. Resources

Those who are interested in having the NCI evaluate a BCS of their intervention should contact Dr. Jeffrey White. His office will send out a packet of materials that will guide the practitioner through the BCS process. Dr. White can be contacted at:

Executive Plaza North
Suite 102
National Institutes of Health
Bethesda, MD 20892

Phone: (301) 435-7980

Audience Questions

Is there a good way to measure quality of life or extension of life?

McCabe: In it’s cleanest form, the Best Case Series is looking for tumor regression. However, in the Kelley Regimen sometimes they do not see regression. In this case they did a small prospective study to measure life extension in the absence of regression.

Hawkins: The NCI will not ignore the obvious. The point of the BCS is to find the "gee whiz" cases whether the outcome be tumor regression or life extension. These cases would function as a basis to do further studies.

What are the ethics involved with doing a randomized trial in which the experimental group does not receive the standard treatment?

Hawkins: This is best avoided by doing studies in those cancers that do not have a standard treatment. Pancreatic cancer is one such cancer. Pancreatic cancer is also a good one to use because the life expectancy is very short, approximately 6 months. By using pancreatic cancer to test one’s therapy, one will get quick and possibly dramatic results regarding efficacy.

Does the NCI approach individuals regarding their treatments?

White: Not really. In fact, this may be an explanation for the differing success rates between the German and US BCS processes.

Comment: One audience member protested the fact that just because someone has previously taken a standard therapy it should not disqualify him or her from these studies. He believed that disqualifying them for these reasons was hypocritical because the reverse is not done. That is, patients in studies for conventional treatments are not asked what CAM therapies they have used. What is to say that the CAM therapies did not produce any positive effect?

| Conference Home Page | Presenter Bios | CMBM Home Page |