Pharma’s Cutting Edge

For this month’s PCE column, I wanted to say something about interim analyses. They’re in the news nearly every day, attesting to their importance, and yet they are not well understood by most people in the industry, the academic community or lay public. Problem is, there’s a lot to say about interim analyses, and I don’t want to short-change the topic. My compromise is to discuss the Data Monitoring Committee (or DMC) this month and save other interim-analysis topics for a future article(s). The primary references are FDA’s March 2006 final guidance on DMCs and EMEA’s (i.e. CHMP) 2005 guideline on DMCs. If you are going to pick only one of these documents to read, I can recommend the EMEA doc for its brevity and the FDA doc for its thoroughness. I would imagine that ICH has some interest in providing a harmonized guideline, but they apparently have higher priority issues to tackle.

DMCs were first used in the 1960s in NIH-sponsored clinical trials. The rationale for the DMC then (when they were primarily referred to as Data and Safety Monitoring Boards, or DSMBs) was the same as it is now: minimize trial operational biases by using sponsor-independent personnel to monitor study integrity and subject safety. But you might have heard the term Independent DMC. Sounds redundant according to the aforementioned rationale, doesn’t it? It’s not. At some point during evolution of the DMC, probably when industry started to sponsor large clinical trials and senior executives began getting cryptic recommendations on their most important investigational drugs from scientific groups without ties to industry, sponsors decided that it probably wasn’t in their interests to let independent DMCs run their pivotal-trial interim analyses. At that time, industry sponsors relaxed the composition of DMCs to include their own employees. Sometimes these employees were behind a “firewall” to minimize study bias. However, it was not unusual for sponsor employees directly involved in trial design, conduct and final analysis to supervise DMC activities, particularly for non-pivotal studies.

Fairly recently, though, the pendulum has swung back towards industry-sponsor independence for DMC members. The reason is simply that investigators in larger trials with “hard” outcomes (e.g. mortality) didn’t look kindly upon industry sponsors as the primary overseers of trial subject safety and study integrity, and neither did regulators. Quite frankly, had industry enjoyed a sterling reputation during the 1990’s, I have little doubt that the pendulum would have stayed where it was. The call for independent DMCs (and independent steering and endpoint committees) is simply another manifestation of the public’s general distrust of anything the industry sponsors. In any case, thus was born again the truly independent DMC, a DMC composed entirely or nearly entirely of non-sponsor personnel. When the sponsor has a member on an independent DMC, he or she usually plays an administrative or oversight role only, without a vote and with limited access to data.

FDA describes factors leading to the increasing use of DMCs today:

• The growing number of industry-sponsored trials with mortality or major morbidity endpoints;
• The increasing collaboration between industry and government in sponsoring major clinical trials, resulting in industry trials performed under the policies of government funding agencies, which often require DMCs;
• Heightened awareness within the scientific community of problems in clinical trial conduct and analysis that might lead to inaccurate and/or biased results, especially when early termination for efficacy is a possibility, and need for approaches to protect against such problems;
• Concerns of IRBs regarding ongoing trial monitoring and patient safety in multicenter trials.

Implicit in the above factors is increasing pressure on industry sponsors to relinquish more of their direct control over clinical-trial design, conduct and analysis. Pressure to use DMCs is just one reflection of this bigger trend. Independent steering committees, publication committees, endpoint committees, etc are others.

There is no U.S. law mandating use of DMCs, except in the emergency-use setting, when informed consent of the participant is excepted [21 CFR 50.24(a)(7)(iv)]. FDA recommends use of a DMC for: “any controlled trial of any size that will compare rates of mortality or major morbidity…DMCs are generally not needed, [however], for trials at early stages of product development.” Let’s just spend a moment on this point. Is FDA not recommending a DMC for most studies because a DMC would not be useful, or because a DMC is not worth the additional effort necessary to implement given its value in early-phase studies? I believe that FDA’s recommendation is based on the value proposition. In other words, this is FDA’s judgment, based on their sense of the value of a DMC to the study sponsor. IRBs, investigators and other regulators might well have differing judgments from FDA. It is crucial for a sponsor to consider these other opinions prior to determining finally whether to implement a DMC, regardless of phase of development. Indeed, phase of development has different meanings for different therapy areas and study designs. [Example: A Phase 2 oncology trial might have a mortality endpoint, whereas a Phase 2 hypertension trial likely will have a blood pressure endpoint.]

FDA goes on to refine the decision drivers for implementing a DMC:

• The study endpoint is such that a highly favorable or unfavorable result, or even a finding of futility, at an interim analysis might ethically require termination of the study before its planned completion;
• There are a priori reasons for a particular safety concern, as, for example, if the procedure for administering the treatment is particularly invasive;
• There is prior information suggesting the possibility of serious toxicity with the study treatment;
• The study is being performed in a potentially fragile population such as children, pregnant women or the very elderly, or other vulnerable populations, such as those who are terminally ill or of diminished mental capacity;
• The study is being performed in a population at elevated risk of death or other serious outcomes, even when the study objective addresses a lesser endpoint;
• The study is large, of long duration, and multi-center.

FDA’s decision framework makes a lot of sense to me. The bottom line to ask yourself as a sponsor: Would a DMC help mitigate risk to subjects, where the potential risk to subjects is potentially only marginally outweighed by potential benefit? Would a DMC help maintain study integrity? If either of these two questions is answered by “Yes” strongly consider using a DMC.

So, who should be on the DMC, and what is the correct number of members? To the latter, keep the number as small as feasible while making sure to have the appropriate expertise on board. FDA says three is a minimum number. I would say that three is generally too few; four provides a margin of safety to cover unforseeable absences. These numbers don’t include personnel necessary to administer all DMC activities (i.e. logistics). The DMC constituency should include: a Chair (who can appoint the other members and is responsible for communications beyond the DMC), one or more statisticians (to perform and/or assist in interpetation of data analyses), clinicians (therapeutic area and/or clinical research experts), and others. Among the “others” to consider, FDA suggests epidemiologists and non-scientists with interest in the study outcome. I consider these reasonable suggestions. Finally, it’s important to assess potential conflicts of interest when deciding whom to appoint to a DMC. Here’s FDA suggestion for doing this:

• Ensure that those with serious conflicts of interest are not included on the DMC;
• Provide disclosure to all DMC members of any potential conflicts that are not thought to impede objectivity and thus would not preclude service on the DMC;
• Identify and disclose any concurrent service of any DMC member on other DMCs of the same, related or competing products.

Once a sponsor determines to form a DMC (and who will administer it–for a large pivotal study strongly consider outsourcing DMC administration), the first step is to create a DMC Charter. The Charter is essentially a contract between the DMC and the sponsor. It describes the DMC mission, membership, and all operating procedures. One of the key parts of this Charter is how data will be shared between the sponsor and the DMC.

FDA describes how this should operate ideally. Basically, the sponsor and its personnel should remain blinded to all comparative data in a blinded trial: “We recommend that any part of the interim report to the DMC that includes comparative effectiveness and safety data presented by study group, whether coded or completely unblinded, be available only to DMC members during the course of the trial, including any follow-up period—that is, until the trial is completed and the blind is broken for the sponsor and investigators.” The way this accomplished easily is by creating a sponsor-indepedent data analysis group (another reason why outsourcing is a good idea). Short of this, create a data-analysis group within the sponsor that isn’t otherwise involved with the study design or conduct.

Having served as a member of an internal data analysis group, I can tell you first-hand that keeping interim analysis findings secret from those who are primarily responsible for study conduct is difficult. Highly anticipated findings, such as those I kept secret, lead some otherwise scrupulous individuals to push the envelope of acceptable inquiry. Even well-intentioned co-workers will attempt to read your body language and will communicate their impressions to personnel on the operations team. My recommendation then is to avoid internal interim data analysis groups whenever possible. When an internal data analysis group appears to be the only feasible solution, isolate individuals on the group as much as possible from those charged with conducting the ongoing study and also those who are planning other activities that depend on the outcomes of the study undergoing interim analysis. Document in writing the steps taken to create and maintain an intact firewall. If any breaches of the firewall occur, document this as well.

Exactly how a DMC should receive study data is another area of controvery. Specifically, some people think that DMCs are better off reviewing data grouped by therapy but blinded to therapy identifier. Most people think safety data should be fully unblinded. FDA believes the best approach is to allow DMCs to review all data unblinded to therapy. I agree. There’s little reason to hamstring a DMC by partial uinblinding as it attempts to fully interpret the benefit-risk balance, particularly when care is taken to prevent study operations from being affected, by isolating the DMC and data anlysis group from study operations and by taking other reasonable measures to preserve data secrecy.

A DMC can be charged with a host of responsibilities that vary from simply monitoring subject safety and overall benefit:risk, to monitoring trial conduct (study integrity) and efficacy and communicating evidence of unexpected efficacy or futility, either of which can result in early study termination. Usually, the DMC has a range of communications it may make to the sponsor (ideally the wording of common communications from DMC to sponsor should be anticipated, agreed to by both parties, and described in the DMC charter) as well as a range of actions it can recommend, short of continuing the study as is or terminating it early. As FDA describes, such DMC recommendations for action can include:

• Changing the eligibility criteria if the risks of the intervention seem to be concentrated in a particular subgroup.
• Altering the product dosage and/or schedule if the adverse events observed appear likely to be reduced by such changes.
• Instituting screening procedures that could identify those at increased risk of a particular adverse event.
• Informing current and future study participants of newly identified risks via changes in the consent form and, in some cases, obtaining reconsent of current participants to continued study participation.

Most commonly, though, DMCs recommend no action be taken, that is, that the study continue as planned. Least commonly, DMCs will recommend that a study arm or an entire study be terminated early. Of intermediate frequency between these extremes is one or more of the recommendations listed above or something similar.

As I mentioned, DMC recommendations are highly scrutinized by interested parties. Besides the sponsor and trial subjects, perhaps the most interested indviduals are investors. DMC members must always be aware of the importance of their findings to all interested parties, without letting this interest influence their recommendations. It’s not as easy as it might seem. For instance, the outcome of a DMC meeting might be highly material to the financial condition of even a very large drug company. U.S. laws prohibit selected disclosure of material information that is known only to company insiders. Securities attornies and the U.S. regulator of securities (SEC) believe that DMC members (including personnel privy to their deliberations or findings or data only the DMC sees) are company insiders. Individuals serving on a DMC can therefore be held liable for advertent or inadvertent disclosure of interim analyses or the findings or recommendations of the DMC beyond the communications channel defined by the DMC Charter. In other words, DMC participants must be very careful to maintain secrecy. Sponsors have an ethical responsibility (if not a legal one) to warn DMC participants of their liability in this regard.

Finally, FDA takes pains to describe ways in which sponsors may reduce the risk of using unblinded interim analyses to plan future studies, although they advise against the practice. I can’t overstate my agreement with this cautious advice. Why would a sponsor use unblinded interim analyses of one study to design another, as opposed to, say, relying only on pooled baseline data to improve decision-making accuracy? Probably the most frequent reasons are to define an event rate by treatment group (as opposed to an overall event rate) and to define a treatment effect. Obviously, a treatment effect can’t be inferred from pooled baseline data. But are these “benefits” worth the risks?

Let’s put aside the risks to an ongoing study of unveiling interim results from the study to the sponsor, which we’ve mentioned earlier, and which are very real and very important. Instead, let’s just consider the risk of using data from an interim analysis to draw conculsions that will drive future study design. This risk will depend primarily on the timing of the interim analysis (i.e. the “N”), the background event rate, and the benefit of the intervention. In other words, exactly the same factors used to determine the statistical power of the study to infer a significant treatment effect. Remember, by definition, a study is typically not powered to detect a clinically significant treatment effect during an early interim analysis. Therefore, if the sponsor uses interim-analysis information to plan for a future study it risks using faulty inferences as its basis for judgment (due to inflated Type II error in particular).

In some situations, even an initially faulty inference may be “cured” by, for example, using an adaptive trial design in the planned study. In other situations, no such cure is feasible. So, to the list of mitigating factors FDA suggests (listed below) to limit a sponsor’s risk of using an unblinded interim analysis to plan for a future study, I would strongly recommend that sponsors: 1. Wait until as long after initiation of the “training” study as possible; and 2. Include in the design of the future study a mechanism for mitigating risk and cost of failure caused by the use of faulty planning data.

FDA’s advice for limiting risks of using an unblinded interim analysis to design a future study:

• Discussion of such an action with FDA in advance. This is particularly advisable when the sponsor intends to use the study in support of a licensing or marketing application.
• Development of appropriate stopping rules and apportionment of type I error (before performing any unblinded interim analysis. This is important because any viewing of study arm-specific effectiveness data by the DMC and/or sponsor in a study of a serious illness raises the possibility that an unanticipated extreme finding of effectiveness might create an ethical imperative to stop the trial, and it would not be possible to quantitate the level of evidence provided by the data if the monitoring plan had not been established prior to data review.
• Determination of the minimum amount of information needed. For example, to assist in defining eligibility criteria for a subsequent trial, the sponsor may wish to know only whether estimates of treatment effect in a subgroup are less or greater than in the overall data set.
• Formulation of written questions, preferably with yes/no rather than numerical answers, that will elicit only that minimal required information and nothing more.
• Receiving only written information regarding the requested data (thereby documenting what was received and avoiding additional unnecessary communications) and abstaining from participation in closed DMC meetings or discussions of data with unblinded DMC members (except as otherwise requested by the DMC).
• Identification of those sponsor employees with a critical “need-to-know” and restriction of such information to those individuals only.
• Ensuring that individuals with access to the information avoid any subsequent role in the management of the trial and minimize interactions with others in that role.
• Ensuring that individuals who have access to such information make every effort to avoid taking actions that will assist others in inferring what the information is.
• Ensuring that reports of study findings describe any access to interim data by individuals involved with study management, and steps taken to prevent such access from potentially biasing the study results.

August 8th, 2006 | Category: Biostatistics, Clinical Research, EMEA, FDA, R&D | Subscribe to comments | Leave a comment | Trackback URL