Overcoming CGT Potency Assay Woes: 4 Best Practices

By Anna Rose Welch, Editorial & Community Director, Advancing RNA

As children, we imagined all manner of frightful things living in our closets, under our beds, or in that creepy alley behind our garage. On the off-chance any of us actually found ourselves living in an episode of The X-Files, I guarantee none of the fearful beings we imagined as children looked anything remotely like an assay or “matrix” of assays. Now, as we wake to our grown-up lives in the ATMP space, it has come to pass that demonstrating our products’ potency can be one of the more “nightmarish” tasks before us.

A few months ago, ARM and ASGCT hosted a refreshingly informal discussion on potency assays between industry members and regulators. After listening to the many questions, concerns, and opinions on demonstrating potency today, I put together part one of this two-part article in which I explore a few factors behind why this analytical exercise can be so fearsome for companies today. If you missed the livestreamed discussion, ARM and ASGCT have released this fact sheet capturing highlights from this discussion. They’re also working on a forthcoming whitepaper that will share all the nitty-gritty details in full.

Though my articles have advised against recreating the wheel more times than I can count, I’d also argue that there is no such thing as “too many” best practices when it comes to handling a daunting industry challenge. So, at the risk of recreating ARM/ASGCT’s forthcoming wheel, I wanted to share a few of the actionable takeaways I walked away with from this working session in the hopes they will be a complementary exploration of the analytical challenge impacting every ATMP company.

1. Reframe The Assay Matrix As A Tool For Long-Term Flexibility

As a very inflexible ballerina, I am better suited than most to understand that the distance between a good stretch and physical therapy-inducing self-destruction is typically microscopic. So, I acknowledge this mindset will be challenging for companies in the throes of extensive and expensive analytical development today.

We may be aligned around the importance of thoroughly understanding our molecules. But we also know we can’t perform and add assays in perpetuity, just for the sake of measuring what we know we can measure that is maybe important. The question then becomes how big should an assay matrix be, and does the matrix need to stay that big forever? In fact, the complexity of MOA and the biological cascade are the commonly cited reasons behind the agency recommending the matrix approach. As I explained in part one, we hope that as we get better at isolating which step(s) within the biological cascade is/are most critical to the potency of our product, we may be able to align around fewer or maybe even a single validated potency assay.

Regardless of the many degrees of “murkiness” through which we’re navigating today, it’s typically true that our ability to ascertain relevance/importance in any educational effort is proportional to the duration and depth of our inquiry. As many of us are still relatively “young” in our understandings of MOA, our earlier-stage analytical undertakings — no matter how plentiful — might not necessarily demonstrate quality or control. (e.g., They are often marked by variability or are not quantitative.)

That’s why I like to think of the ongoing efforts of establishing a matrix as an exercise in flexibility. As my incredibly patient ballet teacher tells me after my many demonstrations of poor balance and form, each attempt is “money in the bank” toward achieving the necessary muscle memory. In the same way, the more we undertake analytically at the start, the more we will learn about our products overtime. Doing so also will lead to us to make our future analytical decisions more reflexively. As one FDA regulator pointed out, a company’s “shots on goal” are inevitably going to get better over time. In other words, we can “characterize” the potency assay matrix as a “flexible” tool that — like our knowledge/thought process — has the potential to and likely will change shape over time.  

2. Don’t Sell Your Science and Thought Processes Short

If the industry and agency relationship had a Facebook status, I dare say it would be “it’s complicated.” On the one hand, companies and the agency are leaning in and learning together, which grants an informal air of camaraderie and collaboration to this important partnership. However, this relationship is still underscored by an occasionally frustrating — but necessary — imbalance of power. After all, the FDA ultimately holds the keys to our programs’ progress, and it signs off on the decisions we make. In turn, disagreements inevitably arise, miscommunications occur, and/or communication doesn’t happen like it should (both in terms of frequency and detail).

Though checks and balances are essential, too many checks — or, rather, the expectations thereof — can lead to anxiety. In the CGT space today, there is admittedly some of this running in parallel to our respect for the FDA as a partner. Our anxiety takes several different shapes, whether it be shying away from carrying out certain assays for fear we’ll be locked into them; glossing over assay variability; or simply reiterating (predominantly theoretical) concerns that our potency matrix will need to be as elaborate as the labyrinth at Versailles.

Now, to be fair, there have been some big upheavals in the past few years linked to potency assays/CMC decisions that encourage this cross-industry inherited PTSD. Communication with the FDA has also been marred with inefficiency — a fact both industry and agency alike acknowledge. Regardless, we’re faced with an uncomfortable truth: Until we have more clinical data and/or the totality of our data tells us and the FDA we can demonstrate potency measuring only one “step” of the biological cascade, we will be faced with more analytical work and greater risk in the near-term than may be necessary long-term.

But there were a few points raised during this working session that remind us of the power we as companies hold in this regulatory relationship. The agency may have the final say on whether we can progress to the next milestone. But we are the only ones in possession of all the scientific and process-related details we and the agency need to move forward in tandem. (Look no farther than the one FDA representative who mentioned they are the primary CMC reviewer for 70 INDS. SEVENTY.)

This is why I’d argue we need to get much better at flaunting all the nitty gritty details of our programs. Unfortunately, the FDA is not a mind reader and/or fortune teller. They are not going to be able to tell, for example, if a basic, bulleted list of assays pre-IND will be suitable for a BLA (or even a decent approach to demonstrating a product’s potency) without knowing the why and how behind our choice of those assays and our control strategy for them.

We’ve heard time and time again that the more detailed and specific the question, the better the answer we’ll receive from the agency. Based on what the agency representatives offered during this ARM/ASGCT working session, it would seem that “more detail” equates to, for example, providing rationales for selecting each assay, along with how we’re running that test, the controls we’ve implemented, and any preliminary data that exists.

So, for example, instead of broadly asking “Is this approach ok?” in a data- and justification-free bubble, consider instead the following example offered during the working session: “Of these 10 potential approaches we could take to demonstrate X, we think this is the one that could best be developed for lot release to demonstrate X, which we believe relates to the function of the product. Should this be something we pursue?”  

Similarly, though SOPs may be deeply meaningful in-house, they won’t necessarily give the whole picture to our CMC reviewers. Yes, they may clearly depict the “how” behind a process; but they won’t, for example, spell out our thought process behind why we chose a certain number of replicates, cell count numbers, reagent concentrations, etc. Not only does this help the FDA understand where we’re coming from, but it’s also an important exercise for our teams to uncover any holes in our own understanding of our decisions.  

Now, it goes without saying that the industry isn’t the only one that needs to improve. The FDA’s communication strategies are far from perfect — a fact which the agency is just as aware of as we are. The CMC Town Halls, PDUFA VII’s promised hiring spree, plus the potential of a future Operation Warp-Speed Gene Therapy pilot program are great examples of their efforts to broadly fix this shortcoming.

Industry folks during the working session also emphasized the inefficiency of company-agency written communications and the misinterpretations that can arise from those written responses, particularly at the pre-IND stage. The need for more informal, face-to-face communication (i.e., teleconferences) — in addition to formal meetings — was an alluring recommendation. This could be particularly beneficial for companies that require an immediate clarification or for whom an agency disagreement over an offered rationale could be solved more quickly by additional dialogue as opposed to endless email chains.  

3. Stop Doing These Three Unhelpful Things

In addition to imagining monsters under the bed, some of our childhood selves may also be guilty of using the clichéd excuse, “The dog ate my homework” (or some form thereof). Unfortunately, we can’t make the same excuse for not having a potency assay early in development. But there are several things we’re doing today that aren’t serving us well on our quest to efficiently demonstrating the potency of our products. These include:

• Being a copy-cat. You may be reading this and thinking, “LOL; how can we be a copy-cat when no one shares anything about their potency assay approaches?” But there was one comment made by an FDA representative that pointed to companies’ over-reliance on “copying what’s out there” as opposed to taking a much more product-specific approach. Literature spelling out potency assay approaches today is essentially non-existent — especially for approved products. Early literature primarily singles out killing and/or interferon-gamma assays, the latter of which an FDA regulator emphasized tends to be one of the industry’s “greatest hits” because of its broad applicability to a variety of different cell types (i.e. T cells, NK cells, etc.). However, as that regulator added, the rationale for using an interferon-gamma assay in a program often stops there. In turn, it behooves companies to have much stronger, more thoughtful rationales behind our assays until the space gets to a point where a more “platform approach” to potency can (perhaps) exist across (some) products.
• Being a gossip. One of my favorite moments of this working session was when a somewhat bewildered regulator said, “We have developers in this room that can tell you we did not require 5, 6, or 7 potency assays to release their product for licensure. I would really like to get a handle on where the idea that we require all those validated tests is coming from.” This tells me two things: one, we enjoy gossiping and airing grievances as much as any other industry. But two: We need to get better about approaching other companies’ concerns and/or experiences with potency assays with the same amount of levelheadedness we would that social media friend who only posts the best or worst things that happen in their lives. There are plenty of grievances and/or regulatory/analytical platform-building best practices to share across companies. But what we hear and observe happening to each individual company from afar is not necessarily reflective of the whole story. (And as I intimated above, we’re not sharing much at all.) If we look beyond our products’ scientific nuances, there’s no way of knowing the if, how, and what of each company’s communication strategy to/with the agency. We don’t know what each company’s totality of evidence demonstrates — or, more importantly, what it doesn’t demonstrate. As my previous section should have emphasized, it would seem a majority of concerns and/or issues encountered are due to a lack of communication and/or shortcomings in asking the questions to which we need answers. Though we can and have learned the importance of prioritizing potency/CMC from others’ downfalls, more stringent analysis of which concerns/experiences we heed is also critical.
• Blaming all variability on the “science.” Our biological products and analytical approaches will always be marked by some inherent percentage of variability. That said, I really liked the reminder during the working session that we shouldn’t allow the inherent variability of biology to be a crutch keeping us from improving our analytical practices. As one FDA representative offered, there is always the opportunity to add more replicates and improve our controls. In turn, we become better acquainted with our processes and products’ overall consistency, which is just as important as understanding and demonstrating our product’s potency.

4. Practice More Intellectual Generosity

We’ve all been in those conference rooms in which some brave soul stands up with a microphone and asks which assays the speaker’s company used for potency. Then the entire room bursts into awkward laughter. Now, to be fair, I don’t blame that poor soul behind the microphone for their muffled “no comment.” But I also don’t blame that similarly poor soul for asking this question and dreaming of getting some semblance of an answer for their bravery.

This ARM/ASGCT working session was a fantastic demonstration of why discussing and understanding potency is so problematic. As it stands today, establishing a successful analytical approach for demonstrating potency and garnering the agencies’ favor is currently a competitive differentiator. As many rightfully pointed out during the discussion, the fewer interactions the industry needs to have with the agency to align around a potency approach, the less compelled the industry may be to keep every step of that process a trade secret. Of course, not all potency-centric information will be broadly sharable/applicable across ATMP products in the future. (For example, the function of an expressed protein — if that’s important to demonstrate long-term — will obviously differ across products and modalities.)

That said, some high-level efforts were offered to bolster industry transparency, including being more open to sharing what didn’t work in demonstrating potency — both for the company/modality or in discussions with the FDA. (I liked this suggestion, though I acknowledge this would leave corporate comms and legal/PR teams bristling).

There were also suggestions offered around creating modality-specific decision trees outlining high-level considerations to help companies interrogate their MOA and establish long-term analytical strategies for characterization, qualification, and validation. A more granular, modality-specific analytical decision tree could also include lists of specific assays to consider and/or best practices for performing those assays.

While establishing these resources will not be the responsibility of individual companies, we do have some fantastic trade groups at our disposal that could be well-equipped. That said, be sure to take advantage of the good work they’ve already completed for us. ARM’s A-Cell & A-Gene both go into great detail on all the critical steps of developing a cell and gene therapy — including the nuances of analytical development.

Trade secrecy has and always will be a double-edged sword simultaneously helping and (in some ways) hindering the industry. But this is why participation in ARM/ASGCT working sessions, organizations like BioPhorum, and (shameless plug) Cell & Gene Collaborative discussion groups are integral additions to the daily grind. Not only do such organizations provide opportunities to meet each other, commiserate, and brainstorm, but they are always establishing — and reestablishing — the industry’s progress toward a future in which more can be openly shared.