Why are clinical trials so expensive? Tales from the beast’s belly

Today I read, with great interest, a recently published STAT News feature by Matt Herper, Here’s why we’re not prepared for the next wave of biotech innovation. The basic thesis of this article is as follows: In the last decade, we have seen dramatic advances in molecular biology, which have enabled the development of a vast array of novel drug modalities and the successful treatment of previously undruggable diseases. The article mentions CAR-T, CRISPR-Cas9, COVID-19 vaccines, and drugs for cystic fibrosis; I myself also think of PCSK9 inhibitors, semaglutide, bispecific antibodies, and many others. However, the infrastructural requirements of modern clinical trials are now so high, at times reaching hundreds of thousands of dollars per patient, that they are limiting our ability to fully capitalize upon these technological developments.

Herper makes a very strong case for this argument, which I will not reiterate here. Instead, I will share some anecdotes about clinical trial development from my time working in biotech, when we hired large numbers of experienced, ex-pharma personnel to help us initiate multiple trials in the span of two years. Typically, one reads about corporate inefficiency in a very abstract manner. Specters of “regulation” or “middle management” and so on are invoked, but seldom do the details of particular scenarios emerge. I suspect this is in part because almost anyone who has a complete, birds-eye view of the whole scenario has already devoted so much of their life to their industry of choice that they have become willfully blind to its flaws. Unusually, I was an engineer who had the opportunity to watch our Phase I and II trials unfold from a good vantage point, and I hope the stories I share will be informative. Overall, they paint a picture of an industry which is severely lacking in human capital and fully captured by bureaucratic tendencies.

Contracting out clinical trial management to external organizations

Clinical trials are very complicated endeavors. If you are a very large pharmaceutical company, like Pfizer or Amgen, you may have developed all the internal resources necessary to run clinical trials by yourself. However, even if you are a large company, you may be operating at full capacity or lack personnel in key regions of the world. On the other hand, medium and small biotechs certainly do not have the capital to keep themselves staffed with a full retinue of potentially underutilized staff. For these reasons, it is very common for clinical trial management to be outsourced to contract research organizations (CROs), large external entities who will assign your clinical trial a project team of 20+ people (each of whom is probably working on multiple other trials at the same time) and help you execute the trial as desired.

There are many organizations that specialize in this service, such as IQVIA (formerly Quintiles and IMS Health), PRA Health Services, ICON, PPD, and innumerable others, ranging from boutique operations with 5-10 people who specialize in a niche field or in one particular geographic area to enormous corporate entities with tens of thousands of employees like PPD. To select a specific vendor, therefore, the contracting entity typically goes through a process of soliciting bids from several different CROs, winnowing them down to a shortlist, and setting up meetings to evaluate them in greater depth.

While superficially quite reasonable, actually going through this process is an extended exercise in sheer absurdity. Consider the initial bid solicitation process. There is no open forum where companies post an RFP (“request for proposal”) and receive proposals from CROs; instead, typically the company’s internal project manager will arbitrarily select a manageable number of CROs (in my experience, around 5-7) and personally reach out to their contacts inside the CRO to initiate the RFP process. (It is important to note that this is a deeply incestuous industry, so it is not too hard for senior personnel who have been “around the block” to have contacts in all of the large organizations.)

For the clinical trials I was involved in, the (very senior, ex-pharma) project manager put together a very detailed RFP. After all, if you’re going to ask for information, you may as well ask for a lot, right? Aside from a massive, 10-page table of requirements, this included free-form questions such as:

  • Please discuss your strategy for rapid site budget and contract negotiation and execution.
  • What tools do you use to determine fair market value in developing site budgets?
  • Do you have experience working with Functional Service Providers (FSPs) who perform contract and budget negotiation and who handle site payments? Please describe.
  • In what countries do you provide legal representation? What are the locations for this service?
  • How are issues and data questions escalated to the sponsor? What is the communication plan and who is the main contact at the CRO?

The astute reader may suspect that these questions (over 30 of them!) are not particularly helpful. How, indeed, are “issues … escalated?” One would hope they are escalated quickly and competently, I suppose? Exactly how is it informative to have a full tabulation of every single country in which the CRO could conceivably “provide legal representation,” and what would the context of this representation even entail? How does one “discuss” a general “strategy for … contract negotiation?” I would simply… negotiate well…?

Furthermore, imagine that you are a large CRO and a small biotech asks if you would like to submit a bid to run their clinical trial. You of course rationally know that they are also soliciting proposals from at least several other CROs; also, the details of their trial are probably reasonably complicated and would take a long time to deeply understand. Rationally, it does not make much sense for you to expend great effort into formulating a fully personalized response. It certainly does not make sense for you spend $200+/hr employee time writing detailed, thoughtful answers to all 30+ of these questions.

Unsurprisingly, what happens is that CROs essentially have enormous templates of boilerplate text about their capabilities, and they just send these back after slotting in the details of a provisional team, making up some rough numbers about trial recruitment here and there, and putting together a hugely overinflated budget.

The following response is representative:

Wow. Really? The program leader will be the primary contact and they’re expected to maintain regular communication? What exactly would they do otherwise? This answer is basically “we’ll handle it” except expanded into two paragraphs of meaningless bloat. There are, in this proposal from a major CRO, fully fifty-seven pages, nearly all filled to the brim with copy-pasted boilerplate.

Suppose one attempts to quantitatively compare the costs estimated by each CRO. This is not a trivial task, considering that each of them breaks down their costs in a different way! Nevertheless, if one expends several hours to consolidate all the spreadsheets together, we see a remarkable range of variation:

The numbers vary so drastically that one might as well just throw darts at a pinboard. Why does one CRO estimate $133k in regulatory costs (no idea what that means exactly, given that filing in this non-USA jurisdiction is very straightforward), and another literally $0? Exactly how does “Data Management,” which mainly entails setting up proprietary database software to capture the structured data we need, vary from $168k to as much as $431k? (Note that the data management costs do not include the licensing fee for the software chosen, which ranges from $75k to over $300k.) What, exactly, accounts for the differences in in “Clinical Monitoring” activities that entails cost variations of over half a million dollars? There is literally no way to understand why the costs vary so much across the different sections, and in the end the decision was largely made on the basis of overall cost.

So now we have 5+ huge PDFs full of the biotech equivalent of Lorem ipsum dolor sit amet... and a bunch of cost numbers we barely understand. There is no real way to compare hundreds of pages of generically useless content against each other, so several candidates are eliminated based on simple criteria, typically if their estimated costs are truly enormous relative to the other options. The worst, however, is yet to come. Once the shortlist has been selected, it is time to engage in the theatrics of the “bid defense.”

For those not acquainted with a “bid defense” in this context, I will explain how you might set one up:

  • Gather 10+ employees from the client
  • Gather 10+ employees from the CRO
  • Schedule a four hour meeting
  • Listen to each employee from the CRO dutifully go through their assigned section of the 200-slide-long deck and recite a moderately well practiced script

The introductions alone usually take half an hour. (They are also entirely pointless.) Very senior level employees are often invited to these meetings, resulting in a nominal cost of tens of thousands of dollars’ worth of hourly wages. (They are probably tabbed out and working on something else 95% of the time.) The content is extremely boilerplate. Wow! You can use $GENERIC_INDUSTRY_SOFTWARE for your electronic data capture system! Am I supposed to be impressed?

In practice, there are several essential points that are discussed in the Q&A sections. For example, we were sometimes very concerned about trial recruitment, so we would inquire extensively about patient identification methods. One wants to understand not only the CRO’s technical capabilities but, basically, how smart they are and how motivated they seem to figure out good solutions to the challenges of your specific trial. However, this questioning would typically involve only about 3-4 people total and take around 30 minutes max. In spite of that, we nevertheless went through this entire four-hour exercise for every single shortlisted CRO for every single clinical trial that we attempted to initiate. It is challenging, and maybe even impossible, for me to really describe how unpleasant these were.

Note that everything I have described so far is normal biotech/pharma industry practice. In fact, it is probably a full standard deviation better than industry practice, because while we hired people with prior experience in large pharmaceutical companies or CROs, the fact that they even considered working at an obscure biotech startup means the population of good applicants is highly selected for the relatively nonconformist and entrepreneurial. (The key word here is “relatively.”)

Now it is time to make a decision. There may be several more small rounds of back-and-forth inquiry, perhaps some light negotiation on the budgets, but ultimately the CRO will not be inclined to do much more work for free. One might imagine that there is actually a lot of room for negotiation, given the enormous range of variations in the estimated costs across different CROs, and indeed I myself thought the same; however, it seems that in practice, there is an “acceptable degree” of negotiation and it is generally challenging to reduce any given CRO’s budget by more than, say, 10%.

In principle, this decision is a very consequential one, and it is very important to choose the right CRO. The success or failure of the clinical trial, and therefore the success or failure of the company as a whole, could potentially hinge on the right decision. As such, the company is, theoretically, incentivized to engage in an effective decision-making process. Ultimately, though, when one is down to a shortlist of 2-3 reasonable CROs, it basically reduces to a question of “who has the best vibes?” Essentiallyーonce you have made sure that all the hard criteria are met (geographic location, technical capabilities, etc.), you ultimately have very little to go off aside from your vague gut feeling about who is more competent and motivated.

The industry standard, however, seems to be to engage in a farcical “data-driven” scoring process where you draw up an enormous table of the different candidates and assign them crude scores:

(There is much more to this table that I have not shown here. The reader should note that the construction of this table was considered “excellent work.”)

There is one candidate, in the rightmost column, with a very low score. This particular CRO was only included because someone internally really wanted them to be considered, but the project manager was too nice to simply say “no, these people suck, so we won’t even bother scoring them.” To be honest, the rest of the scores are meaningless .How exactly does one count the value of a “good” rating versus the value of an “average” rating? The choice of “categories” also affects the rating substantially; for example many of the categories (not shown) are basically proxies for the size of the CRO, resulting in much higher overall scores for large companies. Perhaps one wants a large CRO, but if so, it would be far more efficient to encode that preference directly rather than pretending that the large CRO is good because of all its high scores in categories which implicitly measure company size! Essentially, my point is that this entire scoring exercise was arbitrary and pointless, as it can simply be designed to encode whatever preferences you held in the beginning.

In the end, we came to a decision which was more or less a priori known without the need to engage in the final evaluation process at all. Based on certain of my colleagues’ comments, I am supposed to understand that this was a remarkably fast and efficient processーrelative to the industry standard, that is.

The logistics of setting up a clinical trial

Once you have selected a CRO, the journey has only begun. In between the CRO “kickoff meeting” (yet another interminable, multi-hour, 20+ participant meeting where a great amount of boilerplate text is recited and, eventually, a plan of action is promulgated) and the enrollment of the first trial patient, there is a huge amount of setup that needs to be performed, including regulatory filings, clinical site identification (you need actual hospitals to run the trial in!), questionnaire construction, medical device procurement, and so on.

Take, for example, the problem of site identification. Essentially, you need to find clinics who are willing and able to enroll patients in your trial. In return for doing so (potentially a very logistically complex task), they are compensated handsomely. The typical clinician is not set up to field these inquiries, so it is not uncommon for clinics to join larger organizations which act as an intermediary between CROs and clinicians (one example is Clinitrials in Australia). Trial design has increased substantially in complexity over the last decade (everyone wants to measure every conceivable biomarker at every timepoint, etc.), and correspondingly these supra-clinic organizations have taken on increasingly larger roles relative to the participation of individual clinicians within unrelated institutions. At an abstract level, this is likely a contributor to the rise in per-patient costs: these organizations add an additional layer of bureaucracy and extract rents, while the incentive of the CRO is not to minimize “pass-through” costs to the client (which can certainly come in over budget!) but instead to sign on clinics quickly and start the study. Personallyーthere was an intermediary clinical organization we attempted to work with which was so obviously attempting to grift money from our coffers, the entire relationship escalated into the level of back-and-forth legal threats.

As with the initial stage of CRO selection, there is no real centralized marketplace for a company to post a clinical trial synopsis and solicit open bids from interested clinicians. (Realistically, many companies would not even want to participate in such an open market due to leakage of confidential information.) As such, initial contact with clinics typically depends on the submission of an initial feasibility questionnaire to a large number of clinics derived from a combination of the CRO’s internal databases and personal connections of whoever is involved in management of the trial. If the clinic appears superficially qualified and willing to participate in the trial, it then undergoes several more rounds of review, in-person or remote visits to formally “qualify” the site, site-specific budgetary negotiations, preparation of site-specific regulatory paperwork, and so on.

What was most notable about this process was the extreme heterogeneity between clinics. Some clinics are quite used to trial participation and proceed through the entire qualification and training process with no issue. Other clinics are very particular, have special requests about the protocol, require dealing with very intransigent doctors, and take weeks to respond to simple inquiries. The extent to which one has to put up with the more difficult clinics is largely a function of the anticipated trial recruitment difficulty as well as the desired trial size. Still, to be honest, this was probably one of the better-functioning parts of the entire clinical trial effort.

Another part of organizing clinical trial logistics is setting up what is known, variably, as the IRT (Interactive Response Technology), IVRS (Interactive Voice Response System), or any number of other very similar acronyms. Basically, this is the drug inventory logistics system which ensures that (1) clinics have enough drug on hand at all times to handle new patients and (2) patients are always being dispensed the correct treatment. For example, you might have 500 bottles of drug product and 10 participating clinics, but you don’t know what rate each clinic will enroll patients at; also, once you send drug product to a clinic, it can be very challenging to get it back. For that reason, you might want to “seed” each clinic with, say, 20 bottles of drug product, then replenish each clinic’s supply as it diminishes over time. Separately, it’s not uncommon for a patient’s course of treatment to change during a trial; for example, if certain blood tests report unusual changes, you may wish to reduce the dosage of drug supplied to the patient. This of course requires more backend management and record-keeping. Finally, you need a web-based interactive system that lets different parties monitor the status of drug supply, request more bottles if needed, report issues with deliveries, and so on.

I bet that you, my reader, are thinking something similar to what I did when I first heard about these systems: “That doesn’t sound too complicated! Even if you were starting from scratch, it wouldn’t take 1-2 moderately competent software engineers more than one week to code up everything you need, including an easy-to-use web interface.” You might even go further and imagine that the companies who have specialized in providing such systems have already developed a robust codebase to work from, and therefore only need to spend a day or two encoding the specific requirements of a given trial’s specifications.

Want to guess how much we were quoted for our IRT system?

To be clear, we are literally talking about a glorified SQL database, with basically zero scalability demands, plus a frontend that lets you click buttons to execute a couple of simple queries. I wish that I could actually show you what it’s like to click around this systemーit really gives you the feeling of being transported back to the year 2005, as though you’re using some incredibly janky piece of accounting software on Windows XP and with negative levels of polish.

I should also mention the following:

  • It took several months for this IRT provider to “program” (?) the system
  • The 100-page specifications document, that our very senior ex-pharma professionals supposedly read and signed off on, contained fundamental errors
  • We therefore ended up somehow canceling this contract and rushing to engage a different, still very expensive (>$100k) vendor for our IRT needs

One cannot help but get the impression that this is an industry which is, at best, severely lacking in anything that could be called “human capital” and, in reality, simply deeply unserious.

More generally, it is unusual that despite the very large amount of boilerplate content that every single actor in this ecosystem has spent thousands of hours of manpower developing, there seem to have been very few actual gains made in simplifying common tasks or processes! For example, in any given clinical trial protocol, there are typically a variety of exams that need to be performed at specific timepoints: ophthalmological exams, cardiovascular exams, blood draws, and so on. As such, it’s standard practice to give clinicians a worksheet to fill out as they perform the exam, so that the data in the worksheet can be recorded.

One might assume that a CRO, who has run tens if not hundreds of thousands of clinical trials, would have a standardized, efficient process for taking a trial protocol’s specifications of these exams and translating them into formatted worksheets! Apparently, however, that was not the case. Despite literally paying the CRO to run the trial for us, we had to manually generate pages upon pages of these worksheets ourselves:

This is not a difficult task per se. There is just something vaguely absurd about the fact that basically every single clinical trial is crudely designing the same (or very similar) worksheets over and over again, with essentially zero effort made to eliminate redundant work even within the confines of a single organization who has every incentive to simplify these processes! Instead, what happens is that for every single trial, new worksheets are made; the content of the worksheets is used as an input for the setup of the electronic data capture (EDC) system, and because the worksheets are not standardized in any way, the EDC setup then requires a great deal of manual setup and inspection; finally, even though you need a new worksheet for every single exam visit that a given procedure is performed (for example, you might want to perform blood draws on clinic visits 5, 7, and 9), the industry standard for creating a “booklet” of worksheets with slightly modified headings on every page is “copy paste the same content in Microsoft Word over and over again, manually change exam dates and version numbers, and visually inspect the final PDF for errors.”

Nothing about this process makes sense. I hate to be the person who says “well, smart tech people could make this so simple,” but it just literally seems to be true.

There are so many examples of horrible, atrocious inefficiencies that I encountered while dealing with the logistics of clinical trial setup that I could easily triple the length of this section if I wanted to. Here’s a light sampling of some other issues:

  • We paid five figures to a highly-recommended firm specializing in developing patient recruitment materials; received draft advertisements with blatant grammatical errors, clear misunderstandings about the target population, and low-quality stock images
  • Computerized systems like the IRT or the EDC are supposedly “validated” by a process called User Acceptance Testing (UAT), where the vendor gives the client a 1-2 hour long list of carefully defined steps to execute on the web interface and at least 3 highly paid employees on the client side go through the exact same steps in dutiful compliance (note, obviously, that if the testing steps are explicitly defined, it should be possible to just automatically check that the system responds correctly to them, and it definitely should not take more than a single tester)
  • Every single example mentioned so far probably involved 10+ hour-long meetings with 5+ participants, the majority of whom never said more than one or two words in any given meeting

Even to this day, I have difficulty wrapping my head around the sheer scope of incompetence that I encountered on all sides of this process.

Everyone in this industry is acclimated to a very low level of productivity

What is even worse than rank incompetence is the fact that even people who are very competentーscientists with invaluable technical knowledge, for exampleーseem to have just given up and grown acclimated to endless bureaucracy and low productivity.

Let me supply a concrete example: Our company had a large batch of technical documents which we needed to translate from an Asian language to English. These documents were supplied as Microsoft Word files, and for whatever reason the translation agency returned the translations as separate Word documents in this format:

It’s obviously very challenging to read a dense technical document, full of chemical structures and labels all over the page, if you’re constantly referring back to a table of translations. The solution? The two contractors assigned to manage the chemical formulation process (we are talking about billing rates of $300+/hr easily) manually copy pasted translations from the table into the source document.

Each document was easily 5+ pages of content, some perhaps 20+ or longer. There were probably 100+ such documents.

Why would a professional translation company, with a strong reputation for competence in translations of technical documents relevant to clinical development, not have an internal tool that lets them automatically insert a table of translations back into the source document? Why would our entire team of ex-pharma professionals look at this manual copy-paste process, nod their heads, and think, “well, that’s slow, but there’s just no better way, we’ll just have to pay the contracting hours?” How does this sort of situation even arise?

In the end, I realized this was happening, spent an hour or two writing 50 lines in Python to parse Word documents and automatically replace source text with the translated content, and probably saved upper 5 figures’ worth of billable hours in the process. But how many companies are out there where this doesn’t happen? This is but a single example of quiet acceptance of gross inefficiency that I came across in my time in this industry. I will leave the natural inference to the reader.

There is a quote from the prolific internet schizopoaster Ron Maimon of which I am quite fond:

Rudeness is ESSENTIAL, as it is the tool that is most effective for alienating yourself from the lowlifes and scumbags, the moderators, the top writers, so that one is not influenced by their officious thetans, which have since been clinging to me like snot from a sneezy nose. Polite speech is like passing a turd orally, the only ones who don’t mind the taste are those whose heads are already acclimated to the colon environment.” (February 14, 2014, in the Facebook group “Quora Top Writers”)

Setting aside what one thinks about the value of rude speech, I bring up this quote to propose my own, parallel version: “Corporate inefficiency is like passing a turd orally, the only ones who don’t mind the taste are those whose brains have already fully atrophied in the meeting-room environment.” Sadly, this seems to often be the case with professionals with decades of experience working in clinical development. They possess a great deal of invaluable knowledge, but they also fully corrupt the corporate environment.

Suppose that one tries to push back on inefficiency in generalーsomeone naive but well-intentioned, like myself, who is tired of the Long March of interminable meetings. In fact, I expressed my concerns about the proliferation of unproductive meetings in the clinical department to the C-suite executives (with whom I remain on good terms!) and, together, we carefully planned a series of reforms to company culture designed to reduce meeting burden. We hoped to implement a soft cap on the number of meeting participants and the length of a meeting, to normalize efficient behavior such as leaving meetings midway if you no longer need to participate in the remainder, starting meetings on time without 15 minutes of small talk, and a mandated review of all recurring meetings with the intention of pruning overall meeting load by >50%.

Ironically, on the morning that we planned on roll out these reforms, we received urgent and vehement protests from a member of middle management who was notorious for obsessively overscheduling meetings with huge participant lists just for the sake of “inclusion” and “engagement.” Because this particular manager was currently playing a crucial role in managing the rollout of several clinical trials at the same time, executive management judged that the risk of alienating him was too high, and to this day I believe these reforms remain unimplemented (although I hear he has since left the company, so perhaps my slides will eventually see the light of day).

Closing thoughts

Why is this industry so cursed? Some hypotheses:

  • There is fundamentally a lack of talent flowing into the space, because everyone smart is eaten by tech or finance (the only reason I had any insight into this at all was because I was an ML-focused engineer in a biotech company heavily funded by tech VCs)
  • Everyone who is reasonably smart and joins this industry eventually either leaves or simply gives up on any hope of positive change to the status quo; to justify the state of affairs to themselves, they invent nonsensical copes about how things couldn’t be any other way
  • The massive amount of undocumented specialist knowledge that you need to efficiently run a clinical development program strongly favors incumbents, preventing new market entrants from easily competing on the basis of cost or competenceーe.g., it doesn’t matter if your firm has 170 IQ engineers if they simply don’t know all the One Weird Tricks about how to get around the FDA’s catch-22s
  • Because the cost of failure is so high, well-capitalized companies will always favor established methods, even if they are slow and inefficient, as long as they present a reasonable chance of getting the job done eventually
  • To effectively run a clinical trial, you must hire on a large number of ex-pharma professionals for their specialized knowledge, and that fundamentally degrades the culture of the company beyond recognition

To be honest, I don’t have any great solutions to propose. The problems seem nearly intractable in their scope and magnitude. Even if a return to positive real rates leads to a renewed focus on the “world of atoms,” it could take decades for the industry to become more efficient!

Good luck!

November 9th, 2022 | Posted in Biology

15 Responses to “Why are clinical trials so expensive? Tales from the beast’s belly”

  1. Wednesday assorted links - Marginal REVOLUTION Says:

    […] 6. More on the problems with the current system of clinical trials. […]

  2. Ryan Miller Says:

    Why do you think this is unique to clinical trials? Have you ever been on the client side of an ERP buy, or any other large, complicated, expensive vertical market software? It’s dysfunctional, but it seems like it’s the inevitable outcome of trying to outsource a core business function that lacks a standardized interface definition (as with horizontal market software like email or word processing). See what even getting a point of sale system installed costs in money and time.

  3. Ann Olmsted Says:

    What oft was thought, but ne’er so well express’d,
    Something, whose truth convinc’d at sight we find,
    That gives us back the image of our mind.

    Thank you for posting this.

  4. David Lilienfeld Says:

    Having been in pharma for 3 decades, I find it interesting when people argue that the current scheme is inefficient and urge reforms. Fine. Except that when there are reforms, you get thalidomide/vioxx/phen-fen, etc. And each time you have such episodes, it undercuts the public’s faith in the system, leading to even more onerous requirements for marketing approval. The nuclear power industry in the US has gone through this process to the point that it is practically impossible to get a cost-competitive nuclear plant built in the US.

    It is possible that the current system is, for the info required for approval, the best that can be done. I don’t think that it’s necessarily reflective of low productivity any more than WIlliam Halsted’s slowness in the OR was reflective of poor productivity. He got superior results from that slowness/diligence.

    Maybe the issue isn’t how to do clinical development less expensively as much as how to eliminate candidates before they go into clinical development. Not enough discussion of that.

  5. milkyeggs Says:

    > Maybe the issue isn’t how to do clinical development less expensively as much as how to eliminate candidates before they go into clinical development. Not enough discussion of that.

    There are literally hundreds of books and papers about how to reduce clinical failure rates with complex in vivo tox testing, phenotypic screening, more translatable models, etc

  6. Kevin Says:

    Wonderful insight that aligns with my precise experience.

  7. Mason Bogue Says:

    >Except that when there are reforms, you get thalidomide/vioxx/phen-fen, etc.

    I see no evidence that any of these events occurred because of reforms, deregulation, or anything like that. Notwithstanding that the industry might not actually need reforms in the policy sense but rather innovation from the private sector.

    In the case of rofecoxib, Merck manipulated data — some of which had already been sent to the FDA — and no changes were made to the approval process in response. In fact, let me quote Wikipedia for a second:

    >Following the 2005 FDA Advisory Committee, the FDA issued a memo concluding that data from large long-term controlled clinical trials do not clearly demonstrate that COX-2 selective agents (including rofecoxib) have a greater risk of serious cardiovascular events than non-selective NSAIDs.


    It turns out that the whole drug class that Vioxx was based on — which, including Advil and Aleve, was grandfathered into approval — carried the same risks. The FDA ultimately recommended that Vioxx be returned to market, but Merck declined to do so, likely due to bad press. Is this what I’m supposed to see as the output of a functioning, competent system?

    For completeness, fenfluramine (the toxic half of “fen–phen”) was introduced to the market in the 1970s, long before modern regulatory processes existed. Thalidomide, on the other hand, was never permitted on the US market at all.

    Overall, these examples offer no defense of the current system of relay-race buck-passing except as misunderstood boogeymen that are used to mislead the public.

  8. Oliver Eidel Says:

    This is so, so interesting. Thanks so much for posting this!

    You already touched upon a few hypotheses why the industry has arrived at this state. There’s one I’d like to highlight: You, now having gone through this experience, would actually be in a perfect position to found your own CRO and implement all your efficiency learnings there. However, you look back upon this experience and think it was super painful and you’d prefer to do fun stuff like developing software instead.

    That, for me, is the main reason. There are smart, structured, pragmatic people who got acquainted with this industry, but they choose to leave it.

    And that’s sad!

    So the solution would be to find a group of smart, structured, pragmatic people with a high tolerance for bureaucratic pain and give them the resources to make this industry more efficient.

    And this is actually happening: That’s exactly what we’re doing at OpenRegulatory, albeit for medical device software and not for clinical trials. Among other things, we’ve published all our document templates for free (also on GitHub), host a free Slack community for companies to help each other out, and offer a compliance SaaS with transparent and fair pricing. It’s still a long way, but it’s a start 🙂


  9. Brian Moore Says:

    “Nothing about this process makes sense. I hate to be the person who says “well, smart tech people could make this so simple,” but it just literally seems to be true.”

    This sentence can be repeated at hundreds of points in the health care process, but there are a limited number of smart tech people in the world, and they have been scared off of health projects due to regulatory/legal/moral barriers. So you get… a different type of tech. See: EMRs.

  10. John YS Says:

    I had the same experience as the author when I first started learning about clinical trials from the perspective of the software world.

    The amount of money invested in clinical trials and lack of visibility is staggering. We’ve companies that spend the entirety of their $100Ms in funding and not realized until month(s) after the fact when their accruals caught up!

    A lot of it *is* messed up and can be improved. Indeed, we’ve have more demand than we can service since we productized what we’re working on at Condor (https://condorsoftware.notion.site/About-Condor-d8a466f7722e46e0b68ec5c8895e783c).

    However, similar to what others have alluded, Chesterton’s Fence is a relevant concept here (https://fs.blog/chestertons-fence/). It’s critical to understand the rational reasons why the industry is so apparently messed up before you can fix them affectively. I wouldn’t be working on the problem we’re tackling today, if it weren’t for my cofounder’s deep knowledge across several disciplines in the industry.

    One thing that continues to make a meaningful impression on me is the mission driven motivation behind so many people working in the industry of all levels. Consistently from the chief scientist to the entry level accountant you hear people talk about how they tie their work to the patients. Many software companies say they are “changing lives,” many life science workers can actually point to saving them. It’s been a big motivating factor for my cofounder and I to realize the same companies we are helping extend R&D runway are the ones making cancer drugs for some of those close to us.

    Messy industry dynamics and backwards software means there are lots of BHAGs (https://www.jimcollins.com/concepts/bhag.html) to tackle.

    Always down to talk shop on clinical trial finance, software, or ways to collaborate with good people whose interest this piques! Contact in bio.

  11. Amy Gomez Says:

    Much of your concerns are valid and you are painting a real.picture of the “way it has always been” nevertheless there are groups of companies/ clinical operations professional doing things differently and proposing ways to change the paradigm.

    It’s not an easy task. We need Ops leader that understand what technology can really do for the field, that are unafraid of taking risk and demand and foster better process and relationships with development stakeholders. I know great examples of what can be done, we need the industry as a whole to get there.

  12. Melissabime Says:

    I totally understand the frustrations experienced by the author of this post. I am a nurse and my exposure to the Clinical Trial Industry came from a more bottoms up entry point. Its mind boggling to the the total costs CRO’s charge for the services compared to the eventual experience for the staff and patients taking part in these studies. CRO’s are in dire need of disruption. I also agree about the incestuous industry hence the very slow progress being observed. It’s hard to show any progress when it’s the same people moving around the same companies and implementing ideas that are somewhat outdated. Also, from a cost perspective, i think the future is also coming to a place where Clinical trials get outsourced outside the US. Sites and PI’s outside the US should be empowered to run this clinical trials as they provide a more cost effective solution with regards to operational costs and also access to an ever growing number of patients, reducing clinical trials timelines that lead to increased costs. I understand the need for CRO’s to maintain quality, and they can do this by building out the right relationships and partnerships with overseas contractors, but i believe in the long run, this is the solution. I believe this so much that i launched my own CRO company. Infiuss Health. The goal of this company is to unlock the potential for studies to the run by CROs and smaller life science companies outside the US. We are starting with Africa. As soon as we can unlock this population that has traditionally just been taboo to life science companies, imagine the lengths we could go with with access other geographies.

  13. David C. Norris Says:

    The concept of https://en.wikipedia.org/wiki/Rent-seeking could be useful in developing this argument more formally. It looks to me as if you’ve laid out the necessary connections already, especially in the 3rd and 5th closing bullet-points.

    One might say that *mediocrity* is able to ‘extract rents’ from the regulatory burdens imposed on the industry.

  14. ICYMI – Recommended Reading and Listening for Biotech Innovators - TimmermanReport.com Says:

    […] lose perspective on some of the grueling verities of our work. I can imagine no better tonic than this blog post, describing in excruciatingly painful, all-too-familiar detail the preposterous dance […]

  15. Covid 11/17/22: Slow Recovery | Don't Worry About the Vase Says:

    […] Some anecdotes about clinical trials and how impossible we have made doing them at any reasonable cost, with typical prices at hundreds of thousands of dollars per patient. […]

Leave a Reply