Evidence & Performance Blog

Christophe De Vleeschouwer, Manager Medical Operations for Cordis Medical (a Johnson & Johnson company) in Belgium, is exposing his thoughts and needs about measuring performance for large cohort clinical registry. Christophe has met us last week, and Alex took his confessions.

[ADS] Can you please tell us about your experience in managing Large cohort Registry?

[CDV] My main current responsibility is to manage a WW Registry involving > 300 sites and > 50 countries. This company sponsored registry is the largest (with 15.000 patients included) ever conducted in Interventional Cardiology and in the Drug Eluting Stent Business. After approximately 2 years of enrollment, we are now entering a phase of long-term follow-up for every patient. They are contacted at 1, 6, 12, 36 and 60 months post-procedure. This registry has been set-up as a routine post-marketing surveillance activity but is also dedicated to answer long-term safety questions and therefore is a very comprehensive and large collection of data. An e-CRF has very logically been selected to host this large amount of information. Only since the upcoming and well adoption of the EDC technology, the management of these very large registries has been made possible.

[ADS] Comparing to Phase III, IV, what are, for you, the main difference in the operational management of large cohort post marketing registry?

[CDV] The scale and the size is the major difference between the typical phase III-IV studies and the large cohort registries. However there are also a few other nuances that make the management of these large cohort different to phase III-IV studies :

1°) as per essence the pivotal studies are targeting selected patients, selected centers and operators. In a large registry, the goal is to give a real life picture, promoting the use of virgin centers in terms of clinical study experience from various geographical regions with different clinical practice settings. The planning and timelines tracking of such project is even more complex due to the large involvement of sites and countries with different languages, regulations, …

2°) the large numbers allow identification of potential rare but serious problems. Events with incidence around 1% of the population will be captured in sufficient cases to be able to do some trendings and calculated risks, etc …

3°) the population included is larger in terms of medical history differencies, severity, complexity, etc … The results are supposed to reflect the real life image and can be sliced into many different sub-groups.

Taking into account all these points above, the management of such projects will require - Good coordination skills for the control of the set-up and running phases to avoid major timelines shift and cost deviations 

- Good data analyses capabilities to avoid any bad surprise on trending and target cleaning and on-line review. Indeed in such big project where on-site monitoring is often limited, the ongoing in-depth analyses of the data can avoid big deviations or issues in particular sites

This day to day management is often difficult, and especially with such database and project size due to the lack of pre-existing comprehensive tools and reports.

[ADS] Is there for you an easy way to evaluate the overall performance of a clinical trial?

[CDV] No. All depends on what you want to assess within the clinical trial : is this about the site, about the study itself, the people and CROs involved ?

Is it to assess the timelines, the quality, the costs ?

Of course, they are all important, nevertheless every project team or organization has its own idea based on their beliefs and current priorities …

Who is the requester and what are the needs ? A study team will be interested in intra-project performance measures. The manager of the department will wish to have cross-study metrics …

Anyone who worked in management of clinical studies has been interested to know the timing needed between a protocol is approved and ready to be launched and the first patient is included. However, they are many more metrics that can calculated !

So far, this field of clinical performance has been very marginally explored and often based on very specific & isolated needs. There are only very few organizations who use it as a business tool to take informed decisions. A real methodology is often lacking and no formal consensus exist on what and how to do !

[ADS] Could you give us an idea of some of the outstanding issues you faced in assessing the performance of these type of CT:

[CDV] There are many issues that we may face when we want to assess the performance :

1°) So often, as project manager, we are so much looking into the very next tasks and urgencies we need to resolve that we do not have the time to stop and have this helicopter view on what we do

2°) The many sources of information from different electronic providers (EDC, CTMS, IVRS, …) that are linked to the study life cycle require a real gymnastic to capture the necessary data to do the metrics.

3°) The integration of similar data from these various electronic providers is often not happening for technical, time and cost reasons and impose us to create our own files to run some metrics. This task is often complex because we have to play with so many files formats, the information can be redundant and the quality of the data can be questioned when copy-paste is done …

[ADS] What can metrics bring to your project  — in other words, how will they affect the overall performance of the registry.

[CDV] There are many benefits as explained above but of course at the stage of my project (post-inclusion), these can better help to draw some lessons on future projects than to react within the current one.

For instance I decided to apply a metrics on the EDC backlog time between the inclusion of the patients and the completion of the procedure in the e-CRF. If you combine this with the inclusion rate of the site and the patients with major deviations, it can give a very good quality indicator on what the site delivered !

By specific metrics on the sites and countries we will be able to better pro-actively plan any future registry in a specific country and know where to go if we need to speed-up the enrollment for instance …

[ADS] Given that you were at the “5th Clinical Trial Performance Metrics Conference”in London , was there anything you heard or saw presented there that you would think would be worthwhile to add? What are you point of view about the benefits of the Clinical Trial Performance Management?

[CDV] The conference helped me to understand that:

 1°) there are many initiatives ongoing in several companies looking at specific clinical metrics depending on the company needs : some are looking at countries performances, others at cross-study benchmark, quality management, service providers, etc …

This interest is not bound to the size of the company, the quantities of studies managed or the therapeutics ! Any organization will find benefits in developing and mastering these tools.

2°) the external environment and pressure on deliverables from financial institutions looking at R&D pipelines will make these performance indicators even more required in the future and organizations that will implement the right tools and be transparent will be the most successful

3°) there is a current lack of standardized approach on the topic and guidelines, consensus will be needed. A methodology would bring a better and faster adoption. The existing tools are mainly company made and are very focused, using the old technological way of consolidating data and reporting. CTMS applications are not filling this gap and many organizations still have to gather information from different places

4°) there are very few external benchmarks provided by specialized companies. These have primarily put the accent on the external benchmark but are not really helping for the internal considerations

5°) Many organizations, consultants or service providers are focusing on the metrics analyzes of recruitment speed, which is according to me a very limited step in the overall process. 

We were very pleased to meet Christophe for this interview and we thank him for taking the time to respond to our questions in such a thoroughly way.