Spatio-Temporal Analysis of Patient-Derived Organoid Videos Using Deep Learning for the Prediction of Drug Efficacy

by   Leo Fillioux, et al.

Over the last ten years, Patient-Derived Organoids (PDOs) emerged as the most reliable technology to generate ex-vivo tumor avatars. PDOs retain the main characteristics of their original tumor, making them a system of choice for pre-clinical and clinical studies. In particular, PDOs are attracting interest in the field of Functional Precision Medicine (FPM), which is based upon an ex-vivo drug test in which living tumor cells (such as PDOs) from a specific patient are exposed to a panel of anti-cancer drugs. Currently, the Adenosine Triphosphate (ATP) based cell viability assay is the gold standard test to assess the sensitivity of PDOs to drugs. The readout is measured at the end of the assay from a global PDO population and therefore does not capture single PDO responses and does not provide time resolution of drug effect. To this end, in this study, we explore for the first time the use of powerful large foundation models for the automatic processing of PDO data. In particular, we propose a novel imaging-based high-throughput screening method to assess real-time drug efficacy from a time-lapse microscopy video of PDOs. The recently proposed SAM algorithm for segmentation and DINOv2 model are adapted in a comprehensive pipeline for processing PDO microscopy frames. Moreover, an attention mechanism is proposed for fusing temporal and spatial features in a multiple instance learning setting to predict ATP. We report better results than other non-time-resolved methods, indicating that the temporality of data is an important factor for the prediction of ATP. Extensive ablations shed light on optimizing the experimental setting and automating the prediction both in real-time and for forecasting.


page 3

page 4

page 6


Ins-ATP: Deep Estimation of ATP for Organoid Based on High Throughput Microscopic Images

Adenosine triphosphate (ATP) is a high-energy phosphate compound and the...

Personalized Pancreatic Tumor Growth Prediction via Group Learning

Tumor growth prediction, a highly challenging task, has long been viewed...

SystemMatch: optimizing preclinical drug models to human clinical outcomes via generative latent-space matching

Translating the relevance of preclinical models (in vitro, animal models...

Prediction of drug effectiveness in rheumatoid arthritis patients based on machine learning algorithms

Rheumatoid arthritis (RA) is an autoimmune condition caused when patient...

Dose-response modeling in high-throughput cancer drug screenings: A case study with recommendations for practitioners

Personalized cancer treatments based on the molecular profile of a patie...

Towards Automated Tuberculosis detection using Deep Learning

Tuberculosis(TB) in India is the world's largest TB epidemic. TB leads t...

TripHLApan: predicting HLA molecules binding peptides based on triple coding matrix and transfer learning

Human leukocyte antigen (HLA) is an important molecule family in the fie...

Please sign up or login with your details

Forgot password? Click here to reset