Each model was assessed using the same cross-validation splits as described above. Subsequently, we created a dataset employing a subset of the recently published JUMP-CP dataset13, in combination with activity data from ChEMBL14. This relationship can be learned by a deep learning model using small sets of single-concentration activity readouts. The performance of the model, averaged across the six folds, was measured at a ROC-AUC of 0.744 ± 0.108.
Diversity evaluation
Biochemical assays discussed include fluorescence polarization and anisotropy, FRET, TR-FRET, and fluorescence lifetime analysis. Any submissions which do not use these hosting platforms for images or gifs will receive immediate removal and may result in a ban. Tapping into large internal compound libraries, a vast wealth of knowledge and a wide range of technology platforms can reduce time and cost and increase the likelihood of success without a major upfront investment in infrastructure and personnel. HTS could also serve as an engine to generate high quality big data sets to build AI/ML models. Data generated can be used to guide future compound selection, hit expansion and early Structure-Activity Relationships (SARs). Evotec offers an extensive range of different technologies and platforms for hit confirmation and validation.
- Employing bootstrapping, we consistently observed enrichment ranging from a moderate 1.6-fold to a high 14-fold, correlating with the predictive performance of the assays.
- Small scale screens, n ~ 104 for a target of interest are then used to generate activity readouts for a set of compounds used to train a machine learning model (green active, blue inactive).
- Thus, the activity of many of the compounds are unknown and no loss signal backpropagated from those neurons.
- Cells were treated with compounds at 10 μM for 48 h and then stained with 500 nM MitoTracker (ThermoFisher, M22426) for 30 minutes.
- We selected a structurally diverse set of 8,300 compounds to be representative of a larger HTS screening library.
Access a Diverse Well Curated Compound Library for HTS
Specifically, D shows a hierarchical clustering of the expression of untreated controls against treated with the 4 unknown compounds. Large established CROs such as Evotec have state-of-the-art infrastructure already in place for HTS as well as access to multidisciplinary teams. High throughput screening is performed at an early phase in the lead discovery process.
The trained model is then used to predict the bioactivity of the compounds in the entire compound library, enabling the selection of compounds most likely to modulate the intended target (Fig. 1a). These models rely on compound structure information to make predictions of compound activity on a particular target. Drug discovery campaigns typically rely on high throughput screening (HTS) for hit finding i.e., the process of identifying and selecting chemical compounds with biological activity towards a target and the potential to be developed into a drug. In many cases, the high prediction performance can be achieved using only brightfield images instead of multichannel fluorescence images.
Follow-up screening and calculation of enrichment
In this context, each activity label is based on a single data point, representing a unique compound at a specific concentration in a single microwell from an HTS screen. For each compound we also extracted corresponding single-point bioactivity data from the AstraZeneca HTS database. A The envisioned approach utilizing phenotypic screening for bioactivity prediction.
This allowed us to select the top compounds from the full dataset without data leakage. To assess the diversity of the top ranked compounds according to each predictive model, the top 20 ranked compounds in each test set were compared to the known actives in their respective training set. Performance variations between model types were analysed using Friedman rank sum test, using the assays as blocking factors. The ECFP4 representation was done using 1024-dimensional bit string, which was used as input to the activity prediction model. The identified hyper-parameters were then used to train MLP for 150 epochs, using early stopping based on the validation ROC-AUC performance and learning rate dropping on plateau. Due to the more manageable size and computational requirements of the feature-based model, a larger hyper-parameter tuning was done using nested cross-validation in each of the cross-validation splits.
Unlocking High-Throughput Screening Strategies
Because the initial screening assays are often very simple representations of the target biology, they run the risk of producing false positive and negative results. Because of this, hit finding is generally done with simple assays such as biochemical assays to enrich the compound set before more resource-intense assays can be used further down the cascade. Accurate bioactivity prediction using morphological profiles could streamline the process, enabling smaller, more focused compound screens. Another important aspect of cell-based HT assays is the response of the organism of interest through the primary screen. On the other hand, cell-based assays discussed include viability, reporter gene, second messenger, and high-throughput microscopy assays.
- The compounds found in the JUMP-CP dataset were cross-referenced with those available with activity Potency readouts in the ChEMBL database (version 33).
- A single High Content Imaging screen using Cell Painting is used to generate phenotypic representations of each compound in the compound library m ~ 106.
- The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material.
- The ROC-AUC reported in our experiments was computed using only the randomly selected compounds to keep the values comparable with the primary assay.
- The majority of the top-ranked 5% of compounds were randomly sampled and included in the follow-up assay, along with selection of compounds selected uniformly at random (at least 1000 compounds in total).
A hyper-parameter search was performed using nested cross-validation in each one of the cross-validation splits, using three splits for training, one for nested validation and one for nested testing. All models were trained on two NVIDIA-Tesla 32 Gb GPUs, using pytorch DDP38. Area Under the Receiver Operating Characteristic Curve (ROC-AUC) was used to evaluate the model’s ability to separate the actives from inactive. The models were trained with Binary Cross-Entropy combined with Focal Loss37.
Expansive and High-Quality Compound Collection for HTS
We validated the model’s performance in follow-up experiments in secondary assays. These types of data are relatively inexpensive to produce and can therefore readily be used as a basis for training the prediction models for targets. Furthermore, the enrichment levels we observed were high enough to reduce cost and speed up the screening process by filtering in silico compounds according to the ones the model predicts to be active. In summary, the assays probed in our follow-up experiments showed that the model performance was conserved through replication, even perhaps slightly better than what was expected.
To validate the reproducibility of our approach, we applied the same predictive framework to two publicly available datasets. This practice is commonly performed in Structure Activity Relationship (SAR) modeling. The dark blue line represents the average ROC-curve, the shaded area represents the standard-deviation intervals and the faded lines ROC-curves of individual cross-validation splits. A single High Content Imaging screen using Cell Painting is used to generate phenotypic representations of each compound in the compound library m ~ 106. Phenotypic profiles can encompass a wide range of biological responses, including changes in cell morphology, proliferation, gene or protein expression, and physiological functions. A positive control in this type of vegas casino download assay ensures that the population will not grow.
Broad Institute of Harvard, Chemical Biology Program, Massachusetts Institute of Technology, Cambridge, U.S.A.
Given the fact that not all compounds have been tested in all assays, the label matrix is incomplete. Using the activity flag field binary labels were assigned for each compound-assay datapoint. The compounds found in the JUMP-CP dataset were cross-referenced with those available with activity Potency readouts in the ChEMBL database (version 33). These have been imaged following the protocol described in13, using U-2 OS cells, treated with compounds at 10 μM.
Our image-based model on average outperforms a structure-based model. Concurrent work has shown that cell feature-based model performance is often comparable23 or slightly superior24, aligning with our observations. There has been limited work comparing models based on chemical structure and those based on imaging data22. These early studies employed binary activity data derived from dose-response curves, expressed as pXC506 or IC50/EC508 of the given compound in the given assay. Previous work by Simm et al. and Hofmarcher et al. established the information link between phenotypic screening data and assay activity6,8.
Our analysis revealed that compounds predicted from images showed lower structural similarity i.e., greater chemical diversity, than structure-based approaches. Although the brightfield image-based approach was outperformed by the fluorescence-based approach, it was still able to predict 49% of the assays with a ROC-AUC above 0.7 and even 5% above 0.9. This dataset included 209 assays comprising 10,574 compounds8,12, where binary activity data was derived from dose-response curves (IC50/EC50) of each compound in a given assay. Initially, we assessed our framework’s performance on a dataset established by Hofmarcher and colleagues8, demonstrating end-to-end learning with convolutional neural networks (CNNs) for biological assay prediction from Cell painting images. Our results demonstrate the capability of models trained on phenotypic data combined with a few hundred single-concentration data points, to predict compound activity reliably and efficiently across diverse targets in a realistic drug screening scenario. As only a few hundred activity data points are needed to train the predictive model for a particular target and assay, assays of higher complexity and biological relevance could potentially be used.
This shows that the information captured in brightfield images can be linked to bioactivity in a wide range of assays and targets, which may justify using brightfield images in some cases despite their slightly inferior performance. The fluorescence and brightfield images were used to train ResNet50 models, while the cell-features and structure-based data were used to train multi-layer perceptrons (See Materials and Methods for details). The performance difference between structure- and image-based model is significant for cell-based assays but does not reach significance for biochemical assays. While structure-based bioactivity prediction is attractive as it requires no in vitro data, alternative input representation can avoid the problems SAR models have with scaffold hopping and increasing the diversity amongst predicted hits.
The colors -red, white, and blue in the heatmap represent the relative expression levels as high, medium, and low respectively. 106c.f.u per mL were transferred in a 96-well microtiter plate and the primary screening was performed in the epMOTION 5075. The dose-response curve may inform us about the efficacy of a library compound or an unknown small molecule. If screening is performed for the inhibitory effect, then positive controls would consist of a molecule that is lethal to the bacterium of interest. When designing experiments for small molecule screening, appropriate controls are required.
The randomly sampled compounds were also used to assess the baseline hit rate for each of the assays, which was used for the enrichment analysis. The likelihood of activity for each of the compounds in all six test-splits were combined and ranked together, based on the prediction of the Cell Painting Fluorescent Whole Image ResNet50. In each of the available follow-up assays a baseline estimate of assay activity was established by probing at least 500 randomly sampled compounds. In addition, a random subset of compounds was also sampled for follow-up screening and used to calculate ROC-AUC metrics in the follow-up assays. A model was trained for each of the cross-validation splits with the best performing model checkpoint, based on validation-set performance, being used to predict the likelihood of activity for the respective test set split. A model was trained for each of the cross-validation splits, with the best performing model checkpoint based on validation-set performance being used to predict the likelihood of activity for the respective test set split.
Compounds that were structurally similar, based on ECFP-4 clustering, were assigned to the same fold to measure the ability of the model to identify actives in unknown regions of the compound space. We selected a structurally diverse set of 8,300 compounds to be representative of a larger HTS screening library. Phenotypic profiles are derived from cells, tissues, or even whole organisms, and contain information on the characteristics or behaviors of these complex biological systems in response to perturbations with small molecule compounds or other drug modalities.