Session Z18: Data science, AI, and machine learning in physics II

Optical label-free determination of mitochondrial dynamics using deep learning

Association of mitochondria and cancer growth has been a topic of interest for researchers over many decades [1-2]. Understanding the changes in the structure and function of mitochondria in cancer cells typically involves electron and fluorescence microscopies. These methods, although widely adopted in the biomedical field due to the high resolution of electron microscopy and high specificity of fluorescence microscopy, are still not well suited for long term observation of live cells as electron microscopy involves destructive sample preparation and fluorescence microscopy carries inherent risk of phototoxicity and photobleaching. Here, we present an optical, label-free, deep learning enabled mitochondria detection technique for live mammalian cells and demonstrate the applicability of the proposed method in characterizing the dynamics of mitochondria in HeLa, Neuroglioblastoma, and CHO cells. We observed that as compared to the whole cell, mitochondria are more deterministic in their dynamics as indicated by a statistically significant reduction of the diffusion coefficient between them.

1. Kroemer, G., 2006. Mitochondria in cancer. Oncogene, 25(34).

2. Vyas, S., Zaganjor, E. and Haigis, M.C., 2016. Mitochondria and cancer. Cell, 166(3), pp.555-566.   

Longitudinal Interpretability of Deep-Learning based Breast Cancer Risk Prediction Model

When developing models intended for clinical applications, understanding which part of the input contributed the most to the final decision is crucial. Our study brings interpretability to a Breast Cancer Risk (BCR) prediction by exploring whether the Deep Learning (DL) model relies on the laterality of the breast, where cancer ultimately develops, and how this reliance evolves.

The Mirai DL model was employed for BCR predictions using a dataset containing 1210 mammography studies of patients who developed cancer within six years. Each study included four images (two views per breast). For each BCR prediction, gradients with respect to each pixel of the four input images were calculated. The attribution (importance to model prediction) across both views for each breast was summed. It was assumed that the cancer would form in the breast with higher relative attribution. ROC-AUC analysis was performed for six-time points (during cancer and each year before cancer from 2y-6y).

Results showed that the model heavily relies on the attribution from the breast where cancer is already detectable (AUC=0.92±0.02). For the studies where the cancer was not yet present, the high AUC for 2y (0.91±0.07) decreased monotonically to 0.51±0.15 for 6y before cancer.

This interpretability research suggests that the model's predictions for time points closer to cancer occurrence primarily rely on the signal derived from the breast where cancer will form. In contrast, the relative difference is random for more distant time points.   

Assessing the impact of CNN architectures for whole organ segmentation on predictive models of organ toxicity

Segmentation of disease and critical structures from medical images is a critical task that enables development of predictive models of treatment response and treatment-related toxicities. Convolutional neural networks (CNN) are often used for this task. However, the impact assessment of CNN segmentation model architectures on predictive models’ performance is incipient. Here, we perform such assessment on a ¹⁸F-FDG PET histogram metrics-based model for predicting organ inflammation.

Two CNN architectures (DeepMedic, nnUNet) were employed to segment bowel, lungs and thyroid on the CT scans of melanoma cancer patients; from which the PET signal indicative of organ inflammation was extracted. This signal was used to predict organ toxicity via classical statistical and machine learning models. Model performance was assessed using area under the receiver operating characteristic curve. Model’s sensitivity to CNN architecture was analyzed.

Dice similarity coefficient of organ segmentation was 0.96±0.06 (mean±sd) in bowel, 0.87±0.07 in lungs and 0.61±0.16 in thyroid accounting for differences in different CNN architectures. Different CNN architectures had no significant impact on prediction of organ toxicities (z-test, p>0.05).

Our findings suggest that PET-derived, segmentation-based organ toxicity biomarkers are robust against different CNN architectures.   

Spatially Resolved IR Hyperspectral Imaging for Malignant Cell Detection

To systematically investigate the IR spectral alterations associated with carcinogenesis and tumor progression, we initiated a research program to spatially resolve chemistry from IR hyperspectral imaging of individual cells. When an instrument measures the missing light at the detector, the generated IR spectrum is a consequence of both absorbances and scattering of the particle. This phenomenon often manifests as intense baseline profiles, fringes, band distortion, and peak position and intensity changes. Consequently, a recorded spectrum hardly reflects the pure absorbance that carries the molecular composition of the cell. Instead, the obtained IR spectrum is a superposition of the pure absorbance of the molecules and signals from scattering as well as other optical phenomena that are dependent on the geometric cross-section of the sample and the wave nature of light. Since the goal is to determine the internal structure of cells, we want to ensure that the scattering contribution to the measured absorbance is removed so that only the pure absorbance of the functional groups is recovered. It then becomes necessary to find ways to extract the space-dependent complex index of refraction from the measured absorbance spectra and, by extension, solve an inverse scattering problem.   

Radiomics assisted machine learning model for predication of prostate specific antigen levels

We proposed a machine learning model to predict prostate specific antigen (PSA) levels for intermediate or high-risk prostate cancer patients undergoing definitive treatment course. 

For this purpose, a data set consists of 100 localized prostate cancer patients treated with a combination of radiation therapy and Androgen Deprivation Therapy (ADT). Pretreatment imaging data including Magnetic Resonance Imaging (MRI), Computed Tomography (CT), Positron Emission Tomography (PET/CT), and Prostate Specific Antigen (PSMA) scans were used to extract the radiomic features with reference to follow-up PSA levels.   

Patient's demographics, tumor stage, Gleason score, histopathology, radiation therapy dose, immunotherapy dose, pre and post treatment PSA levels along with significant radiomic features are used in training of machine learning model.  

We expect this model will help clinicians in predicting early prognosis along with treatment intensification options (if required) for localized prostate cancer patients.    

Deep learning for image analysis of breast and prostate cancer cell cultures

We present a machine-learning analysis of phase-contrast microscope images of breast cancer (MDA-MB-231) and prostate cancer (PC3) cell cultures. Semantic segmentation (cell detection) was performed using several variants of the U-net [1] convolutional neural network architecture. Best results were obtained using an Attention U-net [2] with a binary focal loss function. Instance segmentation (cell labeling) was performed using the watershed method. Geometrical properties of each cell (area, solidity, and eccentricity) were computed and their statistics were plotted as a function of time, in order to quantify cell growth under different conditions.   

Structure prediction of iron hydrides across pressure range with transferable machine-learned interatomic potential

Recently, machine-learned interatomic potentials (ML-IAPs) have emerged as a solution to the computational limitations of density functional theory (DFT)-based approaches, enabling the modeling of large systems with hundreds or even thousands of atoms. Here, we demonstrate the efficacy of automated and systematic methods for training and validating transferable ML-IAPs through global optimization techniques.

We utilize the PyFLAME code [1] to construct a highly transferable neural network potential. With this accurate and fast potential, we systematically investigate the potential energy surfaces (PESs) of FeH through global sampling using the minima hopping method [2] over a wide range of pressures. This comprehensive exploration enables us to predict stable and metastable iron hydrides from 0 to 100 GPa.

Our analysis reveals the experimentally observed global minimum structures -the dhcp, hcp, and fcc phases- in agreement with previous studies. Furthermore, our exploration of the PESs of FeH at various pressures uncovers numerous interesting modifications and stacking faults of the aforementioned phases, including several remarkably low-enthalpy structures.

This investigation led to the discovery of a rich array of novel stoichiometric crystal phases of FeH across a wide pressure range, confirming the presence of coexisting regions containing known FeH structures. This finding demonstrates one of the benefits of using large-scale structure prediction techniques to uncover the PESs of materials.

[1] H. Mirhosseini, H. Tahmasbi, S. R. Kuchana, S. A.  Ghasemi, and T. D. Kühne, Comput. Mater. Sci. 197, 110567 (2021).

[2] M. Amsler and S. Goedecker, J. Chem. Phys. 133, 224104 (2010).   

Development of a Machine Learning Interatomic Potential for Uranium Nitride

Machine learning interatomic potentials (MLIPs) have become widely popular due to their high accuracy, similar to first-principles calculations, at a low computational cost. Recently, we have demonstrated that MLIPs can be very useful to investigate the atomic dynamics of actinide compounds used as nuclear fuels. In this study, we have extended previous work by developing an MLIP for uranium mononitride (UN). We employ an on-the-fly active learning sampling approach to iteratively improve our MLIP by generating an optimal and informative first-principles dataset. We utilize the MLIP to calculate important physical quantities, such as specific heat, bulk modulus, and relevant point defect energies with high accuracy in comparison to first-principles and experimental data. 

LA-UR-23-31944   

Fast Generation of Ab Initio Training Data for Large-Scale Applications of Neural Network Potentials

Fast, accurate calculation of phonon dispersion in large crystal systems proves to be an ongoing challenge due to cubic scaling of total energy calculation using traditional ab initio methods and lack of sufficient accuracy using empirical force field methods. Neural network potentials (NNPs) have recently shown great promise in speeding up the computation, but generating a high-quality training dataset for NNPs has involved taking snapshots from ab initio molecular dynamics simulations, which can take large computational resources by itself, bringing viability of developing and using NNPs for studying phonon properties into question. We propose a method to quickly generate a dataset to train a NNP tailored to perform well on the target system of interest. Taking a regular AB-stacked bilayer graphene unit cell containing 4 atoms, we systematically perform translation, supersizing, and random displacement of atomic positions to generate O(10⁴) small structures, whose total energies can be calculated in an embarrassingly parallel fashion. Using this dataset, we show a good performance in computing total energies and interatomic forces for twisted bilayer graphene structures compared to previous NNP models.   

Graph-Transformer Model for Direct Band Structure Prediction from Crystal Structures

Predicting electronic band structures from crystal structures is essential for efficient materials discovery and design. While previous machine learning models used crystal graph neural networks for single-valued property predictions (such as CGCNN and MEGNet) and (equivariant) graph convolutional neural networks for Hamiltonian estimates (such as DeepH and DeepH-E3), our work introduces a novel approach to learn electronic structures directly from atomic crystal structures. We developed the first end-to-end model that directly predicts band structures from crystal structures. The model combines a crystal graph transformer as an encoder to capture the complex patterns within the crystal and a graph2seq layer as a decoder to convert the encoded crystal information into a sequential representation of the electronic band structures. Our results showcase the ability to accurately predict two bands close to the Fermi level for each crystal. Beyond its current application, our framework can be extended to predict additional energy bands and large-scale crystal systems, offering a faster alternative to traditional density functional theory calculations and enhancing the efficiency of large-scale functional materials discovery and machine learning tasks.   

Global structure optimization and metastable structure enumeration using polynomial machine learning potentials

Machine learning potentials (MLPs) have become indispensable tools for performing efficient and accurate large-scale atomistic simulations and crystal structure predictions. The polynomial MLPs described by polynomial rotational invariants have also been systematically developed for many elemental and alloy systems, and they are available in the Polynomial Machine Learning Repository [1]. We show a procedure for performing structure enumerations, including global structure optimization, accelerated by the polynomial MLPs. The polynomial MLPs are developed from datasets generated from various structures and are accurate for many local minimum structures. However, the MLPs exhibit some ghost local minimum structures and fail to predict the energy for some local minimum structures accurately. Therefore, we iteratively repeat random structure searches and update the MLPs using the density functional theory datasets where structures predicted to be local minima are appended. The current procedure is systematically applied to structure enumerations in the elemental systems of As, Bi, Ga, In, La, P, Sb, Sn, and Te with many complex metastable structures [2] and the alloy systems of Al-Cu [3] and Cu-Ag-Au systems. The current procedure would accelerate global structure searches and expand their search space significantly.

 

[1] A. Seko, J. Appl. Phys. 133, 011101 (2023), https://sekocha.github.io

[2] A. Seko, in preparation.

[3] H. Wakai, A. Seko, and I. Tanaka, J. Ceram. Soc. Jpn., 131, 762 (2023).   

Developing generalizable machine learning models using electronic structure-based features

Machine learning has been widely applied to predict molecular properties (i.e., total energy), by utilizing the patterns and relationships between a system's features and the desired property. Developing generalizable machine learning models capable of making accurate predictions on data not included in the training process requires features with a clear and meaningful connection to the desired molecular property. Recently, features that incorporate information beyond molecular geometry, to include a system's underlying physics, have been demonstrated to result in highly generalizable machine-learning models. In this study, we explore the construction of a feature space that includes the underlying physics of a system by using information obtained from computationally affordable electronic structure calculations, such as Hartree-Fock. We then assess these electronic structure-based features by training a neural network to predict ab initio (i.e., CCSD) formation energies. This research aims to determine if these types of features result in machine learning models that are both accurate and generalizable.   

Accurate Prediction of Magnetic Properties of Permanent Magnets Using Machine Learning

Discovering permanent magnets that do not rely on costly rare-earth elements yet exhibit

performance on par with the current leading neo-magnets is a pressing challenge for scientists

to fulfill the skyrocketing demand for high-performance magnets in electric automotive

industries. Theoretically, this endeavor involves predicting intrinsic and extrinsic magnetic

properties to identify optimal materials. While traditional ab initio density functional theory (DFT) proves

useful in calculating properties like saturation magnetization, magnetic anisotropy, and the

Curie temperature for simpler systems, we cannot compute coercivity. Moreover, the

theoretical determination of macroscopic coercive properties is poor, largely due to Brown's

paradox. To address this limitation, we employ DFT by

incorporating machine learning (ML) to synthesize experimentally measured magnetic properties and utilize micromagnetic

modeling. This innovative ML methodology enables the precise and accurate prediction of

macroscopic magnetic properties, including the coercivity. The approach is verified on Ce-doped Nd₂Fe₁₄B, and the predicted coercivities are

compared with available experimental data.   

Uncertainty Quantification for Deep Learning-based Metastatic Tumor Delineation on 68Ga-DOTATATE PET/CT Images

Deep learning (DL) based metastatic tumor delineation is an important tool for the automatic evaluation of advanced malignancies. Although recent studies have demonstrated high accuracy in this task, none have quantified the uncertainty of DL predictions, which is critical for clinical use. In this work, we compare DL uncertainty quantification (UQ) methods for metastatic tumor delineation.

A U-Net DL model was trained to delineate tumors on 59 ⁶⁸Ga-DOTATATE PET/CT images using 5-fold cross validation. Three UQ methods were implemented during model inference: Monte Carlo dropout (MCDO), test time augmentation (TTA), and deep ensembles (DE). UQ methods were compared using false positive (FP) filtering performance and correlations with delineation metrics, including Dice coefficient (DSC), cross entropy (CE), and biomarker extraction accuracy (uptake mean and total).

Each UQ method achieved statistical significance between true positive (TP) and FP predicted regions (p<0.001). AUCs±95%CI between FP and TP regions were 0.79±0.02, 0.81±0.02, and 0.80±0.02 for MCDO, TTA, and DE, respectively. Correlations between delineation metrics and uncertainty were ρ_DSC=-0.66, -0.73, -0.70; ρ_CE=0.87, 0.92, 0.93; ρ_mean=0.33, 0.34, 0.28; ρ_total=0.50, 0.57, 0.52 for MCDO, TTA and DE, respectively.

The TTA method demonstrated superior performance across the majority of UQ evaluations. Conveniently, this method also offers the advantage of lower computational demand compared to other methods.