Empowering Research in Chemistry and Materials Science through Intelligent Algorithms

中文 / Bilingual · English version

Jinglong Lin ${}^{a}$ , Fanyang Mo ${}^{a, b, * }$

${}^{a}$ School of Materials Science and Engineering, Peking University, Beijing 100871, PR China

${}^{\mathrm{b}}$ Al for Science (AI4S)-Preferred Program, Peking University Shenzhen Graduate School, Shenzhen 518055, PR China

A R T I C L E I N F O

Keywords:

Intelligent Algorithms

Machine Learning

Deep Learning

A B S T R A C T

In this review, we explore the integration of intelligent algorithms in chemistry and materials science.We begin by delineating the core principles of Machine Learning, Deep Learning, and optimization algorithms, highlighting their bespoke adaptation to these scientific domains. The focus then shifts to the critical processes of data management, including collection, refinement, and feature engineering, alongside strategies for efficient data mining from targeted databases and literatures. Subsequently, we present a concise overview of the diverse applications of these algorithms, emphasizing their transformative impact in both fields. Finally, this review explores the future prospects and challenges of these emerging algorithms.

In the domain of intelligent algorithms, Machine Learning (ML), Deep Learning (DL), and other Optimization Algorithms (OA) form the computational paradigms proficient in harnessing data for sophisticated tasks such as decision-making, prediction, and optimization (Fig. 1). In an era witnessing an explosion of chemical information and advancing computational prowess, data-driven scientific discovery emerges as an imperative research paradigm [1-4]. However, tackling complex systems frequently incurs the ‘curse of dimensionality’, necessitating substantial computational resources [5]. Fortunately, intelligent algorithms offer a potent toolkit for fitting and solving high-dimensional functions, thereby aiding the resolution of mathematical models within intricate systems. Moreover, the multi-scale nature of chemical modeling (micro-meso-macro) challenges the investigation of structure-activity relationships. Notwithstanding these complexities, intelligent algorithms serve as crucial aids in cross-scale modeling, learning the mappings between micro and macro scales, even in instances where these relationships may outstrip our current theoretical grasp.

In the realms of chemistry and materials science, challenges can be broadly categorized into three domains: substance screening, elucidation of mechanisms, and process optimization (Fig. 2). This review delves into the role of intelligent algorithms in solving these challenges. Beginning with the foundational theories of intelligent algorithms, we aim to enrich the reader’s understanding of their workings. We subsequently delve into data acquisition and processing methodologies from diverse sources for algorithmic consumption. Following this, we elucidate the distinct applications of various intelligent algorithms within the chemistry and materials science domains, supplementing these explorations with specific use-cases.

This article aspires to illuminate current trajectories and provide foresight into future developments in applying intelligent algorithms within the chemical domain. Antecedent reviews have typically centered on the intrinsic challenges of chemistry, yet herein we adopt a distinct vantage, threading our narrative with a suite of algorithms and scrutinizing their pertinence to the multifaceted quandaries of the discipline. We hope this review is both informative and inspirational for researchers immersed in the field, as well as academics and engineers aspiring to venture into this dynamic domain.

1. Theoretical basis

1.1. Basics of ML and DL

ML and DL, integral parts of intelligent algorithms, depend on data-driven learning to enable decision-making and predictions. ML branches into supervised learning-employing models like linear regression [10], decision trees [11], support vector machines [12] for predictions based on labelled data, unsupervised learning-utilizing techniques like clustering and principal component analysis [13] to discern structures in unlabeled data, and reinforcement learning-learning through environmental interaction to maximize a reward function [14]. Most ML algorithms can be seamlessly implemented via the scikit-learn Python package [15].

• Corresponding author at: School of Materials Science and Engineering, Peking University, Beijing 100871, PR China.

(F. Mo).

(F. Mo)。

DL, a subfield of ML, attempts to mimic the human brain’s operations, employing Artificial Neural Networks (ANNs) [16] to tackle complex, abstract problems. DL models, such as Convolutional Neural Networks (CNNs) [17] and Recurrent Neural Networks (RNNs) [18], lean heavily on abundant data and computational resources. The training phase of DL typically involves loss function calculations, optimizing model parameters by minimizing this loss function via optimizers. Popular DL frameworks include PyTorch [19], TensorFlow [20], PaddlePaddle [21], MindSpore [22], Keras [23], among others.

In the realm of chemistry and materials, ML and DL enjoy extensive applicability. ML algorithms, for instance, can predict chemical reaction yields and selectivity [24], design novel chemical synthesis routes [25], and facilitate high-throughput material screening [26]. DL, on the other hand, is apt for handling more intricate issues, such as predicting protein three-dimensional structures [27] and analyzing material microstruc-tures [28].

1.2. Basics of optimization algorithms

As a crucial subset of intelligent algorithms, optimization algorithms find extensive utility across scientific and engineering disciplines. Their prime objective is to uncover optimal solutions for a given function. This is achieved through a myriad of strategies including, but not limited to, Simulated Annealing(SA) [29], Genetic Algorithms(GA) [30], Particle Swarm Optimization(PSO) [31], etc. These techniques underscore the diversity and adaptability of optimization algorithms in diverse problem-solving scenarios.

Iterative search methods are a key tool and core mechanism in optimization algorithms. For instance, SA models the annealing procedure in solids to determine the lowest energy state. GA utilize biological evolutionary mechanisms, engendering optimal solutions via repeated selection, crossover, and mutation. PSO drawing upon flock or fish social behaviors, modifies each “particle’s” position and velocity to locate an optimal solution. These optimization algorithms find diverse applications: SA addressing protein folding problems to predict natural protein conformations [32], GA optimizing chemical reaction networks to boost product yields [33], and PSO showing promise in predicting the structure of layered materials [34]. As a whole, these algorithms have catalyzed substantial advancements in fields such as chemistry and materials science, promising substantial potential for future breakthroughs.

Fig. 2. Challenges in the chemistry and materials science. Three categories of challenges: substance screening, mechanism exploration, and process optimization. For substance screening, such as drug virtual screening [6], material high-throughput screening [7], etc.; for mechanism exploration, such as kinetic mechanism exploration [8], structure-activity relationship exploration [9], etc.; for process optimization, such as synthesis route optimization, reaction conditions optimization etc.

Fig. 1. Classification of intelligent algorithms. Intelligent algorithm is composed of ML, DL, and OA. Notably, DL, rooted in neural network architectures, constitutes a subset of ML. Representative algorithms for these domains are depicted in the accompanying figure.

1.3. Adapting intelligent algorithms to the specific demands of chemistry and materials science

ML, DL, and optimization algorithms each exhibit unique problem applicability, strengths, and limitations. The choice of algorithm should align with the specific characteristics and demands of the problem at hand, considering factors such as data type (labelled or unlabeled), problem complexity, and available computational resources.

ML leverages statistical methods to discern and predict underlying data patterns, particularly for problems with clear labels such as reaction activation energies prediction [35] and molecular property prediction [36]. Its advantages include lucid data distribution interpretation and effective input-output relationship modelling. However, ML requires domain expertise and manual feature engineering when grappling with complex, high-dimensional, non-linear problems. In juxtaposition, DL manifests as a powerful tool when confronted with tasks that demand automated extraction and deciphering of inherent features directly from raw data, a scenario exemplified by molecular representation learning [37]. Despite DL’s prowess in recognizing patterns and condensing features within high-dimensional data, it invariably necessitates substantial datasets and computational horsepower for peak performance. Transitioning focus, optimization algorithms chiefly address the search for the best solutions amid a sea of possibilities. The allure of these algorithms stems from their efficiency and global search capacity in pinning down the optimal solution. However, the practical efficacy of these tools often is inherently tethered to the specifics of a given problem, such as the morphology of the objective function and the dimension of the search space.

Additionally, the requirements of chemistry and materials science, as compared to generic modelling scenarios, manifest through several key considerations: the high dimensionality of data; the inherent structural intricacy of data, such as molecular configurations and crystal arrangements; the necessity for domain-specific expertise; the presence of data uncertainties and noise; multi-scale complexities; and a focused attention towards model transparency and interpretability, given its implications for scientific discoveries. Accordingly, the design and adjustment of intelligent algorithms should be specifically tailored to these challenges, to ensure their aptness for addressing such multifaceted requirements.

2. Data collection and preprocessing

Intelligent algorithm research pivots on three pillars: data acquisition, feature engineering, and algorithm selection (Fig. 3). Standardized data and feature engineering form the bedrock upon which intelligent algorithms operate, enabling them to effectively sift valuable insights from the labyrinth of complex data, construct precise models, and execute robust predictions.

2.1. Database and data extraction

Numerous chemical databases provide researchers with vast avenues for information acquisition. Here, we briefly encapsulate several prevalent chemical materials databases. PubChem [38], proffered by the NCBI (National Center for Biotechnology Information) in the U.S., is a freely accessible database of vast quantities of compounds, substances, and bioactivity information. ChemSpider [39], an online chemical database furnished by the Royal Society of Chemistry, amalgamates information on over 67 million chemical substances. ChEMBL [40], a database dedicated to drug development, encompasses a plethora of drug compounds, targets, and bioactivity data. ZINC [41], a chemical database designed specifically for virtual screening, contains a multitude of purchasable compounds. The CSD (Cambridge Structural Database) [42] stands as the most comprehensive repository of known organic and metal-organic crystal structures worldwide. Each of these databases furnishes Python APIs for users to extract data in bulk, further facilitating their utility. The iBonD [43] is a user-friendly internet-based databank of heterolytic (pKa) and homolytic (BDE) bond dissociation energies, established by the bond energy team at Tsinghua University and Nankai University, China. The iBonD 2.0 provides the heretofore most comprehensive bond energy collection and the most convenient approach to find the data with its powerful searching engine. The ORD (Open Reaction Database) [44] stands as a comprehensive archive, brimming with detailed chemical reaction information, thereby serving as a foundational data source for practitioners of synthetic chemistry.

The MP (Materials Project) [45], one of the most commonly used materials databases, provides comprehensive data on materials, encompassing theoretical calculations of crystal structures, electronic structures, magnetism, and more. Pymatgen [46], a powerful open-source Python library for materials analysis, proffers highly adaptable classes for representing elements, sites, molecules, and structural objects, along with rich input/output support, encapsulating file formats such as VASP, ABINIT, CIF, among others. Researchers can leverage Pymatgen for seamless integration with the Materials Project REST API. The ASE (Atomic Simulation Environment) [47] is a Python library used for setting up, manipulating, running, and analysing atomic simulations, allowing researchers to perform quantum mechanics and molecular dynamics simulations with greater ease. The CMR (Computational Materials Repository) [48], a multi-topic materials database site based on ASE, includes databases on two-dimensional materials, functional perovskites, and more. The OQMD (Open Quantum Materials Database) [49] is a materials database predicated on first-principle calculations, providing a multitude of information on structures, band gaps, thermodynamics, electronic properties, and more.

Fig. 3. Three elements of intelligent algorithm research. Intelligent algorithm research pivots on three pillars: data acquisition, feature engineering, and algorithm selection. Data can be sourced from databases, literature mining, simulations, or high-throughput experiments. Effective descriptor design, fortified by domain knowledge, is paramount in feature engineering. Algorithm selection hinges on data volume and problem specificity, with potential enhancements from model architecture or physical constraints.

2.2. Leveraging literature: data extraction and utilization

In the realm of chemistry, the literature landscape is replete with a wealth of original data, rendering it a vital artery for researchers on their quest for data resources. Yet, the manual extraction of data from such extensive literature archives proves to be a daunting task, both laborious and time-consuming. The intensity and speed of this manual task scarcely align with the rapid proliferation of literature volumes. More critically, the non-standardized and unsystematic nature of manual data extraction impedes effective data sharing and unified utilization, thus presenting significant challenges. Given this context, the deployment of Natural Language Processing (NLP) [50] technology to automate the dissection and extraction of data from chemical literature is gaining prominence. NLP, with its goal of equipping computers with the capacity to comprehend and apply human language, offers efficient parsing and abstraction capabilities for text, thus paving the way for large-scale data mining of chemical literature.

In the domain of text mining, several crucial steps are involved, including the batch download of documents, which is then followed by NLP tasks such as preprocessing, text classification, table parsing, named entity recognition, and entity relationship extraction (Fig. 4). A paramount task in scientific research is the batch downloading of literature. In response to this, an array of methods has been designed to automatically acquire scientific articles based on Digital Object Identifiers (DOIs), largely harnessing publisher-provided API interfaces like Elsevier’s Scopus and ScienceDirect APIs, Springer Nature’s Spring-erLink API, and Wiley’s Online Library API, among others. Once the documents are downloaded, they undergo preprocessing to exclude nontarget text and ensure data consistency and accuracy. HTML and XML files are parsed, often employing libraries such as Python’s Beau-tifulSoup [51] or lxml [52], to extract text effectively between tags. Following the extraction, related text blocks are merged for coherence, and non-target metadata such as article numbers, and publication information are filtered out through specific rules or regular expressions. Furthermore, to avoid discrepancies that might hamper subsequent data analysis and information extraction, it’s crucial to standardize the format of data units, such as temperature, pressure, and concentration units. This is typically accomplished using mapping dictionaries [53]. The processed text is then divided into words or phrases and subjected to stem extraction or lemmatization to minimize the impact of vocabulary morphological changes on data analysis. This phase generally employs a stem extraction or lemmatization algorithm such as the Porter algorithm [54].

Within the text mining landscape, text classification serves as a pivotal role, filtering out salient information from vast full-text sentences through a meticulous organization based on specific keywords or semantic traits. This process is bolstered by semi-automated dictionaries utilizing word vector techniques such as Word2Vec [55] or GloVe [56], and rule-based recognition methods. Despite the manual rule-setting requirement, the latter demonstrates impressive accuracy for structured text. Table parsing transforms raw XML and HTML table data into structured cell arrays. It begins with extraction of table content using explicit tags, resulting in nested lists representing table rows and cells. Potential challenges, such as cell spanning or data inconsistency, are addressed during this process. Named entity recognition, another crucial element, involves automated identification and categorization of named entities, often aided by regular expressions matching entity expressions in a specific literature field. Finally, entity relationship extraction identifies semantic links between named entities in the text. Rule-based methods, such as dependency parsing [57], offer high precision, but they require substantial manual effort and continuous rule updates to accommodate complex or evolving text structures. Pattern-matching methods, such as those using neural network architectures like Transformer models [58], capture more complex relational patterns but require substantial training data. Convolutional neural networks adapted for graph structured data, such as Graph Convolutional Networks (GCN) [59], are also instrumental in discerning complex patterns and relationships in text. While they can efficiently extract local and global features from the data, their performance relies heavily on the quality and amount of training data available. In conclusion, each method, from rule-based approaches to ML algorithms, carries its unique applications and limitations, necessitating a careful selection based on practical needs.

2.3. Data cleaning and feature engineering

Data cleansing forms the cornerstone of data preprocessing, addressing pivotal concerns like missing data, outliers, and data duplication. For missing values, typical strategies involve excluding data with absent information, employing statistical techniques such as mean, median, or mode imputation, or interpolating using predictive models. Outliers can be managed through their identification followed by elimination, replacement, or through variable transformations. To maintain the uniqueness of the data set, the removal of duplicate entries is recommended. Beyond data cleaning, data transformation refines raw data into forms more amenable to analysis or model construction. These transformations can take multiple forms, including logarithmic conversions, standardization, or normalization. Standardization and normalization, essential facets of data normalization, alter the scale and distribution of data to fit a unified standard, thereby improving the effectiveness of subsequent data analysis and model development. Python’s numpy [60] library offers comprehensive utilities for both data cleaning and transformation.

In the realm of cheminformatics, molecular descriptors act as numerical quantifiers of a vast array of compound-specific structural, physical, and chemical data. These descriptors are broadly categorized into zero-, one-, two-, and three-dimensional classifications. Zero-dimensional descriptors, which include fundamental physicochemical properties such as boiling points, solubility, and pKa, are readily ascertainable and conceptually straightforward. However, their utility is somewhat curtailed by their inability to capture the complexities of advanced molecular structures. One-dimensional descriptors include character encoding methods such as SMILES [61]. A SMILES string consists of a series of characters, including atomic symbols, bond connections, and parentheses to represent branched structures. Venturing into the realm of two-dimensional descriptors, these not only encompass the molecular topology but also a variety of molecular fingerprints, including but not limited to MACCS Keys [62] and Morgan [62], which can be derived from SMILES. Their extensive applicability notwithstanding, they fall short in depicting the three-dimensional conformation and stereochemical features of molecules. The three-dimensional descriptors, including Stereochemical Descriptors, Geometric Descriptors, and Quantum Chemical Descriptors, offer a heightened level of accuracy, delving into the molecular conformation with finesse. However, this precision comes at the price of elevated computational demands and a pronounced sensitivity to molecular conformational variations, necessitating rigorous conformation generation and refinement. In an encouraging development, Liu et al. have introduced Auto3D [63], a notably efficient algorithm facilitating the swift acquisition of molecules’ optimal conformations. The novel geometrically enhanced molecular representation learning method (GEM) proposed by Fang [64] and others is characterized by representing a molecule with two graphs simultaneously: one is an atom-bond graph, and the other is a bond-angle graph. This design effectively addresses the issue of three-dimensional spatial description of molecules. In addition to these previously discussed descriptor types, innovative approaches continue to emerge, such as the spectroscopic descriptors [65] proposed by Jiang et al., harnessing infrared and Raman spectral signals to forecast molecular adsorption-a testament to the ever-evolving landscape of molecular descriptor methodologies. The above techniques are judiciously employed by researchers to encapsulate the dimensional space of chemical reactions [66,67]. However, it is pivotal to note that the aforementioned descriptors, while widely applicable, may not be universally fitting for all problem sets. Hence, it is of crucial importance to design and utilize descriptors that hold a high degree of chemical relevance specific to the problem at hand, manifesting as feature engineering methods embedding domain knowledge. Illustratively, researchers, when employing ML to predict enantioselectivity [68], have developed a novel spatial descriptor, termed as Average Space Occupancy (ASO), recognizing the critical role spatial environment plays in chiral induction. In tandem, they have utilized electrostatic potential as a stereoelectronic descriptor. This strategic approach has yielded exceptional predictive accuracy, with a Mean Absolute Error (MAE) as low as 0.144 kcal/mol.

Fig. 4. General process of text mining. First, batch-download documents via the API. Next, preprocess to eliminate extraneous content. Conduct named entity recognition, isolating desired materials or compounds. Subsequently, correlate these entities with performance descriptors, culminating in database construction.

Table 1

Comparison of Different Molecular Descriptors.

Feature selection stands as a prerequisite to predictive model implementation and bears a significant influence on high-dimensional data handling. Its chief aim is to discern the subset of features most pertinent to the task at hand, enabling effective data dimension reduction, enhanced model efficiency, lower risk of overfitting, and improved model interpretability. Filtering, a foundational feature selection method, relies on statistical characteristics between data features. Filtering methods rank features based on individual feature scores using various statistical techniques. These techniques, such as chi-square tests and ANOVA, employ common criteria such as the Pearson correlation coefficient, information gain, and mutual information. Despite their simplicity and computational efficiency, filtering methods might neglect feature-to-feature interactions. In contrast, the wrapper method, exemplified by algorithms like backward elimination, forward selection, or bi-directional elimination, approaches feature selection as a search problem, using model performance as a measure to determine the quality of feature subsets. Recursive Feature Elimination (RFE) [69], another wrapper method, iteratively builds models and identifies the most and least effective features, incorporating or excluding them from the model. This process continues until all features are exhausted. While wrapper methods can discover feature subsets with superior performance than filtering methods, they come with a higher computational expense. Embedding methods perform feature selection during model training. Certain ML models, such as Decision Tree [70], for instance, assign coefficients or importance scores to features during training, enabling the ranking and selection of the most crucial features. The strength of embedding methods lies in their ability to consider model performance while maintaining relatively low computational costs. Further, in cheminformatics, where feature numbers can dramatically surpass the sample size, sparse representation-based feature selection methods such as Lasso [71], Ridge [72], and Elastic Net [73] prove beneficial.

3. Integrating artificial intelligence in chemistry and materials science

In the following section, we delve into an intensive exploration of several key intelligent algorithms, elucidating their core principles and underscoring their utility within the realms of chemistry and materials science.

3.1. Application of ML algorithms

Random Forests (RF) [74], characterized as a robust supervised learning approach, consist of a multitude of decision trees, each individually trained on different random subsets of the data. During prediction, a consensus is reached through voting on the results from all decision trees, producing the final prediction. The inherent ensemble learning strategy of RF significantly enhances the accuracy and stability of predictions. RF is a fundamentally versatile tool, apt for a wide array of classification and regression challenges within the chemical and materials domains. Remarkably, it often achieves commendable predictive accuracy without the necessity for meticulous parameter tuning. This quality makes RF particularly beneficial in scenarios where the volume of available samples is limited. In a collaborative study by the Doyle Group [75] at Princeton University and researchers at Merck, the RF algorithm was employed to predict the yield of Buchwald-Hartwig coupling reactions (Fig. 5). Initially, they utilized a high-throughput screening platform to rapidly complete 4140 Buchwald-Hartwig coupling reactions, encompassing 15 substrates, 23 isoxazole additives, 4 palladium ligands, and 3 bases. Subsequently, employing domain-knowledge-embedded feature engineering techniques, the researchers extracted attributes from the reaction space through a combination of nuclear magnetic resonance (NMR) data and density functional theory (DFT) computations. This approach yielded a 120-dimensional feature vector. With the reaction yield serving as the label, a sample set was formed. Several common ML algorithms were applied for regression training, among which RF exhibited superior performance, predicting yields with an impressive ${\mathrm{R}}^{2}$ precision of 0.92 . Additionally, the RF model investigated the importance of features in predicting yield. The most significant descriptor was found to be the NMR carbon shift at the C3 position of the isoxazole. Inspired by this revelation, researchers undertook experiments to study reactions with isoxazole additives with varying C3 NMR shifts. They discovered that when the C3 NMR shift of isoxazole was large, an oxidative addition product of Pd and the N-O bond was yielded. Consequently, they concluded that the yield of Buchwald-Hartwig coupling reactions correlated with the competitive reaction of isoxazole and provided a mechanistic explanation. This process of mechanism exploration was indeed sparked by the rule identified by ML. Besides compound yield prediction, RF is also applied in the prediction of NMR chemical shifts [76], pharmacological effect prediction [77], and the forecasting of organic aerosol formation [78], among other areas.

Fig. 5. Intelligent algorithms forecasts the yield of coupling reactions. A) Data encompasses Buchwald-Hartwig cross-coupling reactions, sourced from Merck’s high-throughput screening. B) A 120-dimensional feature engineering approach describes these reactions. C) Among several ML algorithms, the RF model excels. D) Mean square error elucidates feature importance, with isoxazole relating to four of the top five features. Chemical insight based on ML models: Buchwald-Hartwig reaction yield interplays with isoxazole-based competing reactions.

Support Vector Machines (SVM) [12] are a type of supervised learning algorithm predominantly employed for classification problems. Within high-dimensional feature spaces, SVM identify an optimal hyperplane to maximize the margin between two classifications, thereby facilitating optimal categorization. For issues that are linearly inseparable, SVM apply kernel functions to map data onto a higher-dimensional space, enabling non-linear classification. The application of SVM algorithms has been initiated in areas such as materials screening. Rohit Batra and his team utilized SVM for the screening of highly water-stable Metal-Organic Frameworks (MOFs) [79]. The team collected water stability data for more than 200 MOFs and then constructed descriptors for both the metal nodes and organic ligands. The results highlighted the superior performance of the SVM algorithm in a three-class problem, suggesting its applicability in the rapid screening of highly water-stable MOFs. Sun and colleagues, employed SVM for screening high-performance organic photovoltaic (OPV) materials [80]. They initially established a database containing over 1700 OPV donor molecules, classifying materials as “low– performance” or “high-performance” based on their power conversion efficiency (PCE) values. Subsequently, they examined the influence of different types of molecular fingerprints on model performance, discovering that their trained models could rapidly screen OPV donor molecules. These instances of SVM applications illustrate the potential of this tool in materials science, particularly in the rapid and accurate screening of materials with specific desirable properties.

Clustering algorithms are a form of unsupervised learning primarily used to identify natural groupings within data sets. The core principle involves grouping data points based on a specific similarity measure or distance metric, ensuring high similarity amongst data points within the same group and low similarity between data points in different groups. Common clustering algorithms include K-means [81], Spectral Clustering [82], and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [83]. Clustering algorithms are frequently applied to the screening of materials and compounds with specific properties. By conducting straightforward clustering analyses, researchers can study the common properties within a class of materials or compounds, thus proposing new chemical laws or insights. Zhang and colleagues deployed Spectral Clustering and Agglomerative Hierarchical Clustering for screening solid-state lithium-ion conductors (SSLCs) with high conductivity [84]. Utilizing the Inorganic Crystal Structure Database (ICSD), they converted each structure into a structure vector, serving as the input for the model. The SSLCs were grouped into seven classes, followed by a structure-performance relationship study to explore commonalities amongst the structures within the high-conductivity class. Their clustering model successfully differentiated between rapid lithium conduction materials and poor lithium conduction materials, predicting 16 new compounds as solid-state lithium-ion conductors. These newly discovered candidate materials, bearing markedly different structures and chemical compositions compared to known rapid lithium-ion conductors, testify to the effectiveness of clustering methods in uncovering novel materials within extensive materials spaces.

Fig. 6. ${\mathrm{R}}_{\mathrm{f}}$ value prediction harnesses robotics and advanced algorithms. A) A bespoke robotic platform facilitates high-throughput experiments, yielding standardized TLC data. B) The TLC process is characterized using molecular fingerprints, molecular descriptors, and solvent traits. C) Multiple ML models predict with the Ensemble model leading in accuracy. D) The trained model forecasts ${\mathrm{R}}_{\mathrm{f}}$ values across solvent ratios, pinpointing the ratio that maximizes ${\mathrm{R}}_{\mathrm{f}}$ differences between compounds, thereby suggesting optimal separation conditions.

3.2. Application of DL algorithms

In the ensuing section, we elucidate the underlying mechanisms of prevalent DL algorithms, each bolstered by illustrative examples. We aim to illuminate their applications within the realm of chemistry and materials science, and expound on their role in propelling scientific discovery.

3.2.1. ANN-based model

ANN [85] have emerged as a cornerstone in the DL field, employing layered neuron connections to address diverse tasks such as classification, regression, and generation. The efficiency and stability of these models rely on a few key elements: initialization methods, normalization techniques, activation functions, and optimizers. Notable advancements in these areas have been made, such as the Kaiming [86] initialization method which improves convergence speed, and the P-swish [87] activation function that bolsters network performance. New optimizers, such as Adam [88] and AdamW [89], have emerged to expedite and stabilize network training. These developments collectively enhance the capabilities and applications of ANN in the field of chemistry. Although ANN has become quite advanced, it often has a large number of parameters and is prone to overfitting. Consequently, it is not recommended to use ANN when the amount of data is limited.

Mo et al. have pioneered an automated thin-layer chromatography (TLC) platform [90,91], capable of collecting extensive experimental data under standardized conditions (Fig. 6). Leveraging this dataset, which encapsulates 4944 TLC records for 386 compounds, a range of ML techniques were employed to construct surrogate models reflecting the polarity correlations with organic compound structures and their respective retardation factors $\left( {R}_{f}\right)$ . The researchers have ingeniously assembled an ensemble algorithm incorporating ANN, RF, XGBoost [92] (XGB), and LightGBM [93] (LGB) to achieve impressive ${\mathrm{R}}_{\mathrm{f}}$ predictions $\left( {{\mathrm{R}}^{2} = {0.961}}\right)$ . By seamlessly merging robotics and ML, the trained ML models can deliver precise predictions for the ${\mathrm{R}}_{\mathrm{f}}$ value curves of organic compounds under varying solvent combinations, thereby offering general guidance for purification condition selection and hastening the generation and analysis of high-quality TLC data. Luo et al. utilized copious pKa data from the iBonD database to experiment with various ML algorithms for pKa prediction models [36]. They introduced a novel set of descriptors—structure and physical-organic-parameter-based descriptors (SPOC) to represent the electronic and structural features of molecules. Ultimately, ANN yielded optimal predictive performance with a mean absolute error (MAE) as low as ${0.87}\mathrm{{pKa}}$ units. These examples epitomize the potency and adaptability of ANN in the realm of chemistry.

The integration of domain knowledge into the training of ANN, known as Physics-Informed Neural Networks (PINNs) [94], expedites the learning process and enhances prediction accuracy. For instance, our group tried PINNs for ${\mathrm{R}}_{\mathrm{f}}$ value prediction [90], incorporating the chemical principle that the ${\mathrm{R}}_{\mathrm{f}}$ value of a compound increases with the polarity of the developing agent in the same system. When the derivative of ${\mathrm{R}}_{\mathrm{f}}$ with respect to the concentration of a highly polar solvent is less than zero, which represents an atypical physical scenario, they introduce a penalty term into the loss function, as shown as below :

When $\frac{d{R}_{f}}{dC}$ is less than 0, ReLU $\left( {-\frac{d{R}_{f}}{dC}}\right)$ is positive, the loss function increases, and the model is punished; when $\frac{d{R}_{f}}{dC}$ is greater than or equal to 0, ReLU $\left( {-\frac{d{R}_{f}}{dC}}\right)$ is 0, and the loss function does not changes, the model is not subject to additional penalties. In this way, neural networks are guided to learn models that satisfy the laws of physics.

When training an ANN, the focus is not just on prediction accuracy but also model interpretability, a crucial aspect for chemical discovery. To identify pivotal features impacting predicted chemical labels, an array of methods for computing feature importance has been introduced. These methods encompass Sensitivity Analysis [95], LIME (Local Interpretable Model-Agnostic Explanations) [96] and SHAP (SHapley Additive exPlanations) [97], among others. Our group employed Sensitivity Analysis to reveal, for the first time, that the Topological Polar Surface Area (TPSA), a molecular descriptor, exerted the most significant influence on ${\mathrm{R}}_{\mathrm{f}}$ values. This quantifiable correlation between polarity and ${\mathrm{R}}_{\mathrm{f}}$ values underscores the essential role of feature importance computation in chemical discovery.

3.2.2. CNN-based model

Derived from ANNs, CNNs have become an essential tool in image processing and computer vision. With design principles inspired by biological visual systems, CNNs operate through convolution layers to capture local dependency patterns, non-linear activation function layers to enhance network complexity, and pooling layers to control model complexity. Notable variants such as Deep Residual Networks (ResNet [98]) help combat the vanishing gradient problem, facilitating training of deeper networks, while Dilated [99] Convolutions and Group [100] Convolutions offer tools for multi-scale information capture and reduced computational load, respectively.

Capitalizing on its efficacy in extracting salient features from image data, CNNs have been extensively utilized for image processing and recognition in materials science. Instrumental in shaping our understanding of material structure at the atomic scale, electron microscopes such as the Scanning Electron Microscope (SEM), Scanning Transmission Electron Microscope (STEM), and Transmission Electron Microscope (TEM) unfortunately often yield images susceptible to various distortions. Addressing this pervasive issue, Lobato et al. [101] proposed an innovative approach employing CNNs to correct these distortions. Their method, validated on an array of experimental images, demonstrated a substantial enhancement in the signal-to-noise ratio, thereby facilitating an augmented extraction of quantitative structural information from the images. Ziatdinov [102] et al. utilized CNNs for the automatic identification of atomic species and type of defects from scanning probe microscopy images, enhancing efficiency and precision at the atomic scale. This underscores the vital role of CNNs as an efficient instrument, adept at deciphering physical and chemical attributes, particularly when understanding of atomic arrangements and lattice irregularities is scant.

3.2.3. RNN-based model and transformer

RNNs have become instrumental in managing sequential data, courtesy of their ability to retain historical information. Despite initial issues with handling long sequences, variations such as the Long Short-Term Memory (LSTM) [103] and the Gated Recurrent Unit (GRU) [104] have emerged to address these challenges. The LSTM’s gating mechanism allows it to handle long-term dependencies, while the GRU, a simplified version of the LSTM, reduces model complexity, demonstrating the versatility and adaptability of RNNs. In the realm of chemistry, RNNs and their variants have begun to demonstrate potential in the handling of time-series data, such as concentration-time curves. A breakthrough was achieved by Igor Larrosa [105] and colleagues in 2023, when they reported the first application of LSTM models in the study of chemical reaction kinetics, published in Nature. In their novel approach, they generated 5 million kinetics samples through the numerical solution of Ordinary Differential Equations (ODEs) for training and validating their deep neural network. Each sample, composed of four concentration-time distributions from a specific mechanism and a fixed set of kinetic constants, provided comprehensive kinetic information. By inputting the concentration-time series of a specific reaction into the LSTM model, they were able to predict the corresponding reaction mechanism with an astonishing accuracy of 99.9%. Their innovative methodology has been applied to a variety of catalytic reactions including olefin metathesis, cycloaddition, olefin isomerization, C-H amination, and photocatalytic hydrogenation, fostering the discovery of potential mechanisms.

Since its inception in the pivotal paper “Attention is all you need” [106], the Transformer model has garnered widespread attention in the DL community, demonstrating superior performance across various tasks. The self-attention mechanism, the model’s cornerstone, brings substantial benefits in processing sequential data, particularly capturing long-range dependencies. Notably, this ability also enables parallel computation, redressing the limitations inherent in RNN-based models. The self-attention mechanism uniquely processes each input element by considering information from all elements, with reference weights determined by calculating the correlation between the target and other elements. This innovative approach underscores the transformative potential of Transformer models in DL tasks.

In the fields of chemistry and materials science, Transformer models are primarily applied in molecular representation learning, materials representation learning, product prediction, among other aspects. For instance, in molecular representation learning, researchers have proposed models like ChemBERTa [107] and MoLFormer [108]. These models initially represent compounds as SMILES, and leverage the Transformer architecture for pre-training. They capture intricate connections within molecular structures successfully and learn efficient molecular representations via the self-attention mechanism. Particularly noteworthy is the MoLFormer’s performance on quantum chemistry datasets such as QM9, where it outperforms baselines such as A-FP [109] and MPNN [110], underscoring the model’s aptitude in learning 3D molecular features. Elsewhere in organic chemistry, Alpha A. Lee and his team have utilized the Transformer model to predict chemical reaction products [111]. Drawing parallels to text translation in NLP, they represented reactants as reaction sequences and products as product sequences, leading to the development of a Transformer-based Seq2Seq model capable of accurate reaction product prediction. Moreover, in material representation learning, Kim [112] and colleagues proposed a Transformer-based framework for universal MOFs representation learning. They adopted a similar approach, constructing MOF’s local features through a crystal graph convolutional network and obtaining MOF’s global features through molecular dynamics simulation. Both are used as inputs to the Transformer model. Importantly, their work also incorporates interpretability studies, where the attention scores provide insight into which features exhibit significant correlations with the target. They found that, for hydrogen gas adsorption prediction in MOFs, local features are more critical, whereas for MOF’s Bandgap prediction, global features are more crucial, which is a conclusion that resonates with chemical intuition. Their findings spotlight the potential of Transformer models in enhancing model interpretability, thereby contributing to new discoveries in chemistry.

The advantages that models based on the Transformer bring to the materials and chemical sectors compared to traditional ML models are primarily manifested in the following aspects: Firstly, the self-attention mechanism of Transformer models allows the model to effectively capture dependencies over long ranges, which is especially important in understanding complex chemical structures and dynamic material reactions. Secondly, Transformer models can be pre-trained on large-scale datasets and then fine-tuned to specific material science and chemical issues, this transfer learning capability enables the model to quickly adapt and address new domain challenges. Thirdly, the Transformer architecture is inherently suited for parallel computation, meaning that when dealing with large-scale chemical and material databases, it can significantly speed up computation and learning. Based on these advantages, Transformer models will bring opportunities such as rapid knowledge extraction and integration, efficient material discovery, and prediction of complex chemical reactions, among others.

3.2.4. GNN-based model

Emerging in recent years as a novel network structure within the domain of DL, Graph Neural Networks (GNNs) [113] are specifically designed to process graph-structured data. The fundamental operating principle of GNNs is rooted in the so-called message-passing mechanism. In this process, nodes disseminate their features, or “states”, to their neighboring nodes via the edges of the graph. The messages received are aggregated and subsequently employed to update the features of the nodes themselves. In this fashion, the features of nodes are dynamically updated and altered based on their neighborhood relationships within the graph. Obviously, GNNs are highly suited for tasks involving molecular representation learning as well as material representation learning (materials with explicit molecular or crystal structures), where atoms or molecules can act as nodes and chemical bonds or intermolecular interactions as edges. Particularly, network architectures like SchNet [114] can capture the complex features of molecules in three-dimensional space, something that is not achievable with traditional ML algorithms. SchNet, a tailor-made GNN for molecular and materials science conundrums, employs a continuous-filter convolution layer, thereby adroitly grasping the intricacies of atomic environments. This method confers an unparalleled exactness in delineating atomic interactions. Molecular Graph Convolutions (MGC) [115] innovatively adopts an adaptive strategy in manifesting graph representations of atoms and molecules. Further, sophisticated models like Attentive FP [109] and GraphDTA [116] seamlessly weave attention mechanisms into the processing of graph-structured data. This ingenious integration augments their ability to untangle the labyrinthine interplay between complex molecular architectures and their properties.

The separation of enantiomers in chiral molecules constitutes a pivotal and challenging task in experimental chemistry, typically necessitating extensive trial-and-error procedures across various experimental setups. Our group innovatively employed a quantile geometry-enhanced graph neural network (QGeoGNN) to predict the elution times of chiral enantiomers during high-performance liquid chromatography (HPLC) separation (Fig. 7). The researchers initially gathered experimental data from 25,847 molecules, extracted from 644 articles on asymmetric catalysis, which encompassed SMILES, experimental information, HPLC column information, and corresponding retention times. We proceeded to construct the QGeoGNN, a novel network that transmutes the three-dimensional conformations of molecules into two graphs: the atom-bond graph (G) and the bond-angle graph (H). This design was conceived to offer superior representation of three-dimensional structures, thus equipping the network with the capacity to distinguish chiral enantiomers. The effectiveness of this approach far surpasses that of traditional ML algorithms. Additionally, domain knowledge of chromatography was assimilated into the ML model to facilitate multi-column prediction, laying a solid foundation for predictive chromatographic enantioseparation by calculating separation probabilities. The Crystal Graph Convolutional Neural Networks (CGCNNs) [117] distinguish themselves by adopting a distinct methodology for representing crystalline materials. These networks center around the notion of capturing local atomic environments and long-range crystallographic orders using a graph convolutional framework, ultimately enhancing the predictive power in material property forecasting. In a remarkable study, Sanyal and colleagues harnessed the CGCNN in conjunction with transfer learning to simultaneously predict a variety of material characteristics such as Formation Energy, Band Gap, and Fermi Energy across an expansive array of 46774 inorganic crystals [118].

3.2.5. Generative model

Generative models operate by learning the intrinsic data distribution of a training set so that new data points can be created or ‘generated’. They are underpinned by a potent principle: if the model can accurately understand and capture the statistical properties of the input data, it can create novel instances that reflect the same underlying structures and characteristics. Prominent among these generative models are the Generative Adversarial Networks (GANs [119]), Variational Autoen-coders (VAEs [120]), and the recently emergent Diffusion Models [121]. In the fields of chemistry and materials science, generative models are primarily applied to the generation of molecules and materials. It should be noted that since generative models need to learn the statistical patterns of the input data, they often require a substantial amount of training data to ensure that the structures generated are both novel and reliable. However, the cost of data in these domains can be high, therefore, it is advisable to integrate various data augmentation techniques and computational chemistry methods, such as quantum mechanics or molecular dynamics simulations, to ensure a comprehensive training dataset.

Fig. 7. Predicting HPLC retention time with QGeoGNN. A) Retention time data from 644 papers is batch-extracted via NLP, encompassing molecular details and chromatographic conditions, with retention time as the label. B) HPLC chiral separation is represented through QGeoGNN, where molecules are rendered as atom-bond graphs and bond-angle graphs. By embedding experimental conditions and molecular descriptors, this method captures molecules’ 3D attributes, enhancing chiral distinction. C) QGeoGNN outperforms traditional GNN and ML algorithms in single-column prediction and multi-column prediction.

GANs involves a generator (G), tasked with synthesizing convincing data from random noise, and a discriminator (D), committed to distinguishing whether the provided data is authentic or generated by G. Such an adversarial dynamic drives G to progressively excel at fabricating realistic data, while simultaneously sharpening D’s acumen in discerning genuine from generated data. Cao et al. unveiled the MolGAN [122] model, designed for the generation of novel compounds. Experimental validations on the QM9 chemical dataset revealed that their model could yield nearly 100% valid compounds, signifying a promising avenue for the discovery of new molecules.

VAEs encode input data into a latent space, followed by sampling from this space to generate new data. Alán [123] and colleagues proposed a method that leverages VAEs to encode molecular structures into a continuous latent space, followed by sampling to generate potential drug candidates. Jin [124] et al. proposed the JT-VAE model that employs a two-phase process for molecular graph generation, initially creating tree-like scaffolds based on chemical substructures, then amalgamating them into molecules employing a graph message-passing network. This approach enables the progressive expansion of molecules while preserving chemical validity at each stage. Furthermore, Yao [125] and collaborators have ingeniously employed VAEs for the generation of MOFs. By screening the MOFs generated by the model, they discovered high-performing MOFs for carbon dioxide separation that pose robust competition to some of the most effective MOFs and zeolites reported to date.

Diffusion models have emerged in recent years as a captivating class of generative models. The underlying premise involves an iterative process of noise injection and sequential time steps, diffusing the data distribution gradually towards a known simple distribution, such as the Gaussian distribution. When generating new samples, the model guides data in reverse from the simple distribution along the diffusion path back to the original data distribution. Innovatively, Xu and colleagues introduced a novel generative model based on the diffusion process, termed GeoDiff [126], for predicting molecular conformations. Emiel Hoogeboom [127] and collaborators innovatively proposed an E(3) Equivariant Diffusion Model (EDM), a diffusion model for 3D molecule generation that is equivariant to Euclidean transformations. Their model employs an equivariant network that jointly operates on both continuous (atom coordinates) and categorical features (atom types) to learn the denoising of a diffusion process. Remarkably, their proposed approach significantly outperforms previous methods for 3D molecule generation, both in terms of the quality of the generated samples and the efficiency during training.

3.3. Application of optimization algorithms

Bayesian Optimization (BO) [128] serves as a strategic approach for global optimization, each iteration employs a surrogate model, typically a Gaussian process, to approximate the objective function. The strength of BO lies in its ability to conduct effective optimization with fewer samples, making it suitable for situations where sample costs are high or samples are difficult to obtain. However, its performance decreases with high-dimensional problems, and computational expenses increase with the number of iterations. Therefore, for problems where sample costs are significant and precise searching is required, such as in the design of catalysts, BO is highly favored. For instance, in a trailblazing collaborative effort, Cooper [129] of the University of Liverpool and Labman, an automation company, have co-developed an automated system for a mobile robot that targets catalyst screening. The robot emulates human chemists, handling tasks like solid catalyst weighing, liquid distribution, and the activation of experimental apparatus. It also manipulates analytical instruments to inspect experimental outcomes. Over the course of a relentless week, barring recharging sessions, this robot concluded almost 700 experiments. Beyond the initial experimental conditions, the robot, guided by BO, continues to adjust and optimize experimental parameters based on the results of the previous experiments. This demonstrates the great potential of BO in the direction of material screening. In addition to catalyst screening, several researchers have also employed Bayesian Optimization for the process optimization of perovskite solar cells [130,131].

Genetic Algorithms(GA) [132], global optimization techniques mirroring biological evolutionary theory, are based on selection, crossover, and mutation. These algorithms simulate natural selection and genetic inheritance mechanisms for problem-solving. GA is that they are less likely to get trapped in local optima, making them particularly suitable for complex nonlinear problems. However, they may require longer computational times and are sensitive to the settings of algorithm parameters. Within the realms of chemistry and materials science, genetic algorithms are broadly harnessed for structure prediction and optimization. Pioneering work by Deaven [133] et al. in 1995 saw the first application of genetic algorithms for structural optimization of atomic clusters. Oganov [134] and colleagues proposed a method, USPEX, for crystal structure prediction based on genetic algorithms, which has become a widely utilized tool for structure prediction in solid-state physics. Furthermore, Kim [135] et al. utilized a combination of genetic algorithms and ML for polymer design, evolving polymers through crossover, mutation, and selection over 100 generations to produce 132 new, chemically unique polymers.

Particle Swarm Optimization (PSO) [31] is a heuristic global optimization method, inspired by the collective behavior observed in nature, such as flocks of birds or schools of fish. The fundamental principle of PSO is as follows: a “swarm” of particles fly through the search space, with each particle retaining the optimal position information it has found during the search process (that is, its best solution and the global best solution). Each particle updates its velocity and position during each iteration to search for the global optimum. This process allows for global search of the solution space while maintaining excellent local search capabilities. When the problem is relatively simple, not particularly high-dimensional, and a rapid solution is required, PSO can be a good choice. Wang [34] et al. used the PSO algorithm in predicting the structure of layered materials. Their successful predictions included the structures of known 2D materials, comprising single-layer and multi-layer graphene, 2D boron nitride (BN) compounds. This demonstration underscores the potential of the PSO algorithm in the field of structure prediction.

The utilization of prior knowledge in BO allows it to perform well when samples are limited, the randomness and diversity of GA ensure that they do not easily get trapped in local optima when exploring spaces, and the fewer parameters and ease of implementation of PSO enable it to quickly yield satisfactory results for some relatively simple problems. Therefore, selecting the most suitable algorithm for a specific issue is of great importance.

4. Conclusion and outlook

This review embarks by explicating the fundamentals of intelligent algorithms, including ML, DL, and optimization algorithms. It then delineates the modalities of data acquisition (standard databases, literature extraction methodologies), and common techniques of feature engineering, before launching into a comprehensive exposition of various intelligent algorithm applications within the realms of chemistry and materials science. Despite significant advancements in the deployment of intelligent algorithms, we believe that there are primarily three major issues that urgently need to be addressed in the future. First, intelligent algorithms are data-driven models; a vast amount of high-quality data forms the cornerstone of intelligent algorithms. However, in the fields of chemistry and materials, there still exist issues such as high data costs, difficulties in database sharing, and poor data quality. Second, current prevalent intelligent algorithms lack physical concepts; they merely learn patterns or rules from data, which inevitably can lead to results that violate physical laws, and also affect convergence and stability. Therefore, how to embed domain knowledge of chemistry and materials into intelligent algorithms to construct a model that is physically reasonable, mathematically precise, and computationally efficient presents a significant challenge for the future. Third, the ultimate goal for chemists is to learn new knowledge and rules from intelligent algorithms, thereby promoting new scientific discoveries. However, this aspect has not been well-developed at present, and research on model interpretability is still lacking.

Challenges are opportunities, and the path to innovative development in the future lies in how to solve these three issues. First and foremost is the data aspect. The future should prioritize the development of high-throughput technologies and laboratory automation techniques to rapidly acquire a large volume of standardized data; of course, this is mainly a hardware issue. For instance, Jiang [136] and colleagues have pioneered a comprehensive ’end-to-end robotic chemist’ platform that encompasses literature review, autonomous experimental design, and materials development. This platform is adept at identifying the optimum solution from hundreds of millions of possible combinations, thus accelerating the pace of material research and development. Secondly, the future should focus on developing knowledge-embedding techniques to construct intelligent algorithms infused with chemical and materials knowledge, mainly comprising strategies of both soft and hard constraints. Soft constraints typically do not force the model to strictly adhere to certain rules, but rather give the model some degree of freedom, taking domain knowledge into account during the optimization process. The main strategies include theoretically-guided normalization, the design of new descriptors, network topology structure design, convolution kernels and filters design, and loss function design. On the other hand, hard constraints are rules that the model must strictly follow, with main strategies including data decomposition, finite element methods, and domain space constraints. For instance, our research group has constructed a soft-constrained PINN [90] that incorporates the monotonic rule of chromatography into the loss function, while also constructing a hard constraint that restricts the output to $\left( {0,1}\right)$ through the sigmoid activation function, aligning with the real-world behavior of ${\mathrm{R}}_{\mathrm{f}}$ values. Thirdly, the future should focus on the development of knowledge discovery technologies based on intelligent algorithms, to extract undiscovered knowledge from data and push the boundaries of human cognition. The main strategies include sparse regression, genetic algorithms, symbolic mathematics, etc. For example, Jiang and others used the SISSO method to obtain a concise mathematical formula for the spectrum-property relationship [65], which is interpretable, thus achieving the transfer application of knowledge across different systems. Our vision for the future is this: Rapid data acquisition through laboratory automation, researchers using these high-quality data to construct intelligent algorithms with knowledge embedding, and through knowledge discovery technologies, uncover new knowledge and rules, which in turn further guide algorithm design, forming a closed loop. The application of intelligent algorithms within the domains of chemistry and material science holds a plethora of possibilities yet to be fully explored. It is our aspiration that the insights and perspectives shared within this paper will serve as a source of inspiration for researchers, collectively steering the development and growth of this field.

Declaration of Competing Interest

I, Fanyang Mo, hereby declare that I have no known competing financial interests or personal relationships that could have influenced the work reported in this manuscript. I have received no financial support from any organization for the submitted work, have no financial relationships with any organizations that might have an interest in the submitted work, and have no other relationships or activities that could appear to have influenced the submitted work.

References

[1] A. Clauset, D.B. Larremore, R.J.S. Sinatra, Data-driven predictions in the science of science, Science 355 (2017) 477-480.

[2] K. Jorner, A. Tomberg, C. Bauer, C. Sköld, P.-O. Norrby, Organic reactivity from mechanism to machine learning, Nat. Rev. Chem. 5 (2021) 240-255.

[3] Z. Tu, T. Stuyver, C.W. Coley, Predictive chemistry: machine learning for reaction deployment, reaction development, and reaction discovery, Chem. Sci. 14 (2023) 226-244.

[4] L.C. Yang, L.J. Zhu, S.Q. Zhang, X. Hong, Machine learning prediction of structure-performance relationship in organic synthesis, Chin. J. Chem. 40 (2022) 2106-2117.

[5] T. Poggio, et al., Why and when can deep-but not shallow-networks avoid the

[6] C. McInnes, Virtual screening strategies in drug discovery, Curr. Opin. Chem. Biol. 11 (2007) 494-502.

[7] J. Greeley, T.F. Jaramillo, J. Bonde, I. Chorkendorff, J.K. Nørskov, Computational high-throughput screening of electrocatalytic materials for hydrogen evolution, Nat. Mater. 5 (2006) 909-913.

[8] J. Li, et al., A comprehensive kinetic mechanism for $\mathrm{{CO}},{\mathrm{{CH}}}_{2}\mathrm{O}$ , and ${\mathrm{{CH}}}_{3}\mathrm{{OH}}$ combustion, Int. J. Chem. Kinet. 39 (2007) 109-136.

[9] C. Hansch, Quantitative approach to biochemical structure-activity relationships, Acc. Chem. Res. 2 (1969) 232-239.

[10] D. Maulud, A.M. Abdulazeez, A review on linear regression comprehensive in machine learning, J. Appl. Sci. Technol. Trends 1 (2020) 140-147.

[11] C. Kingsford, S.L. Salzberg, What are decision trees? Nat. Biotechnol. 26 (2008) 1011-1013.

[12] M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, B. Scholkopf, Support vector machines, IEEE Intell. Syst. their Appl. 13 (1998) 18-28.

[13] R. Bro, A.K. Smilde, Principal component analysis, Anal. Methods 6 (2014) 2812-2831.

[14] L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: a survey, J. Artif. Intell. Res. 4 (1996) 237-285.

[15] F. Pedregosa, et al., Scikit-learn: machine learning in Python, J. Mach. Learn. Res. 12 (2011) 2825-2830.

[16] A.K. Jain, J. Mao, K.M.J.C. Mohiuddin, Artificial neural networks: a tutorial, Computer 29 (1996) 31-44.

[17] Li, Z. et al. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems (2021).

[18] L.R. Medsker, L.J.D. Jain, Applications, Recurrent neural networks, Des. Appl. 5 (2001) 2.

[19] Paszke, A. et al. Automatic differentiation in pytorch. (2017).

[20] Abadi, M. et al. in 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265-283.

[21] Y. Ma, D. Yu, T. Wu, H. Wang, PaddlePaddle: an open-source deep learning platform from industrial practice, Front. Data Domputing 1 (2019) 105-115.

[22] Huawei Technologies Co., Huawei MindSpore AI Development Framework, Artif. Intell. Technol. (2022) 137-162.

[23] A. Gulli, S. Pal, Deep learning with Keras, Packt Publishing Ltd, 2017.

[24] D.T. Ahneman, J.G. Estrada, S. Lin, S.D. Dreher, A.G.J.S. Doyle, Predicting reaction performance in C-N cross-coupling using machine learning, Science 360 (2018) 186-190.

[25] B. Mikulak-Klucznik, et al., Computational planning of the synthesis of complex natural products, Nature 588 (2020) 83-88.

[26] K. Muraoka, Y. Sada, D. Miyazaki, W. Chaikittisilp, T.J.N. c Okubo, Linking synthesis and structure descriptors from a large collection of synthetic records of zeolite materials, Nat. Commun. 10 (2019) 4459.

[27] J. Jumper, et al., Highly accurate protein structure prediction with AlphaFold, Nature 596 (2021) 583-589.

[28] Y. Saito, et al., Deep-learning-based quality filtering of mechanically exfoliated 2D crystals, npj Comput. Mater. 5 (2019), 124.

[29] D. Bertsimas, J.J.S. s Tsitsiklis, Simulated annealing, Stat. Sci. 8 (1993) 10-15.

[30] J.H. Holland, Genetic algorithms, Sci. Am. 267 (1992) 66-73.

[31] Kennedy, J. & Eberhart, R. in Proceedings of ICNN'95-international conference on neural networks. 1942-1948 (IEEE).

[32] R. Harada, T. Nakamura, Y. Shigeta, A fast convergent simulated annealing algorithm for protein-folding: simulated annealing outlier FLOODing (SA-OFLOOD) method, Bull. Chem. Soc. Jpn. 89 (2016) 1361-1367.

[33] H. Link, et al., Multi-objective steady state optimization of biochemical reaction networks using a constrained genetic algorithm, Comput. Chem. Eng. 32 (2008) 1707-1713.

[34] Y. Wang, et al., An effective structure prediction method for layered materials based on 2D particle swarm optimization algorithm, J. Chem. Phys. 137 (2012).

[35] T. Lewis-Atwell, P.A. Townsend, M.N. Grayson, Machine learning activation energies of chemical reactions, Wiley Interdiscip. Rev.: Comput. Mol. Sci. 12 (2022), e1593.

[36] Q. Yang, et al., Holistic prediction of the pKa in diverse solvents based on a machine-learning approach, Angew. Chem. Int. Ed. 132 (2020) 19444-19453.

[37] X. Fang, et al., Geometry-enhanced molecular representation learning for property prediction, Nat. Mach. Intell. 4 (2022) 127-134.

[38] S. Kim, et al., PubChem substance and compound databases, Nucleic Acids Res. 44 (2016) D1202-D1213.

[39] H.E. Pence, A. Williams, ChemSpider: An online chemical information resource, J. Chem. Educ. 87 (2010) 1123-1124.

[40] A. Gaulton, et al., ChEMBL: a large-scale bioactivity database for drug discovery, Nucleic Acids Res. 40 (2012) D1100-D1107.

[41] J.J. Irwin, B.K. Shoichet, ZINC- a free database of commercially available compounds for virtual screening, J. Chem. Inf. Model. 45 (2005) 177-182.

[42] C.R. Groom, I.J. Bruno, M.P. Lightfoot, S.C. Ward, The Cambridge structural database, Acta Cryst. 72 (2016) 171-179.

[43] Internet Bond-energy Databank (pKa and BDE)-iBonD Home Page, http://ibond.nankai.edu.cn.

[44] S.M. Kearnes, et al., The open reaction database, J. Am. Chem. Soc. 143 (2021) 18820-18826.

[45] A. Jain, et al., Commentary: the materials project: a materials genome approach to accelerating materials innovation, APL Mater. 1 (2013).

[46] S.P. Ong, et al., Python materials genomics (pymatgen): a robust, open-source python library for materials analysis, Comput. Mater. Sci. 68 (2013) 314-319.

[47] A.H. Larsen, et al., The atomic simulation environment—a python library for working with atoms, J. Phys.: Condens. Matter 29 (2017), 273002.

[48] D.D. Landis, et al., The computational materials repository, Comput. Sci. Eng. 14 (2012) 51-57.

[49] J.E. Saal, S. Kirklin, M. Aykol, B. Meredig, C.J.J. Wolverton, Materials design and discovery with high-throughput density functional theory: the open quantum materials database (OQMD), Jom 65 (2013) 1501-1509.

[50] K. Chowdhary, K.J.F. o a i Chowdhary, Natural language processing, Fundam. Artif. Intell. (2020) 603-649.

[51] L. Richardson, Beautiful soup documentation. (2007).

[52] E. Uzun, T. Yerlikaya, O. Kirat, Bulgaria. Comparison of Python libraries used for Web data extraction, J. Tech. Univ. -Sofia Plovdiv. Branch, Bulg. 24 (2018) 87-92.

[53] Nancy, I. & Véronis, J. in 6th Annual Conference of the Centre for the New Oxford English Dictionary. 52-64 (Citeseer).

[54] P.J.P. Willett, The Porter stemming algorithm: then and now, Program 40 (2006) 219-223.

[55] K.W. Church, Word2Vec, Nat. Lang. Eng. 23 (2017) 155-162.

[56] Pennington, J., Socher, R. & Manning, C.D. in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532-1543.

[57] Nivre, J. in Proceedings of the eighth international conference on parsing technologies. 149-160.

[58] A. Vaswani, et al., Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017).

[59] T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv:1609.02907 (2016).

[60] T.E. Oliphant, Guide to numpy, Vol. 1, Trelgol Publishing USA, 2006.

[61] Toropov, A.A., Toropova, A.P., Mukhamedzhanoval, D.V. & Gutman, I. Simplified molecular input line entry system (SMILES) as an alternative for constructing quantitative structure-property relationships (QSPR). (2005).

[62] A. Cereto-Massagué, et al., Molecular fingerprint similarity search in virtual screening, Methods 71 (2015) 58-63.

[63] Z. Liu, T. Zubatiuk, A. Roitberg, O. Isayev, Auto3d: automatic generation of the low-energy 3d structures with ANI neural network potentials, J. Chem. Inf. Model. 62 (2022) 5373-5382.

[64] X. Fang, et al., Geometry-enhanced molecular representation learning for property prediction, Nat. Mach. Intell. 4 (2022) 127-134.

[65] X. Wang, et al., Quantitatively determining surface-adsorbate properties from vibrational spectroscopy with interpretable machine learning, J. Am. Chem. Soc. 144 (2022) 16069-16076.

[66] J.M. Granda, L. Donina, V. Dragone, D.-L. Long, L. Cronin, Controlling an organic synthesis robot with machine learning to search for new reactivity, Nature 559 (2018) 377-381.

[67] P. Schwaller, et al., Mapping the space of chemical reactions using attention-based neural networks, Nat. Mach. Intell. 3 (2021) 144-152.

[68] A.F. Zahrt, et al., Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning, Science 363 (2019) 5631.

[69] P.M. Granitto, C. Furlanello, F. Biasioli, F. Gasperi, Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products, Chemom. Intell. Lab. Syst. 83 (2006) 83-90.

[70] Y.-Y. Song, L.J.S. Ying, Decision tree methods: applications for classification and prediction, Shanghai Arch. Psychiatry 27 (2015) 130.

[71] H. Zou, The adaptive lasso and its oracle properties, J. Am. Stat. Assoc. 101 (2006) 1418-1429.

[72] A.E. Hoerl, R.W. Kennard, Ridge regression: applications to nonorthogonal problems, Technometrics 12 (1970) 69-82.

[73] H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B: Stat. Methodol. 67 (2005) 301-320.

[74] D.R. Cutler, et al., Random forests for classification in ecology, Ecology 88 (2007) 2783-2792.

[75] D.T. Ahneman, J.G. Estrada, S. Lin, S.D. Dreher, A.G. Doyle, Predicting reaction performance in C-N cross-coupling using machine learning, Science 360 (2018) 186-190.

[76] S.K. Chandy, K. Raghavachari, MIM-ML: a novel quantum chemical fragment-based random forest model for accurate prediction of NMR chemical shifts of nucleic acids, J. Chem. Theory Comput. 19 (2023) 6632-6642.

[77] J.G. Meyer, S. Liu, I.J. Miller, J.J. Coon, A. Gitter, Learning drug functions from chemical structures with convolutional neural networks and random forests, J. Chem. Inf. Model. 59 (2019) 4438-4449.

[78] C. Mouchel-Vallon, A. Hodzic, Toward emulating an explicit organic chemistry mechanism with random forest models, J. Geophys. Res.: Atmos. 128 (2023), e2022JD038227.

[79] R. Batra, C. Chen, T.G. Evans, K.S. Walton, R. Ramprasad, Prediction of water stability of metal-organic frameworks using machine learning, Nat. Mach. Intell. 2 (2020) 704-710.

[80] W. Sun, et al., Machine learning-assisted molecular design and efficiency prediction for high-performance organic photovoltaic materials, Sci. Adv. 5 (2019) 4275.

[81] Hartigan, J.A. & Wong, M.A. Algorithm AS 136: A k-means clustering algorithm. Journal of the royal statistical society. series c 28, 100-108 (1979).

[82] U. Von Luxburg, A tutorial on spectral clustering, Stat. Comput. 17 (2007) 395-416.

[83] Wang, W.-T., Wu, Y.-L., Tang, C.-Y. & Hor, M.-K.. in 2015 International Conference on Machine Learning and Cybernetics (ICMLC). 445-451 (IEEE).

[84] Y. Zhang, et al., Unsupervised discovery of solid-state lithium ion conductors, Nat. Commun. 10 (2019), 5260.

[85] A.K. Jain, J. Mao, K.M. Mohiuddin, Artificial neural networks: A tutorial, Computer 29 (1996) 31-44.

[86] He, K., Zhang, X., Ren, S. & Sun, J. in Proceedings of the IEEE international conference on computer vision. 1026-1034.

[87] Mercioni, M.A. & Holban, S. in 2020 International Symposium on Electronics and Telecommunications (ISETC). 1-4 (IEEE).

[88] Kingma, D.P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:. (2014).

[89] Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:.05101 (2017).

[90] H. Xu, et al., High-throughput discovery of chemical structure-polarity relationships combining automation and machine-learning techniques, Chem 8 (2022) 3202-3214.

[91] H. Xu, D. Zhang, F. Mo, High-throughput automated platform for thin layer chromatography analysis, STAR Protoc. 3 (2022), 101893.

[92] Chen, T. & Guestrin, C. in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785-794.

[93] G. Ke, et al., Lightgbm: A highly efficient gradient boosting decision tree, Adv. Neural Inf. Process. Syst. (30) (2017).

[94] M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686-707.

[95] V. Nourani, M.S. Fard, Sensitivity analysis of the artificial neural network outputs in simulation of the evaporation process at different climatologic regimes, Adv. Eng. Softw. 47 (2012) 127-146.

[96] S. Mishra, B.L. Sturm, S. Dixon, Local interpretable model-agnostic explanations for music content analysis, ISMIR (2017) 537-543.

[97] S. Mangalathu, S.-H. Hwang, J.-S. Jeon, Failure mode and effects analysis of RC members based on machine-learning-based SHapley Additive exPlanations (SHAP) approach, Eng. Struct. 219 (2020), 110927.

[98] He, K., Zhang, X., Ren, S. & Sun, J. in Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778.

[99] Yu, F. & Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:.07122 (2015).

[100] Zhang, T., Qi, G.-J., Xiao, B. & Wang, J. in Proceedings of the IEEE international conference on computer vision. 4373-4382.

[101] Lobato, I., Friedrich, T. & Van Aert, S. Deep convolutional neural networks to restore single-shot electron microscopy images. arXiv preprint arXiv:.17025 (2023).

[102] M. Ziatdinov, et al., Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations, ACS nano 11 (2017) 12742-12752.

[103] A. Graves, A. Graves, Long short-term memory, Supervised Seq. Label. Recurr. Neural Netw. (2012) 37-45.

[104] Chung, J., Gulcehre, C., Cho, K. & Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:. (2014).

[105] J. Burés, I. Larrosa, Organic reaction mechanism classification using machine learning, Nature 613 (2023) 689-695.

[106] A. Vaswani, et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (30) (2017).

[107] Chithrananda, S., Grand, G. & Ramsundar, B.J. a p a ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. (2020).

[108] J. Ross, et al., Large-scale chemical language representations capture mo structure and properties, Nat. Mach. Intell. 4 (2022) 1256-1264.

[109] Z. Xiong, et al., Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism, J. Med. Chem. 63 (2019) 8749-8760.

[110] Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O. & Dahl, G.E. in International conference on machine learning. 1263-1272 (PMLR).

[111] P. Schwaller, et al., Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction, ACS Cent. Sci. 5 (2019) 1572-1583.

[112] Y. Kang, H. Park, B. Smit, J. Kim, A multi-modal pre-training transformer for universal transfer learning in metal-organic frameworks, Nat. Mach. Intell. 5 (2023) 309-318.

[113] Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? arXiv preprint arXiv:.00826 (2018).

[114] K.T. Schütt, H.E. Sauceda, P.-J. Kindermans, A. Tkatchenko, K.-R. Müller, Schnet-a deep learning architecture for molecules and materials, J. Chem. Phys. (148) (2018).

[115] S. Kearnes, K. McCloskey, M. Berndl, V. Pande, P. Riley, Molecular graph convolutions: moving beyond fingerprints, J. Comput. -Aided Mol. Des. 30 (2016) 595-608.

[116] T. Nguyen, et al., GraphDTA: Predicting drug-target binding affinity with graph neural networks, Bioinformatics 37 (2021) 1140-1147.

[117] T. Xie, J.C. Grossman, Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties, Phys. Rev. Lett. 120 (2018), 145301.

[118] Sanyal, S. et al. MT-CGCNN: Integrating crystal graph convolutional neural network with multitask learning for material property prediction. arXiv preprint arXiv:.05660 (2018).

[119] I. Goodfellow, et al., Generative adversarial networks, Commun. ACM 63 (2020) 139-144.

[120] Doersch, C. Tutorial on variational autoencoders. arXiv preprint arXiv:.05908 (2016).

[121] Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. & Ganguli, S. in International conference on machine learning. 2256-2265 (PMLR).

[122] De Cao, N. & Kipf, T. MolGAN: An implicit generative model for small molecular graphs. arXiv preprint arXiv:.11973 (2018).

[123] R. Gómez-Bombarelli, et al., Automatic chemical design using a data-driven continuous representation of molecules, ACS Cent. Sci. 4 (2018) 268-276.

[124] Jin, W., Barzilay, R. & Jaakkola, T. in International conference on machine learning. 2323-2332 (PMLR).

[125] Z. Yao, et al., Inverse design of nanoporous crystalline reticular materials with deep generative models, Nat. Mach. Intell. 3 (2021) 76-86.

[126] Xu, M. et al. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:.02923 (2022).

[127] Hoogeboom, E., Satorras, V.G., Vignac, C. & Welling, M. in International conference on machine learning. 8867-8887 (PMLR).

[128] J. Snoek, H. Larochelle, R.P. Adams, Practical bayesian optimization of machine learning algorithms, Adv. Neural Inf. Process. Syst. (25 ) (2012).

[129] B. Burger, et al., A mobile robotic chemist, Nature 583 (2020) 237-241.

[130] Z. Liu, et al., Machine learning with knowledge constraints for process optimization of open-air perovskite solar cell manufacturing, Joule 6 (2022) 834-849.

[131] J. Zhang, et al., Optimizing perovskite thin-film parameter spaces with machine learning-guided robotic platform for high-performance perovskite solar cells, Adv. Energy Mater. (2023), 2302594.

[132] Sastry, K., Goldberg, D. & Kendall, G. Genetic algorithms. Search methodologies: Introductory tutorials in optimization decision support techniques, (2005) 97-125 .

[133] D.M. Deaven, K.-M. Ho, Molecular geometry optimization with a genetic algorithm, Phys. Rev. Lett. 75 (1995) 288.

[134] C.W. Glass, A.R. Oganov, N. Hansen, USPEX-evolutionary crystal structure prediction, Comput. Phys. Commun. 175 (2006) 713-720.

[135] C. Kim, R. Batra, L. Chen, H. Tran, R. Ramprasad, Polymer design using genetic algorithm and machine learning, Comput. Mater. Sci. 186 (2021), 110067.

[136] Q. Zhu, et al., An all-round AI-Chemist with a scientific mind, Natl. Sci. Rev. 9 (2022), nwac190.