Empowering Research in Chemistry and Materials Science through Intelligent Algorithms

英文原文与中文译文逐段对照 · English original with Chinese translation

Jinglong Lin ${}^{a}$ , Fanyang Mo ${}^{a, b, * }$

林京龙${}^{a}$ ，莫凡洋${}^{a, b, * }$

${}^{a}$ School of Materials Science and Engineering, Peking University, Beijing 100871, PR China

${}^{a}$ 中国北京大学材料科学与工程学院，北京100871

${}^{\mathrm{b}}$ Al for Science (AI4S)-Preferred Program, Peking University Shenzhen Graduate School, Shenzhen 518055, PR China

${}^{\mathrm{b}}$ 中国北京大学深圳研究生院人工智能促进科学(AI4S)优选计划，深圳518055

A R T I C L E I N F O

文章信息

Keywords:

关键词:

Intelligent Algorithms

智能算法

Machine Learning

机器学习

Deep Learning

深度学习

A B S T R A C T

摘 要

In this review, we explore the integration of intelligent algorithms in chemistry and materials science.We begin by delineating the core principles of Machine Learning, Deep Learning, and optimization algorithms, highlighting their bespoke adaptation to these scientific domains. The focus then shifts to the critical processes of data management, including collection, refinement, and feature engineering, alongside strategies for efficient data mining from targeted databases and literatures. Subsequently, we present a concise overview of the diverse applications of these algorithms, emphasizing their transformative impact in both fields. Finally, this review explores the future prospects and challenges of these emerging algorithms.

在本综述中，我们探讨了智能算法在化学和材料科学中的整合。我们首先阐述机器学习、深度学习和优化算法的核心原理，强调它们对这些科学领域的定制化适配。然后重点转向数据管理的关键过程，包括数据收集、精炼和特征工程，以及从目标数据库和文献中进行高效数据挖掘的策略。随后，我们简要概述了这些算法的各种应用，强调它们在这两个领域的变革性影响。最后，本综述探讨了这些新兴算法的未来前景和挑战。

In the domain of intelligent algorithms, Machine Learning (ML), Deep Learning (DL), and other Optimization Algorithms (OA) form the computational paradigms proficient in harnessing data for sophisticated tasks such as decision-making, prediction, and optimization (Fig. 1). In an era witnessing an explosion of chemical information and advancing computational prowess, data-driven scientific discovery emerges as an imperative research paradigm [1-4]. However, tackling complex systems frequently incurs the ‘curse of dimensionality’, necessitating substantial computational resources [5]. Fortunately, intelligent algorithms offer a potent toolkit for fitting and solving high-dimensional functions, thereby aiding the resolution of mathematical models within intricate systems. Moreover, the multi-scale nature of chemical modeling (micro-meso-macro) challenges the investigation of structure-activity relationships. Notwithstanding these complexities, intelligent algorithms serve as crucial aids in cross-scale modeling, learning the mappings between micro and macro scales, even in instances where these relationships may outstrip our current theoretical grasp.

在智能算法领域，机器学习(ML)、深度学习(DL)和其他优化算法(OA)构成了擅长利用数据进行决策、预测和优化等复杂任务的计算范式(图1)。在一个化学信息爆炸且计算能力不断提升的时代，数据驱动的科学发现成为一种必要的研究范式[1 - 4]。然而，处理复杂系统常常会遭遇“维度诅咒”，需要大量计算资源[5]。幸运的是，智能算法提供了一个强大的工具包来拟合和求解高维函数，从而有助于解决复杂系统中的数学模型。此外，化学建模的多尺度性质(微观 - 介观 - 宏观)对结构 - 活性关系的研究提出了挑战。尽管存在这些复杂性，智能算法在跨尺度建模中起着关键辅助作用，能够学习微观和宏观尺度之间的映射关系，即使在这些关系可能超出我们当前理论理解的情况下也是如此。

In the realms of chemistry and materials science, challenges can be broadly categorized into three domains: substance screening, elucidation of mechanisms, and process optimization (Fig. 2). This review delves into the role of intelligent algorithms in solving these challenges. Beginning with the foundational theories of intelligent algorithms, we aim to enrich the reader’s understanding of their workings. We subsequently delve into data acquisition and processing methodologies from diverse sources for algorithmic consumption. Following this, we elucidate the distinct applications of various intelligent algorithms within the chemistry and materials science domains, supplementing these explorations with specific use-cases.

在化学和材料科学领域，挑战大致可分为三个领域:物质筛选、机理阐释和过程优化(图2)。本综述深入探讨智能算法在解决这些挑战中的作用。从智能算法的基础理论开始，我们旨在丰富读者对其工作原理的理解。随后，我们深入研究从不同来源获取并供算法使用的数据采集和处理方法。在此之后，我们阐明各种智能算法在化学和材料科学领域的不同应用，并通过具体用例对这些探索进行补充。

This article aspires to illuminate current trajectories and provide foresight into future developments in applying intelligent algorithms within the chemical domain. Antecedent reviews have typically centered on the intrinsic challenges of chemistry, yet herein we adopt a distinct vantage, threading our narrative with a suite of algorithms and scrutinizing their pertinence to the multifaceted quandaries of the discipline. We hope this review is both informative and inspirational for researchers immersed in the field, as well as academics and engineers aspiring to venture into this dynamic domain.

本文旨在阐明当前的发展轨迹，并对在化学领域应用智能算法的未来发展提供前瞻性见解。先前的综述通常集中在化学的内在挑战上，但在此我们采用了不同的视角，以一系列算法贯穿我们的叙述，并审视它们与该学科多方面难题的相关性。我们希望本综述对沉浸在该领域的研究人员以及渴望涉足这个充满活力领域的学者和工程师都具有信息性和启发性。

1. Theoretical basis

1. 理论基础

1.1. Basics of ML and DL

1.1. 机器学习和深度学习的基础

ML and DL, integral parts of intelligent algorithms, depend on data-driven learning to enable decision-making and predictions. ML branches into supervised learning-employing models like linear regression [10], decision trees [11], support vector machines [12] for predictions based on labelled data, unsupervised learning-utilizing techniques like clustering and principal component analysis [13] to discern structures in unlabeled data, and reinforcement learning-learning through environmental interaction to maximize a reward function [14]. Most ML algorithms can be seamlessly implemented via the scikit-learn Python package [15].

机器学习和深度学习作为智能算法的组成部分，依赖于数据驱动的学习来进行决策和预测。机器学习分为监督学习，使用线性回归[10]、决策树[11]、支持向量机[12]等模型基于标记数据进行预测；无监督学习，利用聚类和主成分分析[13]等技术识别未标记数据中的结构；以及强化学习，通过与环境交互来最大化奖励函数[14]。大多数机器学习算法可以通过scikit-learn Python包[15]无缝实现。

• Corresponding author at: School of Materials Science and Engineering, Peking University, Beijing 100871, PR China.

• 通讯作者地址:中国北京，北京大学材料科学与工程学院，邮编100871。

(F. Mo).

(F. Mo)。

DL, a subfield of ML, attempts to mimic the human brain’s operations, employing Artificial Neural Networks (ANNs) [16] to tackle complex, abstract problems. DL models, such as Convolutional Neural Networks (CNNs) [17] and Recurrent Neural Networks (RNNs) [18], lean heavily on abundant data and computational resources. The training phase of DL typically involves loss function calculations, optimizing model parameters by minimizing this loss function via optimizers. Popular DL frameworks include PyTorch [19], TensorFlow [20], PaddlePaddle [21], MindSpore [22], Keras [23], among others.

深度学习是机器学习的一个子领域，它试图模拟人类大脑的运作方式，利用人工神经网络(ANNs)[16]来解决复杂的抽象问题。深度学习模型，如卷积神经网络(CNNs)[17]和循环神经网络(RNNs)[18]，严重依赖丰富的数据和计算资源。深度学习的训练阶段通常涉及损失函数计算，通过优化器最小化该损失函数来优化模型参数。流行的深度学习框架包括PyTorch [19]、TensorFlow [20]、PaddlePaddle [21]、MindSpore [22]、Keras [23]等。

In the realm of chemistry and materials, ML and DL enjoy extensive applicability. ML algorithms, for instance, can predict chemical reaction yields and selectivity [24], design novel chemical synthesis routes [25], and facilitate high-throughput material screening [26]. DL, on the other hand, is apt for handling more intricate issues, such as predicting protein three-dimensional structures [27] and analyzing material microstruc-tures [28].

在化学和材料领域，机器学习和深度学习具有广泛的适用性。例如，机器学习算法可以预测化学反应产率和选择性[24]，设计新的化学合成路线[25]，并促进高通量材料筛选[26]。另一方面，深度学习适用于处理更复杂的问题，如预测蛋白质三维结构[27]和分析材料微观结构[28]。

1.2. Basics of optimization algorithms

1.2. 优化算法基础

As a crucial subset of intelligent algorithms, optimization algorithms find extensive utility across scientific and engineering disciplines. Their prime objective is to uncover optimal solutions for a given function. This is achieved through a myriad of strategies including, but not limited to, Simulated Annealing(SA) [29], Genetic Algorithms(GA) [30], Particle Swarm Optimization(PSO) [31], etc. These techniques underscore the diversity and adaptability of optimization algorithms in diverse problem-solving scenarios.

作为智能算法的一个关键子集，优化算法在科学和工程学科中有着广泛的应用。它们的主要目标是为给定函数找到最优解。这是通过多种策略实现的，包括但不限于模拟退火算法(SA) [29]、遗传算法(GA) [30]、粒子群优化算法(PSO) [31]等。这些技术突显了优化算法在不同问题解决场景中的多样性和适应性。

Iterative search methods are a key tool and core mechanism in optimization algorithms. For instance, SA models the annealing procedure in solids to determine the lowest energy state. GA utilize biological evolutionary mechanisms, engendering optimal solutions via repeated selection, crossover, and mutation. PSO drawing upon flock or fish social behaviors, modifies each “particle’s” position and velocity to locate an optimal solution. These optimization algorithms find diverse applications: SA addressing protein folding problems to predict natural protein conformations [32], GA optimizing chemical reaction networks to boost product yields [33], and PSO showing promise in predicting the structure of layered materials [34]. As a whole, these algorithms have catalyzed substantial advancements in fields such as chemistry and materials science, promising substantial potential for future breakthroughs.

迭代搜索方法是优化算法中的关键工具和核心机制。例如，SA模拟固体中的退火过程以确定最低能量状态。GA利用生物进化机制，通过反复选择、交叉和变异产生最优解。PSO借鉴鸟群或鱼群的社会行为，修改每个“粒子”的位置和速度以找到最优解。这些优化算法有多种应用:SA用于解决蛋白质折叠问题以预测天然蛋白质构象 [32]，GA用于优化化学反应网络以提高产品产量 [33]，PSO在预测层状材料结构方面显示出前景 [34]。总体而言，这些算法推动了化学和材料科学等领域的重大进展，有望在未来取得重大突破。

Fig. 2. Challenges in the chemistry and materials science. Three categories of challenges: substance screening, mechanism exploration, and process optimization. For substance screening, such as drug virtual screening [6], material high-throughput screening [7], etc.; for mechanism exploration, such as kinetic mechanism exploration [8], structure-activity relationship exploration [9], etc.; for process optimization, such as synthesis route optimization, reaction conditions optimization etc.

图2. 化学和材料科学中的挑战。三类挑战:物质筛选、机理探索和过程优化。对于物质筛选，如药物虚拟筛选 [6]、材料高通量筛选 [7]等；对于机理探索，如动力学机理探索 [8]、构效关系探索 [9]等；对于过程优化，如合成路线优化、反应条件优化等。

Fig. 1. Classification of intelligent algorithms. Intelligent algorithm is composed of ML, DL, and OA. Notably, DL, rooted in neural network architectures, constitutes a subset of ML. Representative algorithms for these domains are depicted in the accompanying figure.

图1. 智能算法分类。智能算法由机器学习(ML)、深度学习(DL)和优化算法(OA)组成。值得注意的是，DL基于神经网络架构，是ML的一个子集。这些领域的代表性算法在附图中示出。

1.3. Adapting intelligent algorithms to the specific demands of chemistry and materials science

1.3. 使智能算法适应化学和材料科学的特定需求

ML, DL, and optimization algorithms each exhibit unique problem applicability, strengths, and limitations. The choice of algorithm should align with the specific characteristics and demands of the problem at hand, considering factors such as data type (labelled or unlabeled), problem complexity, and available computational resources.

机器学习、深度学习和优化算法各自具有独特的问题适用性、优势和局限性。算法的选择应与手头问题的特定特征和需求相匹配，要考虑数据类型(有标签或无标签)、问题复杂性和可用计算资源等因素。

ML leverages statistical methods to discern and predict underlying data patterns, particularly for problems with clear labels such as reaction activation energies prediction [35] and molecular property prediction [36]. Its advantages include lucid data distribution interpretation and effective input-output relationship modelling. However, ML requires domain expertise and manual feature engineering when grappling with complex, high-dimensional, non-linear problems. In juxtaposition, DL manifests as a powerful tool when confronted with tasks that demand automated extraction and deciphering of inherent features directly from raw data, a scenario exemplified by molecular representation learning [37]. Despite DL’s prowess in recognizing patterns and condensing features within high-dimensional data, it invariably necessitates substantial datasets and computational horsepower for peak performance. Transitioning focus, optimization algorithms chiefly address the search for the best solutions amid a sea of possibilities. The allure of these algorithms stems from their efficiency and global search capacity in pinning down the optimal solution. However, the practical efficacy of these tools often is inherently tethered to the specifics of a given problem, such as the morphology of the objective function and the dimension of the search space.

机器学习利用统计方法来识别和预测潜在的数据模式，特别是对于具有明确标签的问题，如反应活化能预测[35]和分子性质预测[36]。其优点包括清晰的数据分布解释和有效的输入-输出关系建模。然而，在处理复杂、高维、非线性问题时，机器学习需要领域专业知识和手动特征工程。相比之下，深度学习在面对需要直接从原始数据中自动提取和解读固有特征的任务时，是一个强大的工具，分子表示学习[37]就是一个例子。尽管深度学习在识别高维数据中的模式和压缩特征方面表现出色，但它始终需要大量的数据集和计算能力才能达到最佳性能。转而关注优化算法，其主要解决在众多可能性中寻找最佳解决方案的问题。这些算法的吸引力在于它们在确定最优解方面的效率和全局搜索能力。然而，这些工具的实际效果往往固有地依赖于给定问题的具体情况，如目标函数的形态和搜索空间的维度。

Additionally, the requirements of chemistry and materials science, as compared to generic modelling scenarios, manifest through several key considerations: the high dimensionality of data; the inherent structural intricacy of data, such as molecular configurations and crystal arrangements; the necessity for domain-specific expertise; the presence of data uncertainties and noise; multi-scale complexities; and a focused attention towards model transparency and interpretability, given its implications for scientific discoveries. Accordingly, the design and adjustment of intelligent algorithms should be specifically tailored to these challenges, to ensure their aptness for addressing such multifaceted requirements.

此外，与一般建模场景相比，化学和材料科学的要求体现在几个关键方面:数据的高维度；数据固有的结构复杂性，如分子构型和晶体排列；领域特定专业知识的必要性；数据不确定性和噪声的存在；多尺度复杂性；以及鉴于模型透明度和可解释性对科学发现的影响而对其给予的特别关注。因此，智能算法的设计和调整应专门针对这些挑战进行定制，以确保它们适合满足此类多方面的要求。

2. Data collection and preprocessing

2. 数据收集与预处理

Intelligent algorithm research pivots on three pillars: data acquisition, feature engineering, and algorithm selection (Fig. 3). Standardized data and feature engineering form the bedrock upon which intelligent algorithms operate, enabling them to effectively sift valuable insights from the labyrinth of complex data, construct precise models, and execute robust predictions.

智能算法研究围绕三个支柱展开:数据采集、特征工程和算法选择(图3)。标准化数据和特征工程构成了智能算法运行的基石，使它们能够有效地从复杂数据的迷宫中筛选出有价值的见解，构建精确的模型，并进行可靠的预测。

2.1. Database and data extraction

2.1. 数据库与数据提取

Numerous chemical databases provide researchers with vast avenues for information acquisition. Here, we briefly encapsulate several prevalent chemical materials databases. PubChem [38], proffered by the NCBI (National Center for Biotechnology Information) in the U.S., is a freely accessible database of vast quantities of compounds, substances, and bioactivity information. ChemSpider [39], an online chemical database furnished by the Royal Society of Chemistry, amalgamates information on over 67 million chemical substances. ChEMBL [40], a database dedicated to drug development, encompasses a plethora of drug compounds, targets, and bioactivity data. ZINC [41], a chemical database designed specifically for virtual screening, contains a multitude of purchasable compounds. The CSD (Cambridge Structural Database) [42] stands as the most comprehensive repository of known organic and metal-organic crystal structures worldwide. Each of these databases furnishes Python APIs for users to extract data in bulk, further facilitating their utility. The iBonD [43] is a user-friendly internet-based databank of heterolytic (pKa) and homolytic (BDE) bond dissociation energies, established by the bond energy team at Tsinghua University and Nankai University, China. The iBonD 2.0 provides the heretofore most comprehensive bond energy collection and the most convenient approach to find the data with its powerful searching engine. The ORD (Open Reaction Database) [44] stands as a comprehensive archive, brimming with detailed chemical reaction information, thereby serving as a foundational data source for practitioners of synthetic chemistry.

众多化学数据库为研究人员提供了获取信息的广阔途径。在此，我们简要概述几个常见的化学材料数据库。美国国立生物技术信息中心(NCBI)提供的PubChem [38] 是一个可免费访问的数据库，包含大量化合物、物质和生物活性信息。英国皇家化学学会提供的在线化学数据库ChemSpider [39] 整合了超过6700万种化学物质的信息。致力于药物开发的数据库ChEMBL [40] 包含大量药物化合物、靶点和生物活性数据。专门用于虚拟筛选的化学数据库ZINC [41] 包含大量可购买的化合物。剑桥结构数据库(CSD)[42] 是全球已知有机和金属有机晶体结构最全面的存储库。这些数据库均为用户提供了Python API，以便批量提取数据，进一步提高了它们的实用性。iBonD [43] 是一个用户友好的基于互联网的异裂(pKa)和均裂(BDE)键解离能数据库，由中国清华大学和南开大学的键能团队建立。iBonD 2.0提供了迄今为止最全面的键能集合，并通过其强大的搜索引擎提供了查找数据最便捷的方法。开放反应数据库(ORD)[44] 是一个全面的存档库，充满了详细的化学反应信息，从而成为合成化学从业者的基础数据源。

The MP (Materials Project) [45], one of the most commonly used materials databases, provides comprehensive data on materials, encompassing theoretical calculations of crystal structures, electronic structures, magnetism, and more. Pymatgen [46], a powerful open-source Python library for materials analysis, proffers highly adaptable classes for representing elements, sites, molecules, and structural objects, along with rich input/output support, encapsulating file formats such as VASP, ABINIT, CIF, among others. Researchers can leverage Pymatgen for seamless integration with the Materials Project REST API. The ASE (Atomic Simulation Environment) [47] is a Python library used for setting up, manipulating, running, and analysing atomic simulations, allowing researchers to perform quantum mechanics and molecular dynamics simulations with greater ease. The CMR (Computational Materials Repository) [48], a multi-topic materials database site based on ASE, includes databases on two-dimensional materials, functional perovskites, and more. The OQMD (Open Quantum Materials Database) [49] is a materials database predicated on first-principle calculations, providing a multitude of information on structures, band gaps, thermodynamics, electronic properties, and more.

材料项目(MP)[45] 是最常用的材料数据库之一，提供有关材料的全面数据，包括晶体结构、电子结构、磁性等的理论计算。Pymatgen [46] 是一个功能强大的用于材料分析的开源Python库，提供了高度可定制的类来表示元素、位点、分子和结构对象，以及丰富的输入/输出支持，涵盖VASP、ABINIT、CIF等文件格式。研究人员可以利用Pymatgen与材料项目REST API进行无缝集成。原子模拟环境(ASE)[47] 是一个用于设置、操作、运行和分析原子模拟的Python库，使研究人员能够更轻松地进行量子力学和分子动力学模拟。基于ASE的多主题材料数据库网站计算材料存储库(CMR)[48] 包括二维材料、功能钙钛矿等数据库。开放量子材料数据库(OQMD)[49] 是一个基于第一性原理计算的材料数据库，提供有关结构、带隙、热力学、电子性质等的大量信息。

Fig. 3. Three elements of intelligent algorithm research. Intelligent algorithm research pivots on three pillars: data acquisition, feature engineering, and algorithm selection. Data can be sourced from databases, literature mining, simulations, or high-throughput experiments. Effective descriptor design, fortified by domain knowledge, is paramount in feature engineering. Algorithm selection hinges on data volume and problem specificity, with potential enhancements from model architecture or physical constraints.

图3. 智能算法研究的三个要素。智能算法研究围绕三个支柱展开:数据获取、特征工程和算法选择。数据可以来自数据库、文献挖掘、模拟或高通量实验。在特征工程中，借助领域知识进行有效的描述符设计至关重要。算法选择取决于数据量和问题的特殊性，并可能通过模型架构或物理约束得到改进。

2.2. Leveraging literature: data extraction and utilization

2.2. 利用文献:数据提取与利用

In the realm of chemistry, the literature landscape is replete with a wealth of original data, rendering it a vital artery for researchers on their quest for data resources. Yet, the manual extraction of data from such extensive literature archives proves to be a daunting task, both laborious and time-consuming. The intensity and speed of this manual task scarcely align with the rapid proliferation of literature volumes. More critically, the non-standardized and unsystematic nature of manual data extraction impedes effective data sharing and unified utilization, thus presenting significant challenges. Given this context, the deployment of Natural Language Processing (NLP) [50] technology to automate the dissection and extraction of data from chemical literature is gaining prominence. NLP, with its goal of equipping computers with the capacity to comprehend and apply human language, offers efficient parsing and abstraction capabilities for text, thus paving the way for large-scale data mining of chemical literature.

在化学领域，文献中充斥着大量原始数据，这使其成为研究人员获取数据资源的重要渠道。然而，从如此海量的文献档案中手动提取数据是一项艰巨的任务，既费力又耗时。这项手动任务的强度和速度与文献数量的快速增长极不匹配。更关键的是，手动数据提取的非标准化和不系统性阻碍了有效的数据共享和统一利用，从而带来了重大挑战。在此背景下，部署自然语言处理(NLP)[50]技术以自动剖析和提取化学文献中的数据正变得越来越重要。NLP旨在使计算机具备理解和应用人类语言的能力，为文本提供高效的解析和抽象能力，从而为化学文献的大规模数据挖掘铺平道路。

In the domain of text mining, several crucial steps are involved, including the batch download of documents, which is then followed by NLP tasks such as preprocessing, text classification, table parsing, named entity recognition, and entity relationship extraction (Fig. 4). A paramount task in scientific research is the batch downloading of literature. In response to this, an array of methods has been designed to automatically acquire scientific articles based on Digital Object Identifiers (DOIs), largely harnessing publisher-provided API interfaces like Elsevier’s Scopus and ScienceDirect APIs, Springer Nature’s Spring-erLink API, and Wiley’s Online Library API, among others. Once the documents are downloaded, they undergo preprocessing to exclude nontarget text and ensure data consistency and accuracy. HTML and XML files are parsed, often employing libraries such as Python’s Beau-tifulSoup [51] or lxml [52], to extract text effectively between tags. Following the extraction, related text blocks are merged for coherence, and non-target metadata such as article numbers, and publication information are filtered out through specific rules or regular expressions. Furthermore, to avoid discrepancies that might hamper subsequent data analysis and information extraction, it’s crucial to standardize the format of data units, such as temperature, pressure, and concentration units. This is typically accomplished using mapping dictionaries [53]. The processed text is then divided into words or phrases and subjected to stem extraction or lemmatization to minimize the impact of vocabulary morphological changes on data analysis. This phase generally employs a stem extraction or lemmatization algorithm such as the Porter algorithm [54].

在文本挖掘领域，涉及几个关键步骤，包括文档的批量下载，随后是诸如预处理、文本分类、表格解析、命名实体识别和实体关系提取等NLP任务(图4)。科学研究中的一项重要任务是文献的批量下载。为此，已经设计了一系列方法来基于数字对象标识符(DOI)自动获取科学文章，主要利用出版商提供的API接口，如爱思唯尔的Scopus和ScienceDirect API、施普林格自然的Spring-erLink API以及威利的在线图书馆API等。文档下载后，进行预处理以排除非目标文本并确保数据的一致性和准确性。解析HTML和XML文件，通常使用诸如Python的BeautifulSoup [51]或lxml [52]等库，以有效地提取标签之间的文本。提取后，合并相关文本块以使其连贯，并通过特定规则或正则表达式过滤掉诸如文章编号和出版信息等非目标元数据。此外，为避免可能妨碍后续数据分析和信息提取的差异，标准化数据单位的格式(如温度、压力和浓度单位)至关重要。这通常使用映射字典[53]来完成。然后将处理后的文本分成单词或短语，并进行词干提取或词形还原，以尽量减少词汇形态变化对数据分析的影响。此阶段通常采用诸如波特算法[54]等词干提取或词形还原算法。

Within the text mining landscape, text classification serves as a pivotal role, filtering out salient information from vast full-text sentences through a meticulous organization based on specific keywords or semantic traits. This process is bolstered by semi-automated dictionaries utilizing word vector techniques such as Word2Vec [55] or GloVe [56], and rule-based recognition methods. Despite the manual rule-setting requirement, the latter demonstrates impressive accuracy for structured text. Table parsing transforms raw XML and HTML table data into structured cell arrays. It begins with extraction of table content using explicit tags, resulting in nested lists representing table rows and cells. Potential challenges, such as cell spanning or data inconsistency, are addressed during this process. Named entity recognition, another crucial element, involves automated identification and categorization of named entities, often aided by regular expressions matching entity expressions in a specific literature field. Finally, entity relationship extraction identifies semantic links between named entities in the text. Rule-based methods, such as dependency parsing [57], offer high precision, but they require substantial manual effort and continuous rule updates to accommodate complex or evolving text structures. Pattern-matching methods, such as those using neural network architectures like Transformer models [58], capture more complex relational patterns but require substantial training data. Convolutional neural networks adapted for graph structured data, such as Graph Convolutional Networks (GCN) [59], are also instrumental in discerning complex patterns and relationships in text. While they can efficiently extract local and global features from the data, their performance relies heavily on the quality and amount of training data available. In conclusion, each method, from rule-based approaches to ML algorithms, carries its unique applications and limitations, necessitating a careful selection based on practical needs.

在文本挖掘领域，文本分类起着关键作用，通过基于特定关键词或语义特征的精心组织，从大量全文句子中筛选出重要信息。这个过程由利用诸如Word2Vec [55]或GloVe [56]等词向量技术的半自动字典以及基于规则的识别方法来支持。尽管需要手动设置规则，但后者在处理结构化文本时显示出令人印象深刻的准确性。表格解析将原始XML和HTML表格数据转换为结构化单元格数组。它首先使用显式标签提取表格内容，得到表示表格行和单元格的嵌套列表。在此过程中解决诸如单元格跨列或数据不一致等潜在挑战。命名实体识别是另一个关键要素，涉及自动识别和分类命名实体，通常借助正则表达式匹配特定文献领域中的实体表达式。最后，实体关系提取识别文本中命名实体之间的语义链接。基于规则的方法，如依存句法分析[57]，具有高精度，但需要大量人工工作和持续更新规则以适应复杂或不断演变的文本结构。模式匹配方法，如使用诸如Transformer模型[58]等神经网络架构的方法，能够捕捉更复杂的关系模式，但需要大量训练数据。适用于图结构数据的卷积神经网络(如Graph Convolutional Networks (GCN) [59])也有助于识别文本中的复杂模式和关系。虽然它们可以有效地从数据中提取局部和全局特征，但其性能严重依赖于可用训练数据的质量和数量。总之，从基于规则的方法到机器学习算法，每种方法都有其独特的应用和局限性，需要根据实际需求仔细选择。

2.3. Data cleaning and feature engineering

2.3. 数据清洗与特征工程

Data cleansing forms the cornerstone of data preprocessing, addressing pivotal concerns like missing data, outliers, and data duplication. For missing values, typical strategies involve excluding data with absent information, employing statistical techniques such as mean, median, or mode imputation, or interpolating using predictive models. Outliers can be managed through their identification followed by elimination, replacement, or through variable transformations. To maintain the uniqueness of the data set, the removal of duplicate entries is recommended. Beyond data cleaning, data transformation refines raw data into forms more amenable to analysis or model construction. These transformations can take multiple forms, including logarithmic conversions, standardization, or normalization. Standardization and normalization, essential facets of data normalization, alter the scale and distribution of data to fit a unified standard, thereby improving the effectiveness of subsequent data analysis and model development. Python’s numpy [60] library offers comprehensive utilities for both data cleaning and transformation.

数据清洗是数据预处理的基石，它解决了诸如缺失数据、异常值和数据重复等关键问题。对于缺失值，典型策略包括排除漏失信息的数据，采用均值、中位数或众数插补等统计技术，或使用预测模型进行插值。异常值可以通过识别后进行消除、替换或通过变量变换来处理。为保持数据集的唯一性，建议删除重复条目。除了数据清洗，数据变换将原始数据提炼成更适合分析或模型构建的形式。这些变换可以有多种形式，包括对数转换、标准化或归一化。标准化和归一化作为数据归一化的重要方面，改变数据的规模和分布以符合统一标准，从而提高后续数据分析和模型开发的有效性。Python的numpy [60]库为数据清洗和变换提供了全面的实用工具。

In the realm of cheminformatics, molecular descriptors act as numerical quantifiers of a vast array of compound-specific structural, physical, and chemical data. These descriptors are broadly categorized into zero-, one-, two-, and three-dimensional classifications. Zero-dimensional descriptors, which include fundamental physicochemical properties such as boiling points, solubility, and pKa, are readily ascertainable and conceptually straightforward. However, their utility is somewhat curtailed by their inability to capture the complexities of advanced molecular structures. One-dimensional descriptors include character encoding methods such as SMILES [61]. A SMILES string consists of a series of characters, including atomic symbols, bond connections, and parentheses to represent branched structures. Venturing into the realm of two-dimensional descriptors, these not only encompass the molecular topology but also a variety of molecular fingerprints, including but not limited to MACCS Keys [62] and Morgan [62], which can be derived from SMILES. Their extensive applicability notwithstanding, they fall short in depicting the three-dimensional conformation and stereochemical features of molecules. The three-dimensional descriptors, including Stereochemical Descriptors, Geometric Descriptors, and Quantum Chemical Descriptors, offer a heightened level of accuracy, delving into the molecular conformation with finesse. However, this precision comes at the price of elevated computational demands and a pronounced sensitivity to molecular conformational variations, necessitating rigorous conformation generation and refinement. In an encouraging development, Liu et al. have introduced Auto3D [63], a notably efficient algorithm facilitating the swift acquisition of molecules’ optimal conformations. The novel geometrically enhanced molecular representation learning method (GEM) proposed by Fang [64] and others is characterized by representing a molecule with two graphs simultaneously: one is an atom-bond graph, and the other is a bond-angle graph. This design effectively addresses the issue of three-dimensional spatial description of molecules. In addition to these previously discussed descriptor types, innovative approaches continue to emerge, such as the spectroscopic descriptors [65] proposed by Jiang et al., harnessing infrared and Raman spectral signals to forecast molecular adsorption-a testament to the ever-evolving landscape of molecular descriptor methodologies. The above techniques are judiciously employed by researchers to encapsulate the dimensional space of chemical reactions [66,67]. However, it is pivotal to note that the aforementioned descriptors, while widely applicable, may not be universally fitting for all problem sets. Hence, it is of crucial importance to design and utilize descriptors that hold a high degree of chemical relevance specific to the problem at hand, manifesting as feature engineering methods embedding domain knowledge. Illustratively, researchers, when employing ML to predict enantioselectivity [68], have developed a novel spatial descriptor, termed as Average Space Occupancy (ASO), recognizing the critical role spatial environment plays in chiral induction. In tandem, they have utilized electrostatic potential as a stereoelectronic descriptor. This strategic approach has yielded exceptional predictive accuracy, with a Mean Absolute Error (MAE) as low as 0.144 kcal/mol.

在化学信息学领域，分子描述符是大量化合物特定结构、物理和化学数据的数值量化指标。这些描述符大致分为零维、一维、二维和三维分类。零维描述符包括诸如沸点、溶解度和pKa等基本物理化学性质，易于确定且概念简单。然而，它们无法捕捉高级分子结构的复杂性，其效用有所受限。一维描述符包括诸如SMILES [61]等字符编码方法。一个SMILES字符串由一系列字符组成，包括原子符号、键连接以及表示分支结构的括号。进入二维描述符领域，这些描述符不仅涵盖分子拓扑结构，还包括各种分子指纹，包括但不限于MACCS Keys [62]和Morgan [62]，它们可以从SMILES推导得出。尽管它们具有广泛的适用性，但在描绘分子的三维构象和立体化学特征方面存在不足。三维描述符，包括立体化学描述符、几何描述符和量子化学描述符，提供了更高的准确性，能够精细地探究分子构象。然而，这种精确性是以更高的计算需求和对分子构象变化的显著敏感性为代价的，这需要严格的构象生成和优化。令人鼓舞的是，Liu等人引入了Auto3D [63]，这是一种特别高效的算法，有助于快速获取分子的最佳构象。Fang [64]等人提出的新颖的几何增强分子表示学习方法(GEM)的特点是同时用两个图表示一个分子:一个是原子-键图，另一个是键角图。这种设计有效地解决了分子三维空间描述的问题。除了上述讨论的描述符类型，创新方法不断涌现，例如Jiang等人提出的光谱描述符 [65]，利用红外和拉曼光谱信号预测分子吸附——这证明了分子描述符方法的不断发展。研究人员明智地运用上述技术来封装化学反应的维度空间 [66,67]。然而，需要注意的是，上述描述符虽然广泛适用，但可能并非普遍适用于所有问题集。因此，设计和使用与手头问题具有高度化学相关性的描述符至关重要，这表现为嵌入领域知识的特征工程方法。例如，研究人员在使用机器学习预测对映选择性 [68]时，开发了一种新颖的空间描述符，称为平均空间占有率(ASO)，认识到空间环境在手性诱导中所起的关键作用。同时，他们将静电势用作立体电子描述符。这种策略性方法产生了出色的预测准确性，平均绝对误差(MAE)低至0.144 kcal/mol。

Fig. 4. General process of text mining. First, batch-download documents via the API. Next, preprocess to eliminate extraneous content. Conduct named entity recognition, isolating desired materials or compounds. Subsequently, correlate these entities with performance descriptors, culminating in database construction.

图4. 文本挖掘的一般过程。首先，通过API批量下载文档。接下来，进行预处理以消除无关内容。进行命名实体识别，分离出所需的材料或化合物。随后，将这些实体与性能描述符相关联，最终构建数据库。

Table 1

表1

Comparison of Different Molecular Descriptors.

不同分子描述符的比较

Feature selection stands as a prerequisite to predictive model implementation and bears a significant influence on high-dimensional data handling. Its chief aim is to discern the subset of features most pertinent to the task at hand, enabling effective data dimension reduction, enhanced model efficiency, lower risk of overfitting, and improved model interpretability. Filtering, a foundational feature selection method, relies on statistical characteristics between data features. Filtering methods rank features based on individual feature scores using various statistical techniques. These techniques, such as chi-square tests and ANOVA, employ common criteria such as the Pearson correlation coefficient, information gain, and mutual information. Despite their simplicity and computational efficiency, filtering methods might neglect feature-to-feature interactions. In contrast, the wrapper method, exemplified by algorithms like backward elimination, forward selection, or bi-directional elimination, approaches feature selection as a search problem, using model performance as a measure to determine the quality of feature subsets. Recursive Feature Elimination (RFE) [69], another wrapper method, iteratively builds models and identifies the most and least effective features, incorporating or excluding them from the model. This process continues until all features are exhausted. While wrapper methods can discover feature subsets with superior performance than filtering methods, they come with a higher computational expense. Embedding methods perform feature selection during model training. Certain ML models, such as Decision Tree [70], for instance, assign coefficients or importance scores to features during training, enabling the ranking and selection of the most crucial features. The strength of embedding methods lies in their ability to consider model performance while maintaining relatively low computational costs. Further, in cheminformatics, where feature numbers can dramatically surpass the sample size, sparse representation-based feature selection methods such as Lasso [71], Ridge [72], and Elastic Net [73] prove beneficial.

特征选择是预测模型实现的前提条件，对高维数据处理具有重大影响。其主要目的是识别与手头任务最相关的特征子集，从而实现有效的数据降维、提高模型效率、降低过拟合风险并增强模型可解释性。过滤是一种基础的特征选择方法，它依赖于数据特征之间的统计特性。过滤方法使用各种统计技术根据单个特征分数对特征进行排名。这些技术，如卡方检验和方差分析，采用皮尔逊相关系数、信息增益和互信息等通用标准。尽管过滤方法简单且计算效率高，但可能会忽略特征之间的相互作用。相比之下，包装法以反向消除、正向选择或双向消除等算法为例，将特征选择视为一个搜索问题，使用模型性能作为衡量特征子集质量的指标。递归特征消除(RFE)[69]是另一种包装法，它迭代地构建模型并识别最有效和最无效的特征，将它们纳入或排除在模型之外。这个过程一直持续到所有特征都被用尽。虽然包装法可以发现比过滤方法性能更优的特征子集，但它们的计算成本更高。嵌入法在模型训练期间进行特征选择。某些机器学习模型，例如决策树[70] 在训练期间会为特征分配系数或重要性分数，从而能够对最关键的特征进行排名和选择。嵌入法的优势在于能够在保持相对较低计算成本的同时考虑模型性能。此外，在化学信息学中，特征数量可能会大大超过样本大小，基于稀疏表示的特征选择方法，如套索回归[71]、岭回归[72]和弹性网络[73]被证明是有益的。

3. Integrating artificial intelligence in chemistry and materials science

3. 将人工智能集成到化学和材料科学中

In the following section, we delve into an intensive exploration of several key intelligent algorithms, elucidating their core principles and underscoring their utility within the realms of chemistry and materials science.

在接下来的部分中，我们将深入探讨几种关键的智能算法，阐明它们的核心原理，并强调它们在化学和材料科学领域中的实用性。

3.1. Application of ML algorithms

<### 3.1. 机器学习算法的应用

Random Forests (RF) [74], characterized as a robust supervised learning approach, consist of a multitude of decision trees, each individually trained on different random subsets of the data. During prediction, a consensus is reached through voting on the results from all decision trees, producing the final prediction. The inherent ensemble learning strategy of RF significantly enhances the accuracy and stability of predictions. RF is a fundamentally versatile tool, apt for a wide array of classification and regression challenges within the chemical and materials domains. Remarkably, it often achieves commendable predictive accuracy without the necessity for meticulous parameter tuning. This quality makes RF particularly beneficial in scenarios where the volume of available samples is limited. In a collaborative study by the Doyle Group [75] at Princeton University and researchers at Merck, the RF algorithm was employed to predict the yield of Buchwald-Hartwig coupling reactions (Fig. 5). Initially, they utilized a high-throughput screening platform to rapidly complete 4140 Buchwald-Hartwig coupling reactions, encompassing 15 substrates, 23 isoxazole additives, 4 palladium ligands, and 3 bases. Subsequently, employing domain-knowledge-embedded feature engineering techniques, the researchers extracted attributes from the reaction space through a combination of nuclear magnetic resonance (NMR) data and density functional theory (DFT) computations. This approach yielded a 120-dimensional feature vector. With the reaction yield serving as the label, a sample set was formed. Several common ML algorithms were applied for regression training, among which RF exhibited superior performance, predicting yields with an impressive ${\mathrm{R}}^{2}$ precision of 0.92 . Additionally, the RF model investigated the importance of features in predicting yield. The most significant descriptor was found to be the NMR carbon shift at the C3 position of the isoxazole. Inspired by this revelation, researchers undertook experiments to study reactions with isoxazole additives with varying C3 NMR shifts. They discovered that when the C3 NMR shift of isoxazole was large, an oxidative addition product of Pd and the N-O bond was yielded. Consequently, they concluded that the yield of Buchwald-Hartwig coupling reactions correlated with the competitive reaction of isoxazole and provided a mechanistic explanation. This process of mechanism exploration was indeed sparked by the rule identified by ML. Besides compound yield prediction, RF is also applied in the prediction of NMR chemical shifts [76], pharmacological effect prediction [77], and the forecasting of organic aerosol formation [78], among other areas.

随机森林(RF)[74]是一种强大的监督学习方法，由众多决策树组成，每个决策树都在数据的不同随机子集中单独训练。在预测过程中，通过对所有决策树的结果进行投票达成共识，从而产生最终预测。RF固有的集成学习策略显著提高了预测的准确性和稳定性。RF是一种从根本上来说用途广泛的工具，适用于化学和材料领域内的各种分类和回归挑战。值得注意的是，它通常无需进行细致的参数调整就能实现值得称赞的预测准确性。这种特性使得RF在可用样本数量有限的情况下特别有益。在普林斯顿大学的多伊尔小组[75]和默克公司的研究人员的一项合作研究中，RF算法被用于预测布赫瓦尔德-哈特维希偶联反应的产率(图5)。最初，他们利用高通量筛选平台迅速完成了4140个布赫瓦尔德-哈特维希偶联反应，涉及15种底物、23种异恶唑添加剂、4种钯配体和3种碱。随后，研究人员采用嵌入领域知识的特征工程技术，通过结合核磁共振(NMR)数据和密度泛函理论(DFT)计算从反应空间中提取属性。这种方法产生了一个120维的特征向量。以反应产率作为标签，形成了一个样本集。应用了几种常见的机器学习算法进行回归训练，其中RF表现出卓越的性能，以令人印象深刻的${\mathrm{R}}^{2}$精度0.92预测产率。此外，RF模型研究了特征在预测产率中的重要性。发现最重要的数据描述符是异恶唑C3位置的NMR碳位移。受此发现启发研究人员进行实验研究具有不同C3 NMR位移的异恶唑添加剂的反应。他们发现当异恶唑的C3 NMR位移较大时会产生Pd与N - O键的氧化加成产物。因此他们得出结论布赫瓦尔德-哈特维希偶联反应的产率与异恶唑的竞争反应相关并提供了一个机理解释。这种机理探索过程确实是由机器学习识别出的规则引发的。除了化合物产率预测，RF还应用于NMR化学位移预测[76]、药理效应预测[77]以及有机气溶胶形成预测[78]等其他领域。

Fig. 5. Intelligent algorithms forecasts the yield of coupling reactions. A) Data encompasses Buchwald-Hartwig cross-coupling reactions, sourced from Merck’s high-throughput screening. B) A 120-dimensional feature engineering approach describes these reactions. C) Among several ML algorithms, the RF model excels. D) Mean square error elucidates feature importance, with isoxazole relating to four of the top five features. Chemical insight based on ML models: Buchwald-Hartwig reaction yield interplays with isoxazole-based competing reactions.

<图5. 智能算法预测偶联反应的产率。A)数据涵盖来自默克高通量筛选的布赫瓦尔德-哈特维希交叉偶联反应。B)一种120维特征工程方法描述这些反应。C)在几种机器学习算法中，RF模型表现出色。D)均方误差阐明特征重要性，异恶唑与前五大特征中的四个相关。基于机器学习模型的化学见解:布赫瓦尔德-哈特维希反应产率与基于异恶唑的竞争反应相互作用。

Support Vector Machines (SVM) [12] are a type of supervised learning algorithm predominantly employed for classification problems. Within high-dimensional feature spaces, SVM identify an optimal hyperplane to maximize the margin between two classifications, thereby facilitating optimal categorization. For issues that are linearly inseparable, SVM apply kernel functions to map data onto a higher-dimensional space, enabling non-linear classification. The application of SVM algorithms has been initiated in areas such as materials screening. Rohit Batra and his team utilized SVM for the screening of highly water-stable Metal-Organic Frameworks (MOFs) [79]. The team collected water stability data for more than 200 MOFs and then constructed descriptors for both the metal nodes and organic ligands. The results highlighted the superior performance of the SVM algorithm in a three-class problem, suggesting its applicability in the rapid screening of highly water-stable MOFs. Sun and colleagues, employed SVM for screening high-performance organic photovoltaic (OPV) materials [80]. They initially established a database containing over 1700 OPV donor molecules, classifying materials as “low– performance” or “high-performance” based on their power conversion efficiency (PCE) values. Subsequently, they examined the influence of different types of molecular fingerprints on model performance, discovering that their trained models could rapidly screen OPV donor molecules. These instances of SVM applications illustrate the potential of this tool in materials science, particularly in the rapid and accurate screening of materials with specific desirable properties.

支持向量机(SVM)[12]是一种主要用于分类问题的监督学习算法。在高维特征空间中，SVM识别一个最优超平面以最大化两个分类之间的间隔，从而促进最优分类。对于线性不可分的问题，SVM应用核函数将数据映射到更高维空间以实现非线性分类。SVM算法的应用已在材料筛选等领域展开。罗希特·巴特拉和他的团队利用SVM筛选高度水稳定的金属有机框架(MOF)[79]。该团队收集了200多种MOF的水稳定性数据，然后为金属节点和有机配体构建了描述符。结果突出了SVM算法在一个三类问题中的卓越性能，表明其适用于快速筛选高度水稳定的MOF。孙及其同事利用SVM筛选高性能有机光伏(OPV)材料[80]。他们最初建立了一个包含1700多个OPV供体分子的数据库，根据其功率转换效率(PCE)值将材料分类为“低性能”或“高性能”。随后，他们研究了不同类型分子指纹对模型性能的影响，发现他们训练的模型可以快速筛选OPV供体分子。这些SVM应用实例说明了该工具在材料科学中的潜力，特别是在快速准确筛选具有特定理想特性的材料方面。

Clustering algorithms are a form of unsupervised learning primarily used to identify natural groupings within data sets. The core principle involves grouping data points based on a specific similarity measure or distance metric, ensuring high similarity amongst data points within the same group and low similarity between data points in different groups. Common clustering algorithms include K-means [81], Spectral Clustering [82], and Density-Based Spatial Clustering of Applications with Noise (DBSCAN) [83]. Clustering algorithms are frequently applied to the screening of materials and compounds with specific properties. By conducting straightforward clustering analyses, researchers can study the common properties within a class of materials or compounds, thus proposing new chemical laws or insights. Zhang and colleagues deployed Spectral Clustering and Agglomerative Hierarchical Clustering for screening solid-state lithium-ion conductors (SSLCs) with high conductivity [84]. Utilizing the Inorganic Crystal Structure Database (ICSD), they converted each structure into a structure vector, serving as the input for the model. The SSLCs were grouped into seven classes, followed by a structure-performance relationship study to explore commonalities amongst the structures within the high-conductivity class. Their clustering model successfully differentiated between rapid lithium conduction materials and poor lithium conduction materials, predicting 16 new compounds as solid-state lithium-ion conductors. These newly discovered candidate materials, bearing markedly different structures and chemical compositions compared to known rapid lithium-ion conductors, testify to the effectiveness of clustering methods in uncovering novel materials within extensive materials spaces.

聚类算法是一种无监督学习形式，主要用于识别数据集中的自然分组。其核心原理是基于特定的相似性度量或距离度量对数据点进行分组，确保同一组内的数据点具有高度相似性，而不同组的数据点之间具有低相似性。常见的聚类算法包括K均值算法[81]、谱聚类算法[82]和基于密度的空间聚类应用噪声算法(DBSCAN)[83]。聚类算法经常应用于筛选具有特定性质的材料和化合物。通过进行简单的聚类分析，研究人员可以研究一类材料或化合物中的共同性质，从而提出新的化学规律或见解。Zhang及其同事部署了谱聚类算法和凝聚层次聚类算法来筛选具有高电导率的固态锂离子导体(SSLC)[84]。利用无机晶体结构数据库(ICSD)，他们将每个结构转换为结构向量，作为模型的输入。将SSLC分为七类，然后进行结构-性能关系研究，以探索高电导率类中结构之间的共性。他们的聚类模型成功地区分了快速锂传导材料和不良锂传导材料，预测了16种新化合物为固态锂离子导体。这些新发现的候选材料与已知的快速锂离子导体相比，具有明显不同的结构和化学成分，证明了聚类方法在广阔的材料空间中发现新型材料的有效性。

Fig. 6. ${\mathrm{R}}_{\mathrm{f}}$ value prediction harnesses robotics and advanced algorithms. A) A bespoke robotic platform facilitates high-throughput experiments, yielding standardized TLC data. B) The TLC process is characterized using molecular fingerprints, molecular descriptors, and solvent traits. C) Multiple ML models predict with the Ensemble model leading in accuracy. D) The trained model forecasts ${\mathrm{R}}_{\mathrm{f}}$ values across solvent ratios, pinpointing the ratio that maximizes ${\mathrm{R}}_{\mathrm{f}}$ differences between compounds, thereby suggesting optimal separation conditions.

图6. ${\mathrm{R}}_{\mathrm{f}}$值预测利用了机器人技术和先进算法。A)定制的机器人平台有助于进行高通量实验，产生标准化的薄层色谱(TLC)数据。B)使用分子指纹、分子描述符和溶剂特性对TLC过程进行表征。C)多个机器学习模型进行预测，集成模型在准确性方面领先。D)训练后的模型预测不同溶剂比例下的${\mathrm{R}}_{\mathrm{f}}$值，找出使化合物之间${\mathrm{R}}_{\mathrm{f}}$差异最大化的比例，从而建议最佳分离条件。

3.2. Application of DL algorithms

3.2. 深度学习算法的应用

In the ensuing section, we elucidate the underlying mechanisms of prevalent DL algorithms, each bolstered by illustrative examples. We aim to illuminate their applications within the realm of chemistry and materials science, and expound on their role in propelling scientific discovery.

在接下来的部分中，我们将阐明流行的深度学习算法的潜在机制，并辅以示例说明。我们旨在阐明它们在化学和材料科学领域中的应用，并阐述它们在推动科学发现方面的作用。

3.2.1. ANN-based model

3.2.1. 基于人工神经网络的模型

ANN [85] have emerged as a cornerstone in the DL field, employing layered neuron connections to address diverse tasks such as classification, regression, and generation. The efficiency and stability of these models rely on a few key elements: initialization methods, normalization techniques, activation functions, and optimizers. Notable advancements in these areas have been made, such as the Kaiming [86] initialization method which improves convergence speed, and the P-swish [87] activation function that bolsters network performance. New optimizers, such as Adam [88] and AdamW [89], have emerged to expedite and stabilize network training. These developments collectively enhance the capabilities and applications of ANN in the field of chemistry. Although ANN has become quite advanced, it often has a large number of parameters and is prone to overfitting. Consequently, it is not recommended to use ANN when the amount of data is limited.

人工神经网络[85]已成为深度学习领域的基石，它通过分层神经元连接来处理分类、回归和生成等各种任务。这些模型的效率和稳定性依赖于几个关键要素:初始化方法、归一化技术、激活函数和优化器。在这些领域已经取得了显著进展，例如提高收敛速度的凯明[86]初始化方法，以及增强网络性能的P-swish[87]激活函数。新的优化器，如Adam[88]和AdamW[89]，已经出现以加速和稳定网络训练。这些进展共同增强了人工神经网络在化学领域的能力和应用。尽管人工神经网络已经相当先进，但它通常有大量参数且容易过拟合。因此，当数据量有限时，不建议使用人工神经网络。

Mo et al. have pioneered an automated thin-layer chromatography (TLC) platform [90,91], capable of collecting extensive experimental data under standardized conditions (Fig. 6). Leveraging this dataset, which encapsulates 4944 TLC records for 386 compounds, a range of ML techniques were employed to construct surrogate models reflecting the polarity correlations with organic compound structures and their respective retardation factors $\left( {R}_{f}\right)$ . The researchers have ingeniously assembled an ensemble algorithm incorporating ANN, RF, XGBoost [92] (XGB), and LightGBM [93] (LGB) to achieve impressive ${\mathrm{R}}_{\mathrm{f}}$ predictions $\left( {{\mathrm{R}}^{2} = {0.961}}\right)$ . By seamlessly merging robotics and ML, the trained ML models can deliver precise predictions for the ${\mathrm{R}}_{\mathrm{f}}$ value curves of organic compounds under varying solvent combinations, thereby offering general guidance for purification condition selection and hastening the generation and analysis of high-quality TLC data. Luo et al. utilized copious pKa data from the iBonD database to experiment with various ML algorithms for pKa prediction models [36]. They introduced a novel set of descriptors—structure and physical-organic-parameter-based descriptors (SPOC) to represent the electronic and structural features of molecules. Ultimately, ANN yielded optimal predictive performance with a mean absolute error (MAE) as low as ${0.87}\mathrm{{pKa}}$ units. These examples epitomize the potency and adaptability of ANN in the realm of chemistry.

Mo等人开创了一个自动化薄层色谱(TLC)平台[90,91]，能够在标准化条件下收集大量实验数据(图6)。利用这个包含386种化合物的4944条TLC记录的数据集，采用了一系列机器学习技术来构建反映与有机化合物结构及其各自保留因子$\left( {R}_{f}\right)$的极性相关性的替代模型。研究人员巧妙地组装了一种集成算法，将人工神经网络、随机森林、XGBoost92和LightGBM93结合起来，以实现令人印象深刻的${\mathrm{R}}_{\mathrm{f}}$预测$\left( {{\mathrm{R}}^{2} = {0.961}}\right)$。通过无缝融合机器人技术和机器学习，训练好的机器学习模型可以对不同溶剂组合下有机化合物的${\mathrm{R}}_{\mathrm{f}}$值曲线进行精确预测，从而为纯化条件选择提供一般指导，并加速高质量TLC数据的生成和分析。Luo等人利用来自iBonD数据库的大量pKa数据，对各种用于pKa预测模型的机器学习算法进行了实验[36]。他们引入了一组新的描述符——基于结构和物理有机参数的描述符(SPOC)来表示分子的电子和结构特征。最终，人工神经网络产生了最佳预测性能，平均绝对误差(MAE)低至${0.87}\mathrm{{pKa}}$单位。这些例子体现了人工神经网络在化学领域的效力和适应性。

The integration of domain knowledge into the training of ANN, known as Physics-Informed Neural Networks (PINNs) [94], expedites the learning process and enhances prediction accuracy. For instance, our group tried PINNs for ${\mathrm{R}}_{\mathrm{f}}$ value prediction [90], incorporating the chemical principle that the ${\mathrm{R}}_{\mathrm{f}}$ value of a compound increases with the polarity of the developing agent in the same system. When the derivative of ${\mathrm{R}}_{\mathrm{f}}$ with respect to the concentration of a highly polar solvent is less than zero, which represents an atypical physical scenario, they introduce a penalty term into the loss function, as shown as below :

将领域知识整合到人工神经网络的训练中，即所谓的物理信息神经网络(PINNs)[94]，可以加快学习过程并提高预测准确性。例如，我们小组尝试使用PINNs进行${\mathrm{R}}_{\mathrm{f}}$值预测[90]，纳入了化合物在同一系统中的${\mathrm{R}}_{\mathrm{f}}$值随展开剂极性增加的化学原理。当${\mathrm{R}}_{\mathrm{f}}$对高极性溶剂浓度的导数小于零时，这代表一种非典型物理情况，他们在损失函数中引入一个惩罚项，如下所示:

When $\frac{d{R}_{f}}{dC}$ is less than 0, ReLU $\left( {-\frac{d{R}_{f}}{dC}}\right)$ is positive, the loss function increases, and the model is punished; when $\frac{d{R}_{f}}{dC}$ is greater than or equal to 0, ReLU $\left( {-\frac{d{R}_{f}}{dC}}\right)$ is 0, and the loss function does not changes, the model is not subject to additional penalties. In this way, neural networks are guided to learn models that satisfy the laws of physics.

当$\frac{d{R}_{f}}{dC}$小于0时，ReLU$\left( {-\frac{d{R}_{f}}{dC}}\right)$为正，损失函数增加，模型受到惩罚；当$\frac{d{R}_{f}}{dC}$大于或等于0时，ReLU$\left( {-\frac{d{R}_{f}}{dC}}\right)$为0，损失函数不变，模型不受额外惩罚。通过这种方式，引导神经网络学习满足物理定律的模型。

When training an ANN, the focus is not just on prediction accuracy but also model interpretability, a crucial aspect for chemical discovery. To identify pivotal features impacting predicted chemical labels, an array of methods for computing feature importance has been introduced. These methods encompass Sensitivity Analysis [95], LIME (Local Interpretable Model-Agnostic Explanations) [96] and SHAP (SHapley Additive exPlanations) [97], among others. Our group employed Sensitivity Analysis to reveal, for the first time, that the Topological Polar Surface Area (TPSA), a molecular descriptor, exerted the most significant influence on ${\mathrm{R}}_{\mathrm{f}}$ values. This quantifiable correlation between polarity and ${\mathrm{R}}_{\mathrm{f}}$ values underscores the essential role of feature importance computation in chemical discovery.

在训练人工神经网络时，重点不仅在于预测准确性，还在于模型可解释性，这是化学发现的一个关键方面。为了识别影响预测化学标签的关键特征，已经引入了一系列计算特征重要性的方法。这些方法包括敏感性分析[95]、LIME(局部可解释模型无关解释)[96]和SHAP(SHapley加性解释)[97]等。我们小组首次使用敏感性分析揭示，分子描述符拓扑极性表面积(TPSA)对${\mathrm{R}}_{\mathrm{f}}$值影响最大。这种极性与${\mathrm{R}}_{\mathrm{f}}$值之间的可量化相关性强调了特征重要性计算在化学发现中的重要作用。

3.2.2. CNN-based model

3.2.2. 基于卷积神经网络的模型

Derived from ANNs, CNNs have become an essential tool in image processing and computer vision. With design principles inspired by biological visual systems, CNNs operate through convolution layers to capture local dependency patterns, non-linear activation function layers to enhance network complexity, and pooling layers to control model complexity. Notable variants such as Deep Residual Networks (ResNet [98]) help combat the vanishing gradient problem, facilitating training of deeper networks, while Dilated [99] Convolutions and Group [100] Convolutions offer tools for multi-scale information capture and reduced computational load, respectively.

卷积神经网络源自人工神经网络，已成为图像处理和计算机视觉中的一项重要工具。受生物视觉系统启发的设计原则，卷积神经网络通过卷积层来捕捉局部依赖模式，通过非线性激活函数层来增强网络复杂性，并通过池化层来控制模型复杂性。诸如深度残差网络(ResNet [98])等著名变体有助于解决梯度消失问题，便于训练更深的网络，而扩张卷积 [99] 和分组卷积 [100] 分别提供了多尺度信息捕获和降低计算量的工具。

Capitalizing on its efficacy in extracting salient features from image data, CNNs have been extensively utilized for image processing and recognition in materials science. Instrumental in shaping our understanding of material structure at the atomic scale, electron microscopes such as the Scanning Electron Microscope (SEM), Scanning Transmission Electron Microscope (STEM), and Transmission Electron Microscope (TEM) unfortunately often yield images susceptible to various distortions. Addressing this pervasive issue, Lobato et al. [101] proposed an innovative approach employing CNNs to correct these distortions. Their method, validated on an array of experimental images, demonstrated a substantial enhancement in the signal-to-noise ratio, thereby facilitating an augmented extraction of quantitative structural information from the images. Ziatdinov [102] et al. utilized CNNs for the automatic identification of atomic species and type of defects from scanning probe microscopy images, enhancing efficiency and precision at the atomic scale. This underscores the vital role of CNNs as an efficient instrument, adept at deciphering physical and chemical attributes, particularly when understanding of atomic arrangements and lattice irregularities is scant.

鉴于卷积神经网络在从图像数据中提取显著特征方面的有效性，它已被广泛用于材料科学中的图像处理和识别。诸如扫描电子显微镜(SEM)、扫描透射电子显微镜(STEM)和透射电子显微镜(TEM)等电子显微镜在塑造我们对原子尺度材料结构的理解方面发挥了重要作用，但不幸的是，它们产生的图像往往容易受到各种失真的影响。为了解决这个普遍问题，洛巴托等人 [101] 提出了一种创新方法，即使用卷积神经网络来校正这些失真。他们的方法在一系列实验图像上得到了验证，证明了信噪比有了显著提高，从而有助于从图像中增强定量结构信息的提取。齐亚季诺夫 [102] 等人利用卷积神经网络从扫描探针显微镜图像中自动识别原子种类和缺陷类型，提高了原子尺度的效率和精度。这凸显了卷积神经网络作为一种有效工具的重要作用，尤其在对原子排列和晶格不规则性了解甚少的情况下，它擅长解读物理和化学属性。

3.2.3. RNN-based model and transformer

<#### 3.2.3. 基于循环神经网络的模型和变换器

RNNs have become instrumental in managing sequential data, courtesy of their ability to retain historical information. Despite initial issues with handling long sequences, variations such as the Long Short-Term Memory (LSTM) [103] and the Gated Recurrent Unit (GRU) [104] have emerged to address these challenges. The LSTM’s gating mechanism allows it to handle long-term dependencies, while the GRU, a simplified version of the LSTM, reduces model complexity, demonstrating the versatility and adaptability of RNNs. In the realm of chemistry, RNNs and their variants have begun to demonstrate potential in the handling of time-series data, such as concentration-time curves. A breakthrough was achieved by Igor Larrosa [105] and colleagues in 2023, when they reported the first application of LSTM models in the study of chemical reaction kinetics, published in Nature. In their novel approach, they generated 5 million kinetics samples through the numerical solution of Ordinary Differential Equations (ODEs) for training and validating their deep neural network. Each sample, composed of four concentration-time distributions from a specific mechanism and a fixed set of kinetic constants, provided comprehensive kinetic information. By inputting the concentration-time series of a specific reaction into the LSTM model, they were able to predict the corresponding reaction mechanism with an astonishing accuracy of 99.9%. Their innovative methodology has been applied to a variety of catalytic reactions including olefin metathesis, cycloaddition, olefin isomerization, C-H amination, and photocatalytic hydrogenation, fostering the discovery of potential mechanisms.

循环神经网络在处理序列数据方面发挥了重要作用，这得益于它们能够保留历史信息的能力。尽管在处理长序列时最初存在问题，但诸如长短期记忆网络(LSTM)[103]和门控循环单元(GRU)[104]等变体已经出现，以应对这些挑战。长短期记忆网络的门控机制使其能够处理长期依赖关系，而作为长短期记忆网络简化版本的门控循环单元则降低了模型复杂性，展示了循环神经网络的通用性和适应性。在化学领域，循环神经网络及其变体已开始在处理时间序列数据(如浓度 - 时间曲线)方面展现出潜力。2023年，伊戈尔·拉罗萨[105]及其同事取得了一项突破，他们在《自然》杂志上发表了长短期记忆网络模型在化学反应动力学研究中的首次应用。在他们的新方法中，他们通过常微分方程(ODE)的数值解生成了500万个动力学样本，用于训练和验证他们的深度神经网络。每个样本由来自特定机制的四个浓度 - 时间分布和一组固定的动力学常数组成，提供了全面的动力学信息。通过将特定反应的浓度 - 时间序列输入到长短期记忆网络模型中，他们能够以惊人的99.9%的准确率预测相应的反应机制。他们的创新方法已应用于包括烯烃复分解、环加成、烯烃异构化、C - H胺化和光催化氢化在内的各种催化反应，促进了潜在机制的发现。

Since its inception in the pivotal paper “Attention is all you need” [106], the Transformer model has garnered widespread attention in the DL community, demonstrating superior performance across various tasks. The self-attention mechanism, the model’s cornerstone, brings substantial benefits in processing sequential data, particularly capturing long-range dependencies. Notably, this ability also enables parallel computation, redressing the limitations inherent in RNN-based models. The self-attention mechanism uniquely processes each input element by considering information from all elements, with reference weights determined by calculating the correlation between the target and other elements. This innovative approach underscores the transformative potential of Transformer models in DL tasks.

自其在具有里程碑意义的论文《你只需要注意力》[106]中诞生以来，变换器模型在深度学习社区中受到了广泛关注，在各种任务中都展示了卓越的性能。自注意力机制作为该模型的基石，在处理序列数据方面带来了显著优势，特别是在捕捉长程依赖关系方面。值得注意的是，这种能力还实现了并行计算，弥补了基于循环神经网络模型固有的局限性。自注意力机制通过考虑所有元素的信息来独特地处理每个输入元素，参考权重通过计算目标与其他元素之间的相关性来确定。这种创新方法凸显了变换器模型在深度学习任务中的变革潜力。

In the fields of chemistry and materials science, Transformer models are primarily applied in molecular representation learning, materials representation learning, product prediction, among other aspects. For instance, in molecular representation learning, researchers have proposed models like ChemBERTa [107] and MoLFormer [108]. These models initially represent compounds as SMILES, and leverage the Transformer architecture for pre-training. They capture intricate connections within molecular structures successfully and learn efficient molecular representations via the self-attention mechanism. Particularly noteworthy is the MoLFormer’s performance on quantum chemistry datasets such as QM9, where it outperforms baselines such as A-FP [109] and MPNN [110], underscoring the model’s aptitude in learning 3D molecular features. Elsewhere in organic chemistry, Alpha A. Lee and his team have utilized the Transformer model to predict chemical reaction products [111]. Drawing parallels to text translation in NLP, they represented reactants as reaction sequences and products as product sequences, leading to the development of a Transformer-based Seq2Seq model capable of accurate reaction product prediction. Moreover, in material representation learning, Kim [112] and colleagues proposed a Transformer-based framework for universal MOFs representation learning. They adopted a similar approach, constructing MOF’s local features through a crystal graph convolutional network and obtaining MOF’s global features through molecular dynamics simulation. Both are used as inputs to the Transformer model. Importantly, their work also incorporates interpretability studies, where the attention scores provide insight into which features exhibit significant correlations with the target. They found that, for hydrogen gas adsorption prediction in MOFs, local features are more critical, whereas for MOF’s Bandgap prediction, global features are more crucial, which is a conclusion that resonates with chemical intuition. Their findings spotlight the potential of Transformer models in enhancing model interpretability, thereby contributing to new discoveries in chemistry.

在化学和材料科学领域，变换器模型主要应用于分子表示学习、材料表示学习、产物预测等方面。例如，在分子表示学习中，研究人员提出了诸如ChemBERTa [107]和MoLFormer [108]等模型。这些模型最初将化合物表示为SMILES，并利用变换器架构进行预训练。它们成功地捕捉了分子结构内的复杂连接，并通过自注意力机制学习到高效的分子表示。特别值得注意 的是MoLFormer在诸如QM9等量子化学数据集上的性能，它优于诸如A - FP [109]和MPNN [110]等基线，突出了该模型在学习3D分子特征方面的能力。在有机化学的其他领域，阿尔法·A·李及其团队利用变换器模型预测化学反应产物[111]。与自然语言处理中的文本翻译类似，他们将反应物表示为反应序列，将产物表示为产物序列，从而开发出了一种基于变换器的Seq2Seq模型，能够准确预测反应产物。此外，在材料表示学习中，金[112]及其同事提出了一种基于变换器的通用金属有机框架表示学习框架。他们采用了类似的方法，通过晶体图卷积网络构建金属有机框架的局部特征，并通过分子动力学模拟获得金属有机框架的全局特征。两者都用作变换器模型的输入。重要的是，他们的工作还纳入了可解释性研究，其中注意力分数提供了对哪些特征与目标表现出显著相关性的洞察。他们发现，对于金属有机框架中的氢气吸附预测，局部特征更为关键，而对于金属有机框架的带隙预测，全局特征更为关键，这一结论与化学直觉相符。他们的发现凸显了变换器模型在增强模型可解释性方面的潜力，从而为化学领域的新发现做出贡献。

The advantages that models based on the Transformer bring to the materials and chemical sectors compared to traditional ML models are primarily manifested in the following aspects: Firstly, the self-attention mechanism of Transformer models allows the model to effectively capture dependencies over long ranges, which is especially important in understanding complex chemical structures and dynamic material reactions. Secondly, Transformer models can be pre-trained on large-scale datasets and then fine-tuned to specific material science and chemical issues, this transfer learning capability enables the model to quickly adapt and address new domain challenges. Thirdly, the Transformer architecture is inherently suited for parallel computation, meaning that when dealing with large-scale chemical and material databases, it can significantly speed up computation and learning. Based on these advantages, Transformer models will bring opportunities such as rapid knowledge extraction and integration, efficient material discovery, and prediction of complex chemical reactions, among others.

与传统机器学习模型相比，基于变换器的模型给材料和化学领域带来的优势主要体现在以下几个方面:首先，变换器模型的自注意力机制使模型能够有效地捕捉长程依赖关系，这在理解复杂化学结构和动态材料反应中尤为重要。其次，变换器模型可以在大规模数据集上进行预训练，然后针对特定的材料科学和化学问题进行微调，这种迁移学习能力使模型能够快速适应并应对新的领域挑战。第三，变换器架构天生适合并行计算，这意味着在处理大规模化学和材料数据库时，它可以显著加快计算和学习速度。基于这些优势，变换器模型将带来诸如快速知识提取和整合、高效材料发现以及复杂化学反应预测等机会。

3.2.4. GNN-based model

3.2.4. 基于图神经网络的模型

Emerging in recent years as a novel network structure within the domain of DL, Graph Neural Networks (GNNs) [113] are specifically designed to process graph-structured data. The fundamental operating principle of GNNs is rooted in the so-called message-passing mechanism. In this process, nodes disseminate their features, or “states”, to their neighboring nodes via the edges of the graph. The messages received are aggregated and subsequently employed to update the features of the nodes themselves. In this fashion, the features of nodes are dynamically updated and altered based on their neighborhood relationships within the graph. Obviously, GNNs are highly suited for tasks involving molecular representation learning as well as material representation learning (materials with explicit molecular or crystal structures), where atoms or molecules can act as nodes and chemical bonds or intermolecular interactions as edges. Particularly, network architectures like SchNet [114] can capture the complex features of molecules in three-dimensional space, something that is not achievable with traditional ML algorithms. SchNet, a tailor-made GNN for molecular and materials science conundrums, employs a continuous-filter convolution layer, thereby adroitly grasping the intricacies of atomic environments. This method confers an unparalleled exactness in delineating atomic interactions. Molecular Graph Convolutions (MGC) [115] innovatively adopts an adaptive strategy in manifesting graph representations of atoms and molecules. Further, sophisticated models like Attentive FP [109] and GraphDTA [116] seamlessly weave attention mechanisms into the processing of graph-structured data. This ingenious integration augments their ability to untangle the labyrinthine interplay between complex molecular architectures and their properties.

图神经网络(GNNs)[113] 作为深度学习领域近年来新兴的一种新型网络结构，专门用于处理图结构数据。GNNs 的基本运行原理根植于所谓的消息传递机制。在这个过程中，节点通过图的边将其特征或“状态”传播给相邻节点。接收到的消息被聚合起来，随后用于更新节点自身的特征。通过这种方式，节点的特征根据其在图中的邻域关系动态更新和改变。显然，GNNs 非常适合涉及分子表示学习以及材料表示学习(具有明确分子或晶体结构的材料)的任务，其中原子或分子可以充当节点，化学键或分子间相互作用充当边。特别是，像 SchNet [114] 这样的网络架构可以捕捉分子在三维空间中的复杂特征，这是传统机器学习算法无法实现的。SchNet 是一个为分子和材料科学难题量身定制的 GNN，它采用连续滤波卷积层，从而巧妙地把握原子环境的复杂性。这种方法在描绘原子相互作用方面具有无与伦比的精确性。分子图卷积(MGC)[115] 在展示原子和分子的图表示方面创新性地采用了自适应策略。此外，像注意力 FP [109] 和 GraphDTA [116] 这样的复杂模型将注意力机制无缝地融入到图结构数据的处理中。这种巧妙的整合增强了它们解开复杂分子结构与其性质之间错综复杂相互作用的能力。

The separation of enantiomers in chiral molecules constitutes a pivotal and challenging task in experimental chemistry, typically necessitating extensive trial-and-error procedures across various experimental setups. Our group innovatively employed a quantile geometry-enhanced graph neural network (QGeoGNN) to predict the elution times of chiral enantiomers during high-performance liquid chromatography (HPLC) separation (Fig. 7). The researchers initially gathered experimental data from 25,847 molecules, extracted from 644 articles on asymmetric catalysis, which encompassed SMILES, experimental information, HPLC column information, and corresponding retention times. We proceeded to construct the QGeoGNN, a novel network that transmutes the three-dimensional conformations of molecules into two graphs: the atom-bond graph (G) and the bond-angle graph (H). This design was conceived to offer superior representation of three-dimensional structures, thus equipping the network with the capacity to distinguish chiral enantiomers. The effectiveness of this approach far surpasses that of traditional ML algorithms. Additionally, domain knowledge of chromatography was assimilated into the ML model to facilitate multi-column prediction, laying a solid foundation for predictive chromatographic enantioseparation by calculating separation probabilities. The Crystal Graph Convolutional Neural Networks (CGCNNs) [117] distinguish themselves by adopting a distinct methodology for representing crystalline materials. These networks center around the notion of capturing local atomic environments and long-range crystallographic orders using a graph convolutional framework, ultimately enhancing the predictive power in material property forecasting. In a remarkable study, Sanyal and colleagues harnessed the CGCNN in conjunction with transfer learning to simultaneously predict a variety of material characteristics such as Formation Energy, Band Gap, and Fermi Energy across an expansive array of 46774 inorganic crystals [118].

手性分子中对映体的分离是实验化学中一项关键且具有挑战性的任务，通常需要在各种实验设置中进行大量的反复试验。我们团队创新性地采用了分位数几何增强图神经网络(QGeoGNN)来预测高效液相色谱(HPLC)分离过程中手性对映体的洗脱时间(图7)。研究人员最初从644篇关于不对称催化的文章中提取了25,847个分子的实验数据，这些数据包括SMILES、实验信息、HPLC柱信息以及相应的保留时间。我们接着构建了QGeoGNN，这是一种新型网络，它将分子的三维构象转化为两个图:原子 - 键图(G)和键角图(H)。这种设计旨在更好地表示三维结构，从而使网络具备区分手性对映体的能力。这种方法的有效性远远超过传统机器学习算法。此外，色谱领域知识被融入到机器学习模型中以促进多柱预测，通过计算分离概率为预测性色谱对映体分离奠定了坚实基础。晶体图卷积神经网络(CGCNNs)[117] 通过采用一种独特的方法来表示晶体材料而脱颖而出。这些网络围绕使用图卷积框架捕捉局部原子环境和长程晶体学顺序的概念展开，最终提高了材料性能预测的能力。在一项引人注目的研究中，Sanyal及其同事将CGCNN与迁移学习相结合，同时预测了46774种无机晶体中的多种材料特性，如形成能、带隙和费米能 [118]。

3.2.5. Generative model

3.2.5. 生成模型

Generative models operate by learning the intrinsic data distribution of a training set so that new data points can be created or ‘generated’. They are underpinned by a potent principle: if the model can accurately understand and capture the statistical properties of the input data, it can create novel instances that reflect the same underlying structures and characteristics. Prominent among these generative models are the Generative Adversarial Networks (GANs [119]), Variational Autoen-coders (VAEs [120]), and the recently emergent Diffusion Models [121]. In the fields of chemistry and materials science, generative models are primarily applied to the generation of molecules and materials. It should be noted that since generative models need to learn the statistical patterns of the input data, they often require a substantial amount of training data to ensure that the structures generated are both novel and reliable. However, the cost of data in these domains can be high, therefore, it is advisable to integrate various data augmentation techniques and computational chemistry methods, such as quantum mechanics or molecular dynamics simulations, to ensure a comprehensive training dataset.

生成模型通过学习训练集的内在数据分布来运行，以便可以创建或“生成”新的数据点。它们基于一个强有力的原则:如果模型能够准确理解和捕捉输入数据的统计特性，它就可以创建反映相同底层结构和特征的新实例。这些生成模型中突出的有生成对抗网络(GANs [119])、变分自编码器(VAEs [120])以及最近出现的扩散模型 [121]。在化学和材料科学领域，生成模型主要应用于分子和材料的生成。需要注意的是，由于生成模型需要学习输入数据的统计模式，它们通常需要大量的训练数据以确保生成的结构既新颖又可靠。然而，这些领域中的数据成本可能很高，因此，建议整合各种数据增强技术和计算化学方法，如量子力学或分子动力学模拟，以确保有一个全面的训练数据集。

Fig. 7. Predicting HPLC retention time with QGeoGNN. A) Retention time data from 644 papers is batch-extracted via NLP, encompassing molecular details and chromatographic conditions, with retention time as the label. B) HPLC chiral separation is represented through QGeoGNN, where molecules are rendered as atom-bond graphs and bond-angle graphs. By embedding experimental conditions and molecular descriptors, this method captures molecules’ 3D attributes, enhancing chiral distinction. C) QGeoGNN outperforms traditional GNN and ML algorithms in single-column prediction and multi-column prediction.

图7. 使用QGeoGNN预测HPLC保留时间。A)通过NLP从644篇论文中批量提取保留时间数据，涵盖分子细节和色谱条件，以保留时间作为标签。B)HPLC手性分离通过QGeoGNN表示，其中分子被渲染为原子键图和键角图。通过嵌入实验条件和分子描述符，该方法捕捉分子的3D属性，增强手性区分。C)QGeoGNN在单柱预测和多柱预测方面优于传统的GNN和ML算法。

GANs involves a generator (G), tasked with synthesizing convincing data from random noise, and a discriminator (D), committed to distinguishing whether the provided data is authentic or generated by G. Such an adversarial dynamic drives G to progressively excel at fabricating realistic data, while simultaneously sharpening D’s acumen in discerning genuine from generated data. Cao et al. unveiled the MolGAN [122] model, designed for the generation of novel compounds. Experimental validations on the QM9 chemical dataset revealed that their model could yield nearly 100% valid compounds, signifying a promising avenue for the discovery of new molecules.

GANs涉及一个生成器(G)，负责从随机噪声中合成令人信服的数据，以及一个判别器(D)，致力于区分提供的数据是真实的还是由G生成的。这种对抗动态驱使G逐渐擅长制造逼真的数据，同时提高D辨别真实数据和生成数据的敏锐度。Cao等人推出了MolGAN [122] 模型，专为生成新型化合物而设计。对QM9化学数据集的实验验证表明，他们的模型可以产生近100% 的有效化合物，这意味着发现新分子的一条有前景的途径。

VAEs encode input data into a latent space, followed by sampling from this space to generate new data. Alán [123] and colleagues proposed a method that leverages VAEs to encode molecular structures into a continuous latent space, followed by sampling to generate potential drug candidates. Jin [124] et al. proposed the JT-VAE model that employs a two-phase process for molecular graph generation, initially creating tree-like scaffolds based on chemical substructures, then amalgamating them into molecules employing a graph message-passing network. This approach enables the progressive expansion of molecules while preserving chemical validity at each stage. Furthermore, Yao [125] and collaborators have ingeniously employed VAEs for the generation of MOFs. By screening the MOFs generated by the model, they discovered high-performing MOFs for carbon dioxide separation that pose robust competition to some of the most effective MOFs and zeolites reported to date.

VAEs将输入数据编码到一个潜在空间，然后从这个空间进行采样以生成新数据。Alán [123] 及其同事提出了一种利用VAEs将分子结构编码到连续潜在空间，然后进行采样以生成潜在药物候选物的方法。Jin [124] 等人提出了JT-VAE模型，该模型采用两阶段过程进行分子图生成，首先基于化学子结构创建树状支架，然后使用图消息传递网络将它们合并成分子。这种方法能够在每个阶段保持化学有效性的同时逐步扩展分子。此外，Yao [125] 及其合作者巧妙地利用VAEs生成MOF。通过筛选模型生成的MOF，他们发现了用于二氧化碳分离的高性能MOF，与迄今为止报道的一些最有效的MOF和沸石形成了有力竞争。

Diffusion models have emerged in recent years as a captivating class of generative models. The underlying premise involves an iterative process of noise injection and sequential time steps, diffusing the data distribution gradually towards a known simple distribution, such as the Gaussian distribution. When generating new samples, the model guides data in reverse from the simple distribution along the diffusion path back to the original data distribution. Innovatively, Xu and colleagues introduced a novel generative model based on the diffusion process, termed GeoDiff [126], for predicting molecular conformations. Emiel Hoogeboom [127] and collaborators innovatively proposed an E(3) Equivariant Diffusion Model (EDM), a diffusion model for 3D molecule generation that is equivariant to Euclidean transformations. Their model employs an equivariant network that jointly operates on both continuous (atom coordinates) and categorical features (atom types) to learn the denoising of a diffusion process. Remarkably, their proposed approach significantly outperforms previous methods for 3D molecule generation, both in terms of the quality of the generated samples and the efficiency during training.

扩散模型近年来作为一类引人入胜的生成模型出现。其基本前提涉及一个噪声注入和顺序时间步长的迭代过程，将数据分布逐渐扩散到一个已知的简单分布，如高斯分布。在生成新样本时，模型沿着扩散路径从简单分布反向引导数据回到原始数据分布。创新地，Xu及其同事引入了一种基于扩散过程的新型生成模型，称为GeoDiff [126]，用于预测分子构象。Emiel Hoogeboom [127] 及其合作者创新地提出了一种E(3) 等变扩散模型(EDM)，一种用于3D分子生成的扩散模型，对欧几里得变换具有等变性。他们的模型采用一个等变网络，该网络对连续(原子坐标)和分类特征(原子类型)联合操作以学习扩散过程的去噪。值得注意的是，他们提出的方法在生成样本的质量和训练效率方面均显著优于先前的3D分子生成方法。

3.3. Application of optimization algorithms

<### 3.3. 优化算法的应用

Bayesian Optimization (BO) [128] serves as a strategic approach for global optimization, each iteration employs a surrogate model, typically a Gaussian process, to approximate the objective function. The strength of BO lies in its ability to conduct effective optimization with fewer samples, making it suitable for situations where sample costs are high or samples are difficult to obtain. However, its performance decreases with high-dimensional problems, and computational expenses increase with the number of iterations. Therefore, for problems where sample costs are significant and precise searching is required, such as in the design of catalysts, BO is highly favored. For instance, in a trailblazing collaborative effort, Cooper [129] of the University of Liverpool and Labman, an automation company, have co-developed an automated system for a mobile robot that targets catalyst screening. The robot emulates human chemists, handling tasks like solid catalyst weighing, liquid distribution, and the activation of experimental apparatus. It also manipulates analytical instruments to inspect experimental outcomes. Over the course of a relentless week, barring recharging sessions, this robot concluded almost 700 experiments. Beyond the initial experimental conditions, the robot, guided by BO, continues to adjust and optimize experimental parameters based on the results of the previous experiments. This demonstrates the great potential of BO in the direction of material screening. In addition to catalyst screening, several researchers have also employed Bayesian Optimization for the process optimization of perovskite solar cells [130,131].

贝叶斯优化(BO)[128] 是一种全局优化的策略方法，每次迭代使用一个代理模型，通常是高斯过程，来近似目标函数。BO的优势在于能够用较少的样本进行有效的优化，使其适用于样本成本高或难以获取样本的情况。然而，其在高维问题上性能会下降，且计算成本会随着迭代次数增加。因此，对于样本成本高昂且需要精确搜索的问题，如催化剂设计，BO备受青睐。例如，在一项开创性的合作中，利物浦大学的库珀 [129] 和自动化公司Labman共同开发了一种用于移动机器人的自动化系统，目标是催化剂筛选。该机器人模拟人类化学家，处理诸如固体催化剂称重、液体分配和实验仪器激活等任务。它还操作分析仪器检查实验结果。在持续一周的时间里，不包括充电时段，这个机器人完成了近700次实验。除了初始实验条件外，该机器人在BO的引导下，根据先前实验的结果继续调整和优化实验参数。这证明了BO在材料筛选方面的巨大潜力。除了催化剂筛选，几位研究人员还将贝叶斯优化用于钙钛矿太阳能电池的工艺优化 [13０,131]。

Genetic Algorithms(GA) [132], global optimization techniques mirroring biological evolutionary theory, are based on selection, crossover, and mutation. These algorithms simulate natural selection and genetic inheritance mechanisms for problem-solving. GA is that they are less likely to get trapped in local optima, making them particularly suitable for complex nonlinear problems. However, they may require longer computational times and are sensitive to the settings of algorithm parameters. Within the realms of chemistry and materials science, genetic algorithms are broadly harnessed for structure prediction and optimization. Pioneering work by Deaven [133] et al. in 1995 saw the first application of genetic algorithms for structural optimization of atomic clusters. Oganov [134] and colleagues proposed a method, USPEX, for crystal structure prediction based on genetic algorithms, which has become a widely utilized tool for structure prediction in solid-state physics. Furthermore, Kim [135] et al. utilized a combination of genetic algorithms and ML for polymer design, evolving polymers through crossover, mutation, and selection over 100 generations to produce 132 new, chemically unique polymers.

遗传算法(GA)[132] 是借鉴生物进化理论的全局优化技术，基于选择、交叉和变异。这些算法模拟自然选择和遗传遗传机制来解决问题。GA的优点是不太容易陷入局部最优，使其特别适合复杂的非线性问题。然而，它们可能需要更长的计算时间，并且对算法参数的设置敏感。在化学和材料科学领域，遗传算法广泛用于结构预测和优化。1995年，迪文 [133] 等人的开创性工作首次将遗传算法用于原子团簇的结构优化。奥加诺夫 [134] 及其同事提出了一种基于遗传算法的晶体结构预测方法USPEX，该方法已成为固态物理学中广泛使用的结构预测工具。此外，金 [135] 等人将遗传算法和机器学习相结合用于聚合物设计，通过交叉、变异和选择在100代中进化聚合物，以生产132种新的、化学性质独特的聚合物。

Particle Swarm Optimization (PSO) [31] is a heuristic global optimization method, inspired by the collective behavior observed in nature, such as flocks of birds or schools of fish. The fundamental principle of PSO is as follows: a “swarm” of particles fly through the search space, with each particle retaining the optimal position information it has found during the search process (that is, its best solution and the global best solution). Each particle updates its velocity and position during each iteration to search for the global optimum. This process allows for global search of the solution space while maintaining excellent local search capabilities. When the problem is relatively simple, not particularly high-dimensional, and a rapid solution is required, PSO can be a good choice. Wang [34] et al. used the PSO algorithm in predicting the structure of layered materials. Their successful predictions included the structures of known 2D materials, comprising single-layer and multi-layer graphene, 2D boron nitride (BN) compounds. This demonstration underscores the potential of the PSO algorithm in the field of structure prediction.

粒子群优化(PSO)[31] 是一种启发式全局优化方法，灵感来自于自然界中观察到的集体行为，如鸟群或鱼群。PSO的基本原理如下:一群粒子在搜索空间中飞行，每个粒子保留其在搜索过程中找到的最优位置信息(即其最佳解决方案和全局最佳解决方案)。每个粒子在每次迭代中更新其速度和位置以搜索全局最优。这个过程允许在保持出色的局部搜索能力的同时对解空间进行全局搜索。当问题相对简单、维度不是特别高且需要快速求解时，PSO可能是一个不错的选择。王 [34] 等人使用PSO算法预测层状材料的结构。他们成功的预测包括已知二维材料的结构，包括单层和多层石墨烯、二维氮化硼(BN)化合物。这一证明强调了PSO算法在结构预测领域的潜力。

The utilization of prior knowledge in BO allows it to perform well when samples are limited, the randomness and diversity of GA ensure that they do not easily get trapped in local optima when exploring spaces, and the fewer parameters and ease of implementation of PSO enable it to quickly yield satisfactory results for some relatively simple problems. Therefore, selecting the most suitable algorithm for a specific issue is of great importance.

在BO中利用先验知识使其在样本有限时表现良好，GA的随机性和多样性确保它们在探索空间时不容易陷入局部最优，而PSO较少的参数和易于实现使其能够为一些相对简单的问题快速产生满意的结果。因此，为特定问题选择最合适的算法非常重要。

4. Conclusion and outlook

4. 结论与展望

This review embarks by explicating the fundamentals of intelligent algorithms, including ML, DL, and optimization algorithms. It then delineates the modalities of data acquisition (standard databases, literature extraction methodologies), and common techniques of feature engineering, before launching into a comprehensive exposition of various intelligent algorithm applications within the realms of chemistry and materials science. Despite significant advancements in the deployment of intelligent algorithms, we believe that there are primarily three major issues that urgently need to be addressed in the future. First, intelligent algorithms are data-driven models; a vast amount of high-quality data forms the cornerstone of intelligent algorithms. However, in the fields of chemistry and materials, there still exist issues such as high data costs, difficulties in database sharing, and poor data quality. Second, current prevalent intelligent algorithms lack physical concepts; they merely learn patterns or rules from data, which inevitably can lead to results that violate physical laws, and also affect convergence and stability. Therefore, how to embed domain knowledge of chemistry and materials into intelligent algorithms to construct a model that is physically reasonable, mathematically precise, and computationally efficient presents a significant challenge for the future. Third, the ultimate goal for chemists is to learn new knowledge and rules from intelligent algorithms, thereby promoting new scientific discoveries. However, this aspect has not been well-developed at present, and research on model interpretability is still lacking.

本综述首先阐述了智能算法的基础，包括机器学习、深度学习和优化算法。接着描述了数据采集的方式(标准数据库、文献提取方法)以及特征工程的常用技术，随后全面阐述了各种智能算法在化学和材料科学领域的应用。尽管智能算法的部署取得了显著进展，但我们认为未来主要有三个重大问题亟待解决。首先，智能算法是数据驱动的模型；大量高质量的数据构成了智能算法的基石。然而，在化学和材料领域，仍存在数据成本高、数据库共享困难以及数据质量差等问题。其次，当前流行的智能算法缺乏物理概念；它们只是从数据中学习模式或规则，这不可避免地可能导致违反物理定律的结果，也会影响收敛性和稳定性。因此，如何将化学和材料的领域知识嵌入智能算法，构建一个物理上合理、数学上精确且计算高效的模型，是未来面临的重大挑战。第三，化学家的最终目标是从智能算法中学习新知识和规则，从而推动新的科学发现。然而，目前这方面尚未得到充分发展，对模型可解释性的研究仍然不足。

Challenges are opportunities, and the path to innovative development in the future lies in how to solve these three issues. First and foremost is the data aspect. The future should prioritize the development of high-throughput technologies and laboratory automation techniques to rapidly acquire a large volume of standardized data; of course, this is mainly a hardware issue. For instance, Jiang [136] and colleagues have pioneered a comprehensive ’end-to-end robotic chemist’ platform that encompasses literature review, autonomous experimental design, and materials development. This platform is adept at identifying the optimum solution from hundreds of millions of possible combinations, thus accelerating the pace of material research and development. Secondly, the future should focus on developing knowledge-embedding techniques to construct intelligent algorithms infused with chemical and materials knowledge, mainly comprising strategies of both soft and hard constraints. Soft constraints typically do not force the model to strictly adhere to certain rules, but rather give the model some degree of freedom, taking domain knowledge into account during the optimization process. The main strategies include theoretically-guided normalization, the design of new descriptors, network topology structure design, convolution kernels and filters design, and loss function design. On the other hand, hard constraints are rules that the model must strictly follow, with main strategies including data decomposition, finite element methods, and domain space constraints. For instance, our research group has constructed a soft-constrained PINN [90] that incorporates the monotonic rule of chromatography into the loss function, while also constructing a hard constraint that restricts the output to $\left( {0,1}\right)$ through the sigmoid activation function, aligning with the real-world behavior of ${\mathrm{R}}_{\mathrm{f}}$ values. Thirdly, the future should focus on the development of knowledge discovery technologies based on intelligent algorithms, to extract undiscovered knowledge from data and push the boundaries of human cognition. The main strategies include sparse regression, genetic algorithms, symbolic mathematics, etc. For example, Jiang and others used the SISSO method to obtain a concise mathematical formula for the spectrum-property relationship [65], which is interpretable, thus achieving the transfer application of knowledge across different systems. Our vision for the future is this: Rapid data acquisition through laboratory automation, researchers using these high-quality data to construct intelligent algorithms with knowledge embedding, and through knowledge discovery technologies, uncover new knowledge and rules, which in turn further guide algorithm design, forming a closed loop. The application of intelligent algorithms within the domains of chemistry and material science holds a plethora of possibilities yet to be fully explored. It is our aspiration that the insights and perspectives shared within this paper will serve as a source of inspiration for researchers, collectively steering the development and growth of this field.

挑战即机遇，未来创新发展的路径在于如何解决这三个问题。首先是数据方面。未来应优先发展高通量技术和实验室自动化技术，以快速获取大量标准化数据；当然，这主要是硬件问题。例如，Jiang[136]及其同事率先开发了一个全面的“端到端机器人化学家”平台，该平台涵盖文献综述、自主实验设计和材料开发。这个平台擅长从数亿种可能的组合中识别出最优解决方案，从而加快材料研发的步伐。其次，未来应专注于开发知识嵌入技术，构建融入化学和材料知识的智能算法，主要包括软约束和硬约束策略。软约束通常不会强制模型严格遵守某些规则，而是给模型一定的自由度，在优化过程中考虑领域知识。主要策略包括理论指导的归一化、新描述符的设计、网络拓扑结构设计、卷积核和滤波器设计以及损失函数设计。另一方面，硬约束是模型必须严格遵循的规则，主要策略包括数据分解、有限元方法和域空间约束。例如，我们的研究小组构建了一个软约束的PINN[90]将色谱的单调规则纳入损失函数，同时还构建了一个硬约束，通过sigmoid激活函数将输出限制为$\left( {0,1}\right)$，与${\mathrm{R}}_{\mathrm{f}}$值的实际行为一致。第三，未来应专注于基于智能算法的知识发现技术的开发，从数据中提取未发现的知识，拓展人类认知的边界。主要策略包括稀疏回归、遗传算法、符号数学等。例如，Jiang等人使用SISSO方法获得了光谱 - 性质关系的简洁数学公式[65]，该公式具有可解释性，从而实现了知识在不同系统之间的转移应用。我们对未来的愿景是这样的:通过实验室自动化快速采集数据，研究人员利用这些高质量数据构建具有知识嵌入的智能算法，并通过知识发现技术，发现新的知识和规则，进而进一步指导算法设计，形成一个闭环。智能算法在化学和材料科学领域的应用有着众多尚未充分探索的可能性。我们希望本文所分享的见解和观点能为研究人员提供灵感源泉，共同推动该领域的发展与成长。

Declaration of Competing Interest

利益冲突声明

I, Fanyang Mo, hereby declare that I have no known competing financial interests or personal relationships that could have influenced the work reported in this manuscript. I have received no financial support from any organization for the submitted work, have no financial relationships with any organizations that might have an interest in the submitted work, and have no other relationships or activities that could appear to have influenced the submitted work.

我，莫凡洋，特此声明，我不知道有任何可能影响本稿件所报告工作的竞争财务利益或个人关系。我没有从任何组织获得提交工作的资金支持，与任何可能对提交工作感兴趣的组织没有财务关系，并且没有其他可能影响提交工作的关系或活动。

References

参考文献

[1] A. Clauset, D.B. Larremore, R.J.S. Sinatra, Data-driven predictions in the science of science, Science 355 (2017) 477-480.

[2] K. Jorner, A. Tomberg, C. Bauer, C. Sköld, P.-O. Norrby, Organic reactivity from mechanism to machine learning, Nat. Rev. Chem. 5 (2021) 240-255.

[3] Z. Tu, T. Stuyver, C.W. Coley, Predictive chemistry: machine learning for reaction deployment, reaction development, and reaction discovery, Chem. Sci. 14 (2023) 226-244.

[4] L.C. Yang, L.J. Zhu, S.Q. Zhang, X. Hong, Machine learning prediction of structure-performance relationship in organic synthesis, Chin. J. Chem. 40 (2022) 2106-2117.

[5] T. Poggio, et al., Why and when can deep-but not shallow-networks avoid the

[6] C. McInnes, Virtual screening strategies in drug discovery, Curr. Opin. Chem. Biol. 11 (2007) 494-502.

[7] J. Greeley, T.F. Jaramillo, J. Bonde, I. Chorkendorff, J.K. Nørskov, Computational high-throughput screening of electrocatalytic materials for hydrogen evolution, Nat. Mater. 5 (2006) 909-913.

[8] J. Li, et al., A comprehensive kinetic mechanism for $\mathrm{{CO}},{\mathrm{{CH}}}_{2}\mathrm{O}$ , and ${\mathrm{{CH}}}_{3}\mathrm{{OH}}$ combustion, Int. J. Chem. Kinet. 39 (2007) 109-136.

[9] C. Hansch, Quantitative approach to biochemical structure-activity relationships, Acc. Chem. Res. 2 (1969) 232-239.

[10] D. Maulud, A.M. Abdulazeez, A review on linear regression comprehensive in machine learning, J. Appl. Sci. Technol. Trends 1 (2020) 140-147.

[11] C. Kingsford, S.L. Salzberg, What are decision trees? Nat. Biotechnol. 26 (2008) 1011-1013.

[12] M.A. Hearst, S.T. Dumais, E. Osuna, J. Platt, B. Scholkopf, Support vector machines, IEEE Intell. Syst. their Appl. 13 (1998) 18-28.

[13] R. Bro, A.K. Smilde, Principal component analysis, Anal. Methods 6 (2014) 2812-2831.

[14] L.P. Kaelbling, M.L. Littman, A.W. Moore, Reinforcement learning: a survey, J. Artif. Intell. Res. 4 (1996) 237-285.

[15] F. Pedregosa, et al., Scikit-learn: machine learning in Python, J. Mach. Learn. Res. 12 (2011) 2825-2830.

[16] A.K. Jain, J. Mao, K.M.J.C. Mohiuddin, Artificial neural networks: a tutorial, Computer 29 (1996) 31-44.

[17] Li, Z. et al. A survey of convolutional neural networks: analysis, applications, and prospects. IEEE transactions on neural networks and learning systems (2021).

[18] L.R. Medsker, L.J.D. Jain, Applications, Recurrent neural networks, Des. Appl. 5 (2001) 2.

[19] Paszke, A. et al. Automatic differentiation in pytorch. (2017).

[20] Abadi, M. et al. in 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265-283.

[21] Y. Ma, D. Yu, T. Wu, H. Wang, PaddlePaddle: an open-source deep learning platform from industrial practice, Front. Data Domputing 1 (2019) 105-115.

[22] Huawei Technologies Co., Huawei MindSpore AI Development Framework, Artif. Intell. Technol. (2022) 137-162.

[23] A. Gulli, S. Pal, Deep learning with Keras, Packt Publishing Ltd, 2017.

[24] D.T. Ahneman, J.G. Estrada, S. Lin, S.D. Dreher, A.G.J.S. Doyle, Predicting reaction performance in C-N cross-coupling using machine learning, Science 360 (2018) 186-190.

[25] B. Mikulak-Klucznik, et al., Computational planning of the synthesis of complex natural products, Nature 588 (2020) 83-88.

[26] K. Muraoka, Y. Sada, D. Miyazaki, W. Chaikittisilp, T.J.N. c Okubo, Linking synthesis and structure descriptors from a large collection of synthetic records of zeolite materials, Nat. Commun. 10 (2019) 4459.

[27] J. Jumper, et al., Highly accurate protein structure prediction with AlphaFold, Nature 596 (2021) 583-589.

[28] Y. Saito, et al., Deep-learning-based quality filtering of mechanically exfoliated 2D crystals, npj Comput. Mater. 5 (2019), 124.

[29] D. Bertsimas, J.J.S. s Tsitsiklis, Simulated annealing, Stat. Sci. 8 (1993) 10-15.

[30] J.H. Holland, Genetic algorithms, Sci. Am. 267 (1992) 66-73.

[31] Kennedy, J. & Eberhart, R. in Proceedings of ICNN'95-international conference on neural networks. 1942-1948 (IEEE).

[32] R. Harada, T. Nakamura, Y. Shigeta, A fast convergent simulated annealing algorithm for protein-folding: simulated annealing outlier FLOODing (SA-OFLOOD) method, Bull. Chem. Soc. Jpn. 89 (2016) 1361-1367.

[33] H. Link, et al., Multi-objective steady state optimization of biochemical reaction networks using a constrained genetic algorithm, Comput. Chem. Eng. 32 (2008) 1707-1713.

[34] Y. Wang, et al., An effective structure prediction method for layered materials based on 2D particle swarm optimization algorithm, J. Chem. Phys. 137 (2012).

[35] T. Lewis-Atwell, P.A. Townsend, M.N. Grayson, Machine learning activation energies of chemical reactions, Wiley Interdiscip. Rev.: Comput. Mol. Sci. 12 (2022), e1593.

[36] Q. Yang, et al., Holistic prediction of the pKa in diverse solvents based on a machine-learning approach, Angew. Chem. Int. Ed. 132 (2020) 19444-19453.

[37] X. Fang, et al., Geometry-enhanced molecular representation learning for property prediction, Nat. Mach. Intell. 4 (2022) 127-134.

[38] S. Kim, et al., PubChem substance and compound databases, Nucleic Acids Res. 44 (2016) D1202-D1213.

[39] H.E. Pence, A. Williams, ChemSpider: An online chemical information resource, J. Chem. Educ. 87 (2010) 1123-1124.

[40] A. Gaulton, et al., ChEMBL: a large-scale bioactivity database for drug discovery, Nucleic Acids Res. 40 (2012) D1100-D1107.

[41] J.J. Irwin, B.K. Shoichet, ZINC- a free database of commercially available compounds for virtual screening, J. Chem. Inf. Model. 45 (2005) 177-182.

[42] C.R. Groom, I.J. Bruno, M.P. Lightfoot, S.C. Ward, The Cambridge structural database, Acta Cryst. 72 (2016) 171-179.

[43] Internet Bond-energy Databank (pKa and BDE)-iBonD Home Page, http://ibond.nankai.edu.cn.

[44] S.M. Kearnes, et al., The open reaction database, J. Am. Chem. Soc. 143 (2021) 18820-18826.

[45] A. Jain, et al., Commentary: the materials project: a materials genome approach to accelerating materials innovation, APL Mater. 1 (2013).

[46] S.P. Ong, et al., Python materials genomics (pymatgen): a robust, open-source python library for materials analysis, Comput. Mater. Sci. 68 (2013) 314-319.

[47] A.H. Larsen, et al., The atomic simulation environment—a python library for working with atoms, J. Phys.: Condens. Matter 29 (2017), 273002.

[48] D.D. Landis, et al., The computational materials repository, Comput. Sci. Eng. 14 (2012) 51-57.

[49] J.E. Saal, S. Kirklin, M. Aykol, B. Meredig, C.J.J. Wolverton, Materials design and discovery with high-throughput density functional theory: the open quantum materials database (OQMD), Jom 65 (2013) 1501-1509.

[50] K. Chowdhary, K.J.F. o a i Chowdhary, Natural language processing, Fundam. Artif. Intell. (2020) 603-649.

[51] L. Richardson, Beautiful soup documentation. (2007).

[52] E. Uzun, T. Yerlikaya, O. Kirat, Bulgaria. Comparison of Python libraries used for Web data extraction, J. Tech. Univ. -Sofia Plovdiv. Branch, Bulg. 24 (2018) 87-92.

[53] Nancy, I. & Véronis, J. in 6th Annual Conference of the Centre for the New Oxford English Dictionary. 52-64 (Citeseer).

[54] P.J.P. Willett, The Porter stemming algorithm: then and now, Program 40 (2006) 219-223.

[55] K.W. Church, Word2Vec, Nat. Lang. Eng. 23 (2017) 155-162.

[56] Pennington, J., Socher, R. & Manning, C.D. in Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532-1543.

[57] Nivre, J. in Proceedings of the eighth international conference on parsing technologies. 149-160.

[58] A. Vaswani, et al., Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017).

[59] T.N. Kipf, M. Welling, Semi-supervised classification with graph convolutional networks, arXiv preprint arXiv:1609.02907 (2016).

[60] T.E. Oliphant, Guide to numpy, Vol. 1, Trelgol Publishing USA, 2006.

[61] Toropov, A.A., Toropova, A.P., Mukhamedzhanoval, D.V. & Gutman, I. Simplified molecular input line entry system (SMILES) as an alternative for constructing quantitative structure-property relationships (QSPR). (2005).

[62] A. Cereto-Massagué, et al., Molecular fingerprint similarity search in virtual screening, Methods 71 (2015) 58-63.

[63] Z. Liu, T. Zubatiuk, A. Roitberg, O. Isayev, Auto3d: automatic generation of the low-energy 3d structures with ANI neural network potentials, J. Chem. Inf. Model. 62 (2022) 5373-5382.

[64] X. Fang, et al., Geometry-enhanced molecular representation learning for property prediction, Nat. Mach. Intell. 4 (2022) 127-134.

[65] X. Wang, et al., Quantitatively determining surface-adsorbate properties from vibrational spectroscopy with interpretable machine learning, J. Am. Chem. Soc. 144 (2022) 16069-16076.

[66] J.M. Granda, L. Donina, V. Dragone, D.-L. Long, L. Cronin, Controlling an organic synthesis robot with machine learning to search for new reactivity, Nature 559 (2018) 377-381.

[67] P. Schwaller, et al., Mapping the space of chemical reactions using attention-based neural networks, Nat. Mach. Intell. 3 (2021) 144-152.

[68] A.F. Zahrt, et al., Prediction of higher-selectivity catalysts by computer-driven workflow and machine learning, Science 363 (2019) 5631.

[69] P.M. Granitto, C. Furlanello, F. Biasioli, F. Gasperi, Recursive feature elimination with random forest for PTR-MS analysis of agroindustrial products, Chemom. Intell. Lab. Syst. 83 (2006) 83-90.

[70] Y.-Y. Song, L.J.S. Ying, Decision tree methods: applications for classification and prediction, Shanghai Arch. Psychiatry 27 (2015) 130.

[71] H. Zou, The adaptive lasso and its oracle properties, J. Am. Stat. Assoc. 101 (2006) 1418-1429.

[72] A.E. Hoerl, R.W. Kennard, Ridge regression: applications to nonorthogonal problems, Technometrics 12 (1970) 69-82.

[73] H. Zou, T. Hastie, Regularization and variable selection via the elastic net, J. R. Stat. Soc. Ser. B: Stat. Methodol. 67 (2005) 301-320.

[74] D.R. Cutler, et al., Random forests for classification in ecology, Ecology 88 (2007) 2783-2792.

[75] D.T. Ahneman, J.G. Estrada, S. Lin, S.D. Dreher, A.G. Doyle, Predicting reaction performance in C-N cross-coupling using machine learning, Science 360 (2018) 186-190.

[76] S.K. Chandy, K. Raghavachari, MIM-ML: a novel quantum chemical fragment-based random forest model for accurate prediction of NMR chemical shifts of nucleic acids, J. Chem. Theory Comput. 19 (2023) 6632-6642.

[77] J.G. Meyer, S. Liu, I.J. Miller, J.J. Coon, A. Gitter, Learning drug functions from chemical structures with convolutional neural networks and random forests, J. Chem. Inf. Model. 59 (2019) 4438-4449.

[78] C. Mouchel-Vallon, A. Hodzic, Toward emulating an explicit organic chemistry mechanism with random forest models, J. Geophys. Res.: Atmos. 128 (2023), e2022JD038227.

[79] R. Batra, C. Chen, T.G. Evans, K.S. Walton, R. Ramprasad, Prediction of water stability of metal-organic frameworks using machine learning, Nat. Mach. Intell. 2 (2020) 704-710.

[80] W. Sun, et al., Machine learning-assisted molecular design and efficiency prediction for high-performance organic photovoltaic materials, Sci. Adv. 5 (2019) 4275.

[81] Hartigan, J.A. & Wong, M.A. Algorithm AS 136: A k-means clustering algorithm. Journal of the royal statistical society. series c 28, 100-108 (1979).

[82] U. Von Luxburg, A tutorial on spectral clustering, Stat. Comput. 17 (2007) 395-416.

[83] Wang, W.-T., Wu, Y.-L., Tang, C.-Y. & Hor, M.-K.. in 2015 International Conference on Machine Learning and Cybernetics (ICMLC). 445-451 (IEEE).

[84] Y. Zhang, et al., Unsupervised discovery of solid-state lithium ion conductors, Nat. Commun. 10 (2019), 5260.

[85] A.K. Jain, J. Mao, K.M. Mohiuddin, Artificial neural networks: A tutorial, Computer 29 (1996) 31-44.

[86] He, K., Zhang, X., Ren, S. & Sun, J. in Proceedings of the IEEE international conference on computer vision. 1026-1034.

[87] Mercioni, M.A. & Holban, S. in 2020 International Symposium on Electronics and Telecommunications (ISETC). 1-4 (IEEE).

[88] Kingma, D.P. & Ba, J. Adam: A method for stochastic optimization. arXiv preprint arXiv:. (2014).

[89] Loshchilov, I. & Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:.05101 (2017).

[90] H. Xu, et al., High-throughput discovery of chemical structure-polarity relationships combining automation and machine-learning techniques, Chem 8 (2022) 3202-3214.

[91] H. Xu, D. Zhang, F. Mo, High-throughput automated platform for thin layer chromatography analysis, STAR Protoc. 3 (2022), 101893.

[92] Chen, T. & Guestrin, C. in Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. 785-794.

[93] G. Ke, et al., Lightgbm: A highly efficient gradient boosting decision tree, Adv. Neural Inf. Process. Syst. (30) (2017).

[94] M. Raissi, P. Perdikaris, G.E. Karniadakis, Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations, J. Comput. Phys. 378 (2019) 686-707.

[95] V. Nourani, M.S. Fard, Sensitivity analysis of the artificial neural network outputs in simulation of the evaporation process at different climatologic regimes, Adv. Eng. Softw. 47 (2012) 127-146.

[96] S. Mishra, B.L. Sturm, S. Dixon, Local interpretable model-agnostic explanations for music content analysis, ISMIR (2017) 537-543.

[97] S. Mangalathu, S.-H. Hwang, J.-S. Jeon, Failure mode and effects analysis of RC members based on machine-learning-based SHapley Additive exPlanations (SHAP) approach, Eng. Struct. 219 (2020), 110927.

[98] He, K., Zhang, X., Ren, S. & Sun, J. in Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778.

[99] Yu, F. & Koltun, V. Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:.07122 (2015).

[100] Zhang, T., Qi, G.-J., Xiao, B. & Wang, J. in Proceedings of the IEEE international conference on computer vision. 4373-4382.

[101] Lobato, I., Friedrich, T. & Van Aert, S. Deep convolutional neural networks to restore single-shot electron microscopy images. arXiv preprint arXiv:.17025 (2023).

[102] M. Ziatdinov, et al., Deep learning of atomically resolved scanning transmission electron microscopy images: chemical identification and tracking local transformations, ACS nano 11 (2017) 12742-12752.

[103] A. Graves, A. Graves, Long short-term memory, Supervised Seq. Label. Recurr. Neural Netw. (2012) 37-45.

[104] Chung, J., Gulcehre, C., Cho, K. & Bengio, Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:. (2014).

[105] J. Burés, I. Larrosa, Organic reaction mechanism classification using machine learning, Nature 613 (2023) 689-695.

[106] A. Vaswani, et al., Attention is all you need, Adv. Neural Inf. Process. Syst. (30) (2017).

[107] Chithrananda, S., Grand, G. & Ramsundar, B.J. a p a ChemBERTa: large-scale self-supervised pretraining for molecular property prediction. (2020).

[108] J. Ross, et al., Large-scale chemical language representations capture mo structure and properties, Nat. Mach. Intell. 4 (2022) 1256-1264.

[109] Z. Xiong, et al., Pushing the boundaries of molecular representation for drug discovery with the graph attention mechanism, J. Med. Chem. 63 (2019) 8749-8760.

[110] Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O. & Dahl, G.E. in International conference on machine learning. 1263-1272 (PMLR).

[111] P. Schwaller, et al., Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction, ACS Cent. Sci. 5 (2019) 1572-1583.

[112] Y. Kang, H. Park, B. Smit, J. Kim, A multi-modal pre-training transformer for universal transfer learning in metal-organic frameworks, Nat. Mach. Intell. 5 (2023) 309-318.

[113] Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? arXiv preprint arXiv:.00826 (2018).

[114] K.T. Schütt, H.E. Sauceda, P.-J. Kindermans, A. Tkatchenko, K.-R. Müller, Schnet-a deep learning architecture for molecules and materials, J. Chem. Phys. (148) (2018).

[115] S. Kearnes, K. McCloskey, M. Berndl, V. Pande, P. Riley, Molecular graph convolutions: moving beyond fingerprints, J. Comput. -Aided Mol. Des. 30 (2016) 595-608.

[116] T. Nguyen, et al., GraphDTA: Predicting drug-target binding affinity with graph neural networks, Bioinformatics 37 (2021) 1140-1147.

[117] T. Xie, J.C. Grossman, Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties, Phys. Rev. Lett. 120 (2018), 145301.

[118] Sanyal, S. et al. MT-CGCNN: Integrating crystal graph convolutional neural network with multitask learning for material property prediction. arXiv preprint arXiv:.05660 (2018).

[119] I. Goodfellow, et al., Generative adversarial networks, Commun. ACM 63 (2020) 139-144.

[120] Doersch, C. Tutorial on variational autoencoders. arXiv preprint arXiv:.05908 (2016).

[121] Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N. & Ganguli, S. in International conference on machine learning. 2256-2265 (PMLR).

[122] De Cao, N. & Kipf, T. MolGAN: An implicit generative model for small molecular graphs. arXiv preprint arXiv:.11973 (2018).

[123] R. Gómez-Bombarelli, et al., Automatic chemical design using a data-driven continuous representation of molecules, ACS Cent. Sci. 4 (2018) 268-276.

[124] Jin, W., Barzilay, R. & Jaakkola, T. in International conference on machine learning. 2323-2332 (PMLR).

[125] Z. Yao, et al., Inverse design of nanoporous crystalline reticular materials with deep generative models, Nat. Mach. Intell. 3 (2021) 76-86.

[126] Xu, M. et al. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:.02923 (2022).

[127] Hoogeboom, E., Satorras, V.G., Vignac, C. & Welling, M. in International conference on machine learning. 8867-8887 (PMLR).

[128] J. Snoek, H. Larochelle, R.P. Adams, Practical bayesian optimization of machine learning algorithms, Adv. Neural Inf. Process. Syst. (25 ) (2012).

[129] B. Burger, et al., A mobile robotic chemist, Nature 583 (2020) 237-241.

[130] Z. Liu, et al., Machine learning with knowledge constraints for process optimization of open-air perovskite solar cell manufacturing, Joule 6 (2022) 834-849.

[131] J. Zhang, et al., Optimizing perovskite thin-film parameter spaces with machine learning-guided robotic platform for high-performance perovskite solar cells, Adv. Energy Mater. (2023), 2302594.

[132] Sastry, K., Goldberg, D. & Kendall, G. Genetic algorithms. Search methodologies: Introductory tutorials in optimization decision support techniques, (2005) 97-125 .

[133] D.M. Deaven, K.-M. Ho, Molecular geometry optimization with a genetic algorithm, Phys. Rev. Lett. 75 (1995) 288.

[134] C.W. Glass, A.R. Oganov, N. Hansen, USPEX-evolutionary crystal structure prediction, Comput. Phys. Commun. 175 (2006) 713-720.

[135] C. Kim, R. Batra, L. Chen, H. Tran, R. Ramprasad, Polymer design using genetic algorithm and machine learning, Comput. Mater. Sci. 186 (2021), 110067.

[136] Q. Zhu, et al., An all-round AI-Chemist with a scientific mind, Natl. Sci. Rev. 9 (2022), nwac190.