 Full text:
 PDF (eReader) / PDF (Download) (2873 kB)
Interfaces have atomic structures that are significantly different from those in the bulk, and play crucial roles in material properties. The central structures at the interfaces that provide properties have been extensively investigated. However, determination of even one interface structure requires searching for the stable configuration among many thousands of candidates. Here, a powerful combination of machine learning techniques based on kriging and transfer learning (TL) is proposed as a method for unveiling the interface structures. Using the kriging+TL method, thirtythree grain boundaries were systematically determined from 1,650,660 candidates in only 462 calculations, representing an increase in efficiency over conventional allcandidate calculation methods, by a factor of approximately 3,600.
Interfaces play crucial roles in materials properties. The abundant interfaces in polycrystalline materials — namely, the grain boundaries (GBs) — determine electronic and ionic conductivities and mechanical strengths.^{1}^{–}^{4}^{)} Furthermore, interfaces in thin films often endow new functions such as the emergence of twodimensional electron gases or superconductivity.^{5}^{–}^{9}^{)} In addition to such positive effects, negative effects such as embrittlement are also known to originate from interfaces.^{10}^{,}^{11}^{)}
Large interface effects are caused by atomic structures that differ significantly from those in the bulk. Determining the central structures at the interface from which material properties originate and understanding the relationships between structures and these properties are among the most important tasks in materials research. However, the determination of interface structures remains challenging because of the high amount of geometrical freedom at the interface.^{12}^{)} For one type of coincidence site lattice GB in simple metals — a very simplified Σ GB — the number of candidate configurations approaches
Figure 1. (Color online) Schematics of (a) kriging and (b) combined method with transfer learning (kriging+TL). (a) In kriging, each GB has an individual threedimensional search space. (b) By contrast, a common 74dimensional search space is used in the kriging+TL method.
However, the current kriging method requires separate constructions of predictors for different interfaces, as shown in Fig. 1(a), in which the predictors indicated in red, blue, and yellow must be constructed for GB1, GB2, and GB3, respectively. Thus, kriging by itself is still inefficient in performing systematic determinations of many interface structures. If the speed of kriging could be improved and a feasible predictor constructed, the process of structure determination of an interface could dramatically accelerate. Such enhancement would facilitate systematic investigation of interfaces, and would pave the way for a deeper understanding of the mechanisms from which interface properties arise.
In this study, kriging was combined with another machine learning technique, transfer learning (TL), to accelerate the searching process. TL improves learning efficiency by solving a certain task using data and learning results from other related tasks.^{19}^{)} The concept of kriging combined with TL, kriging+TL, for the interface structure searching is schematically illustrated in Fig. 1(b). In the kriging+TL process, a common predictor (green) is used for all tasks, and the data space obtained by a given kriging is transferred to the next kriging. Thus, the data space is shared with all GBs and the predictor gradually become more “intelligent” during the GB calculation process. Thus, the speed, robustness, and feasibility of interface determination can be improved using the kriging+TL method. We demonstrated here that the kriging+TL process is a very powerful method for systematic determination of interface structures.
In this study, 33 Fe [110] symmetric tilt GBs whose structures have been already reported^{20}^{,}^{21}^{)} were systematically investigated. Static lattice calculations using the general utility lattice program (GULP) code^{22}^{)} were performed to optimize the structures and calculate the lattice energy of a supercell GB. Finnis–Sinclairtype interatomic potentials were employed.^{23}^{)}
To find the most stable structure, threedimensional translations with 0.1– Å steps were applied to one side of the grain. The schematics of the GB model, for instance
Figure 2. (Color online) Schematic of the GB model. Here, two crystals for the [110] symmetric tilt Fe
The GB energy,
Kriging is a Gaussian process of nonparametric regression analysis based on Bayesian statistical methods. In the kriging process carried out here, five initial configurations were randomly selected for the first single GB [GB1 in Fig. 1(b)] and then structure optimizations and energy calculations for the selected configurations were performed. Then the prediction model, namely predictor, is constructed based on the Gaussian process supplemented with the Bayesian optimization. The predictor is described by the Gaussian kernel, and the mean and standard deviations at each unobserved point in the search space were obtained from a probability distribution based on the Gaussian process, and the zscore,
As shown schematically in Fig. 1(b), the data space obtained by a given kriging is transferred to the next kriging in the kriging+TL method. Thus, the data space was shared with all GBs. Because the 74dimensional data space was common for other GBs, the data space became “smarter” by repeating the kriging operation. Thus, if the transfer learning was successful, the number of calculations for the respective GBs, namely l, m, and ntimes for GB1, GB2, and GB3, respectively, became smaller (
The misorientation angles, Σ values, GB planes, lattice constants, number of atoms in the supercell, and number of candidate configurations for the employed GBs are listed in the Supplemental materials.^{24}^{)} The total number of candidates for the 33 GBs was 1,650,660, meaning that it was necessary to perform 1,650,660 computations to determine the stable structures of all of the GBs.
Details on kriging for searching GB structures have been described previously^{13}^{)} from the results of studies in which kriging was performed in threedimensional space with x, y, and z translations. Although such “threedimensional kriging” is adequate when searching for a single GB, each GB's threedimensional space cannot be shared with other GBs [Fig. 1(a)], and therefore in this study the data space was expanded to a 75dimensional data space constructed using 74 dimensions of search space and one dimension for the GB energy; a schematic of this expansion is shown in Fig. 1(b) and descriptors for the 74 dimensions are listed in Table I^{,}. Original values of the descriptors listed in Table I^{,} were obtained from the interface models, and square, inverse, exponential, or exponential inverse of them (except for inverse of zero) were also generated and used as descriptors. Then, 74 descriptors in total were used. We used such relatively large number of descriptors because of following reasons; 1) a larger number of suitable descriptors would be helpful to describe the complex data space, and 2) the computational time for regression is not largely changed by the number of the descriptors. In threedimensional kriging, each data space is independent only for respective GBs [Fig. 1(a)]; by using 74dimensional kriging, all GBs can be considered to be in the same data space, making TL possible, as shown in Fig. 1(b).

The feasibility of 74dimensional (74D)kriging was confirmed by calculating the
Figure 3. (Color online) Atomic structures of (a)
The above 74Dkriging process was then combined with a TL process in which knowledge obtained in previous calculations is transferred to succeeding calculations [as shown schematically in Fig. 1(b)]. In the previous 3Dkriging case described above, the kriging process used individual 3D data spaces with essentially independent predictors for each GB [Fig. 1(a)]. In the kriging+TL process, the predictor obtained after the GB1 calculation was preserved and transferred to the next GB2 calculation, by updating the predictor. In this manner, the predictor continuously transferred to succeeding GBs analyses. When the transfer succeeded, the calculation efficiency should have been improved; in other words, the number of calculations needed to reach convergence, l, m, and n and
In the kriging+TL process, the kriging was performed using the order of the Σ value, namely, from
Figure 4. (Color online) (a) Convergence speed ratio attained by transfer learning (TL). The convergence speed ratio corresponds to the number of calculations using TL divided by the number without TL; thus, smaller values indicate faster searching. (b) GB energies as a function of tilt angle calculated by kriging+TL compared with previous study results.^{20}^{,}^{21}^{)} Individual points are the GB energies of the 33 GBs listed in Supplementary material.^{24}^{)}
The predictor was then continuously updated and applied to the other 31 GBs. As shown in Fig. 4(a), kriging+TL always had a faster (up to 70% acceleration) convergence rate than that of kriging without TL, suggesting that the predictor gradually became more “intelligent” during the GB calculation process and transferred its knowledge to successive krigings.
Figure 4(b) plots the calculated GB energies of the 33 GBs as a function of the misorientation angle. To confirm the validity of the above calculations, these are compared to GB energies reported in previous molecular dynamics simulations.^{20}^{,}^{21}^{)} Because of the differences in the empirical potential used, the absolute GB energies differed from the those in previous reports.^{19}^{)} However, the trend seen in the GB energy (convex shapes with some cusps) reproduces those seen in previous work. This clearly indicates that using TL provided acceleration while maintaining robustness. Even though there was a total of 1,650,660 candidate structures for the 33 GBs, kriging+TL required only 462 computations to determine the actual 33 GB structures. In other words, the proposed kriging+TL method was approximately 3,600 times more efficient than conventional allcandidate calculation method, which in turn is three times more efficient than kriging without TL.
In summary, kriging+TL, a powerful combination of machine learning techniques, was shown to accelerate interface structure searches. It was confirmed that in the kriging+TL process the prediction model, or predictor, became increasingly more intelligent by sequentially transferring knowledge between process iterations. The proposed kriging+TL method was approximately 3,600 times more efficient than the conventional allcandidate calculation methods. It was demonstrated through this study that transfer learning dramatically improved the kriging convergence speed.
In crystalline materials, there is a very wide variety of interface types with differing atomic structures that can govern many types of material properties. The systematic investigation of a variety of interfaces is therefore indispensable in gaining an understanding of interface properties. To perform such systematic investigations, the kriging+TL presented here could be quite powerful, and we believe that the proposed method will pave the way for the investigation and design of material interfaces.
Acknowledgements
This study was supported by the Japan Science and Technology Agency–Precursory Research for Embryonic Science and Technology (JSTPRESTO, JPMJPR16NB 16814592), Japan, GrantsinAid for Scientific Research from the Ministry of Education, Culture, Sports, Science and Technology, Japan (MEXT; Nos. 25106003 and 17H06094), and the special fund of Institute of Industrial Science, The University of Tokyo (Tenkai5504850104).
References
 1 K.S. Chang, Y.F. Lin, and K.L. Tung, J. Power Sources 196, 9322 (2011). 10.1016/j.jpowsour.2011.07.085 Crossref, Google Scholar
 2 D. Chen, X.F. Zhang, and R. O. Ritchie, J. Am. Ceram. Soc. 83, 2079 (2000). 10.1111/j.11512916.2000.tb01515.x Crossref, Google Scholar
 3 J. P. Buban, K. Matsunaga, J. Chen, N. Shibata, W. Y. Ching, T. Yamamoto, and Y. Ikuhara, Science 311, 212 (2006). 10.1126/science.1119839 Crossref, Google Scholar
 4 K. Matsunaga, H. Nishimura, H. Muto, T. Yamamoto, and Y. Ikuhara, Appl. Phys. Lett. 82, 1179 (2003). 10.1063/1.1555690 Crossref, Google Scholar
 5 S. Thiel, G. Hammerl, A. Schmehl, C. W. Schneider, and J. Mannhart, Science 313, 1942 (2006). 10.1126/science.1131091 Crossref, Google Scholar
 6 H. Ohta, S. Kim, Y. Mune, T. Mizoguchi, K. Nomura, S. Ohta, T. Nomura, Y. Nakanishi, Y. Ikuhara, M. Hirano, H. Hosono, and K. Koumoto, Nat. Mater. 6, 129 (2007). 10.1038/nmat1821 Crossref, Google Scholar
 7 T. Mizoguchi, H. Ohta, H.S. Lee, N. Takahashi, and Y. Ikuhara, Adv. Funct. Mater. 21, 2258 (2011). 10.1002/adfm.201100230 Crossref, Google Scholar
 8 A. Ohtomo and H. Y. Hwang, Nature 427, 423 (2004). 10.1038/nature02308 Crossref, Google Scholar
 9 J. Matsuno, N. Ogawa, K. Yasuda, F. Kagawa, W. Koshibae, N. Nagaosa, Y. Tokura, and M. Kawasaki, Sci. Adv. 2, e1600304 (2016). 10.1126/sciadv.1600304 Crossref, Google Scholar
 10 G. H. Li and L. D. Zhang, Scr. Metall. 32, 1335 (1995). 10.1016/0956716X(95)00167T Crossref, Google Scholar
 11 R. Schweinfest, A. T. Paxton, and M. W. Finnis, Nature 432, 1008 (2004). 10.1038/nature03198 Crossref, Google Scholar
 12 A. Sutton and R. Balluffi, Interfaces in Crystalline Materials Clarendon (Oxford University Press, Oxford, U.K., 1995). Google Scholar
 13 S. Kiyohara, H. Oda, K. Tsuda, and T. Mizoguchi, Jpn. J. Appl. Phys. 55, 045502 (2016). 10.7567/JJAP.55.045502 Crossref, Google Scholar
 14 S. Kikuchi, H. Oda, S. Kiyohara, and T. Mizoguchi, Phys. B: Condens. Matter, (2017) in press. Google Scholar
 15 T. Ueno, T. D. Rhone, Z. Hou, T. Mizoguchi, and K. Tsuda, Mater. Discovery 4, 18 (2016). 10.1016/j.md.2016.04.001 Crossref, Google Scholar
 16 S. Kiyohara, H. Oda, T. Miyata, and T. Mizoguchi, Sci. Adv. 2, e1600746 (2016). 10.1126/sciadv.1600746 Crossref, Google Scholar
 17 C. E. Rasmussen, Gaussian Processes for Machine Learning (MIT Press, Cambridge, MA, 2006). Google Scholar
 18 D. M. Packwood and T. Hitosugi, Appl. Phys. Express 10, 065502 (2017). 10.7567/APEX.10.065502 Crossref, Google Scholar
 19 S. J. Pan and Q. Yang, IEEE Trans. Knowl. Data Eng. 22, 1345 (2010). 10.1109/TKDE.2009.191 Crossref, Google Scholar
 20 R. J. Kurtz and H. L. Heinisch, J. Nucl. Mater. 329–333, 1199 (2004). 10.1016/j.jnucmat.2004.04.262 Crossref, Google Scholar
 21 E. Nakajima and M. Takeuchi, TetsutoHagane 86, 357 (2000). 10.2355/tetsutohagane1955.86.5_357 Crossref, Google Scholar
 22 J. D. Gale, J. Chem. Soc., Faraday Trans. 93, 629 (1997). 10.1039/a606455h Crossref, Google Scholar
 23 M. W. Finnis and J. E. Sinclair, Philos. Mag. A 50, 45 (1984). 10.1080/01418618408244210 Crossref, Google Scholar
 24 (Supplemental Material) List of calculated GBs in this study. In all, 33 [110] symmetric tilt GBs of iron were calculated. Misorientation angle, Σ value, interface plane, number of atoms in the supercell, size of the supercell, and number of candidate configuration are listed. Google Scholar