Compact representations of microstructure images using triplet networks
Source abstract
Abstract
Abstract The microstructure of a material, typically characterized through a set of microscopy images of two-dimensional cross-sections, is a valuable source of information about the material and its properties. Every pixel of the image is a degree of freedom causing the dimensionality of the information space to be extremely high. This makes it difficult to recognize and extract all relevant information from the images. Human experts circumvent this by manually creating a lower-dimensional representation of the microstructure. However, the question of how a microstructure image can be best represented remains open. From the field of deep learning, we present triplet networks as a method to build highly compact representations of the microstructure, condensing the relevant information into a much smaller number of dimensions. We demonstrate that these representations can be created even with a limited amount of example images, and that they are able to distinguish between visually very similar microstructures. We discuss the interpretability and generalization of the representations. Having compact microstructure representations, it becomes easier to establish processing–structure–property links that are key to rational materials design.
Source: crossref · Quality: baseline
Research context
OpenAlex topics: Machine Learning in Materials Science · Cell Image Analysis Techniques · Image Processing Techniques and Applications
SciVX discovery tags are terminology-based. OpenAlex topics are source-provided classifications.
References (37)
Show source-provided references
1. Zaefferer, S., Ohlert, J. & Bleck, W. A study of microstructure, transformation mechanisms and correlation between microstructure and mechanical properties of a low alloyed TRIP steel. Acta Mater. 52, 2765–2778 (2004).10.1016/j.actamat.2004.02.044 ↗
2. Olson, G. B. Computational design of hierarchically structured materials. Science 277, 1237–1242 (1997).10.1126/science.277.5330.1237 ↗
3. Panchal, J. H., Kalidindi, S. R. & McDowell, D. L. Key computational modeling issues in Integrated Computational Materials Engineering. Comput. Aided Des. 45, 4–25 (2013).10.1016/j.cad.2012.06.006 ↗
4. Torquato, S. Statistical description of microstructures. Annu. Rev. Mater. Res. 32, 77–111 (2002).10.1146/annurev.matsci.32.110101.155324 ↗
5. Chowdhury, A., Kautz, E., Yener, B. & Lewis, D. Image driven machine learning methods for microstructure recognition. Comput. Mater. Sci. 123, 176–187 (2016).10.1016/j.commatsci.2016.05.034 ↗
6. Gola, J. et al. Advanced microstructure classification by data mining methods. Comput. Mater. Sci. 148, 324–335 (2018).10.1016/j.commatsci.2018.03.004 ↗
7. DeCost, B. L., Francis, T. & Holm, E. A. Exploring the microstructure manifold: Image texture representations applied to ultrahigh carbon steel microstructures. Acta Mater. 133, 30–40 (2017).10.1016/j.actamat.2017.05.014 ↗
8. Fullwood, D. T., Niezgoda, S. R. & Kalidindi, S. R. Microstructure reconstructions from 2-point statistics using phase-recovery algorithms. Acta Mater. 56, 942–948 (2008).10.1016/j.actamat.2007.10.044 ↗
9. Schmidhuber, J. Deep learning in neural networks: an overview. Neural Networks 61, 85–117 (2015).10.1016/j.neunet.2014.09.003 ↗
10. Hoffer, E. & Ailon, N. In International Workshop on Similarity-Based Pattern Recognition 84–92 (Springer, Cham, 2015).10.1007/978-3-319-24261-3_7 ↗
11. Friedman, J., Hastie, T. & Tibshirani, R. The Elements of Statistical Learning (Springer, New York, 2001).10.1007/978-0-387-21606-5 ↗
12. Gola, J. et al. Objective microstructure classification by support vector machine (SVM) using a combination of morphological parameters and textural features for low carbon steels. Computat. Mater. Sci. 160, 186–196 (2019).10.1016/j.commatsci.2019.01.006 ↗
13. Azimi, S. M., Britz, D., Engstler, M., Fritz, M. & M ücklich, F. Advanced steel microstructural classification by deep learning methods. Sci. Rep. 8, 1–14 (2018).10.1038/s41598-018-20037-5 ↗
14. Olson, G. B. & Azrin, M. Transformation behavior of TRIP steels. Metall. Trans. A 9, 713–721 (1978).10.1007/BF02659928 ↗
15. DeCost, B. L. & Holm, E. A. A computer vision approach for automated analysis and classification of microstructural image data. Comput. Mater. Sci. 110, 126–133 (2015).10.1016/j.commatsci.2015.08.011 ↗
16. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. Preprint at https://arxiv.org/abs/1409.1556 (2014).
17. DeCost, B. L., Lei, B., Francis, T. & Holm, E. A. High throughput quantitative metallography for complex microstructures using deep learning: a case study in ultrahigh carbon steel. Microsc. Microanal. 25, 21–29 (2019).10.1017/S1431927618015635 ↗
18. Ajioka, F., Wang, Z.-L., Ogawa, T. & Adachi, Y. Development of high accuracy segmentation model for mmicrostructure of steel by deep learning. ISIJ Int. 60, 954–959 (2020).10.2355/isijinternational.ISIJINT-2019-568 ↗
19. Xie, S., Girshick, R., Dollar, P., Tu, Z. & He, K. Aggregated residual transformations for deep neural networks. In IEEE Conf. Computer Vision and Pattern Recognition 5987-5995 (IEEE, New York, 2017).10.1109/CVPR.2017.634 ↗
20. Baldi, P. In ICML Workshop on Unsupervised and Transfer Learning 37–49 (Microtome, Brooklyn, 2012).
21. Pratt, L. Y. In Advances in Neural Information Processing Systems 204–211 (Morgan Kaufmann, Burlington, 1993).
22. Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. In Advances in Neural Information Processing Systems 3320–3328 (Curran, Brooklyn, 2014).
23. Schroff, F., Kalenichenko, D. & Philbin, J. Facenet: A unified embedding for face recognition and clustering. In IEEE Conf. Computer Vision and Pattern Recognition 815–823 (IEEE, New York, 2015).10.1109/CVPR.2015.7298682 ↗
24. Breiman, L. Random forests. Machine Learn. 45, 5–32 (2001).10.1023/A:1010933404324 ↗
25. Maaten, L. V. D. & Hinton, G. Visualizing data using t-SNE. J. Machine Learn. Res. 9, 2579–2605 (2008).
26. Webel, J., Gola, J., Britz, D. & Mücklich, F. A new analysis approach based on Haralick texture features for the characterization of microstructure on the example of low-alloy steels. Mater. Charact. 144, 584–596 (2018).10.1016/j.matchar.2018.08.009 ↗
27. Larochelle, H., Erhan, D. & Bengio, Y. Zero-data learning of new tasks. In AAAI Conf. Artificial Intelligence 646–651 (AAAI, Palo Alto, 2008).
28. Rohrbach, M., Stark, M. & Schiele, B. Evaluating knowledge transfer and zero-shot learning in a large-scale setting. In IEEE Conf. Computer Vision and Pattern Recognition 1641–1648 (IEEE, New York, 2011).10.1109/CVPR.2011.5995627 ↗
29. Smith, J. et al. Linking process, structure, property, and performance for metal-based Additive Manufacturing: computational approaches with experimental support. Comput. Mech. 57, 583–610 (2016).10.1007/s00466-015-1240-4 ↗
30. Paszke, A. et al. In Advances in Neural Information Processing Systems 8026–8037 (Curran, Brooklyn, 2019).
31. Howard, J. & Gugger, S. Fastai: a layered API for deep learning. Information 11, 108 (2020).10.3390/info11020108 ↗
32. Wu, C.-Y., Manmatha, R., Smola, A. J. & Krahenbuhl, P. Sampling matters in deep embedding learning. In The IEEE Int. Conf. Computer Vision 2859–2867 (IEEE, New York, 2017).10.1109/ICCV.2017.309 ↗
33. Oh Song, H., Xiang, Y., Jegelka, S. & Savarese, S. Deep metric learning via lifted structured feature embedding. In The IEEE Conf. Computer Vision and Pattern Recognition 4004–4012 (IEEE, New York, 2016).10.1109/CVPR.2016.434 ↗
34. Hermans, A., Beyer, L. & Leibe, B. In defense of the triplet loss for person re-identification. Preprint at https://arxiv.org/abs/1703.07737 (2017).
35. Probst, P., Wright, M. N. & Boulesteix, A.-L. Hyperparameters and tuning strategies for random forest. WIREs Data Mining Knowledge Discov. 9, e1301 (2019).10.1002/widm.1301 ↗
36. Cortes, C. & Vapnik, V. Support-vector networks. Machine Learn. 20, 273–297 (1995).
37. Duarte, E. & Wainer, J. Empirical comparison of cross-validation and internal metrics for tuning SVM hyperparameters. Pattern Recognit. Lett. 88, 6–11 (2017).10.1016/j.patrec.2017.01.007 ↗