Abstract

Despite the rapid adoption of deep learning models in additive manufacturing (AM), significant quality assurance challenges continue to persist. This is further emphasized by the limited availability of sample objects for monitoring AM-fabricated builds. Thus, this study advances an emerging diffusion generative model, i.e., the denoising diffusion implicit model (DDIM), for layer-wise image augmentation and monitoring in AM. The generative model can be used to generate potential layer-wise variations, which can be further studied to understand their causation and prevent their occurrence. The proposed models integrate two proposed kernel-based distance metrics into the DDIM framework for effective layer-wise AM image augmentation. These newly proposed metrics include a modified version of the kernel inception distance (m-KID) as well as an integration of m-KID and the inception score (IS) termed KID-IS. These novel integrations demonstrate great potential for maintaining both similarity and consistency in AM layer-wise image augmentation, while simultaneously exploring possible unobserved process variations. In the case study, six different cases based on both metal-based and polymer-based fused filament fabrication (FFF) are examined. The results indicate that both the proposed DDIM/m-KID and DDIM/KID-IS models outperform the four benchmark methods, including the popular denoising diffusion probabilistic models (DDPMs), and three other generative adversarial networks (GANs). Overall, DDIM/KID-IS emerges as the best-performing model with an average KID score of 0.840, m-KID score of 0.1185, peak signal-to-noise ratio (PSNR) of 18.150, and structural similarity index measure (SSIM) of 0.173, which demonstrated strong capabilities in generating potential AM process variations in terms of layer-wise images.

Graphical Abstract Figure
Graphical Abstract Figure
Close modal

References

1.
Gao
,
W.
,
Zhang
,
Y.
,
Ramanujan
,
D.
,
Ramani
,
K.
,
Chen
,
Y.
,
Williams
,
C. B.
,
Wang
,
C. C. L.
,
Shin
,
Y. C.
,
Zhang
,
S.
, and
Zavattieri
,
P. D.
,
2015
, “
The Status, Challenges, and Future of Additive Manufacturing in Engineering
,”
Comput. Aided Des.
,
69
, pp.
65
89
.
2.
Liu
,
C.
,
Tian
,
W.
, and
Kan
,
C.
,
2022
, “
When AI Meets Additive Manufacturing: Challenges and Emerging Opportunities for Human-Centered Products Development
,”
J. Manuf. Syst.
,
64
, pp.
648
656
.
3.
Yangue
,
E.
,
Ye
,
Z.
,
Kan
,
C.
, and
Liu
,
C.
,
2023
, “
Integrated Deep Learning-Based Online Layer-Wise Surface Prediction of Additive Manufacturing
,”
Manuf. Lett.
,
35
, pp.
760
769
.
4.
Ye
,
Z.
,
Liu
,
C.
,
Tian
,
W.
, and
Kan
,
C.
,
2021
, “
In-Situ Point Cloud Fusion for Layer-Wise Monitoring of Additive Manufacturing
,”
J. Manuf. Syst.
,
61
, pp.
210
222
.
5.
LeCun
,
Y.
,
Bengio
,
Y.
, and
Hinton
,
G.
,
2015
, “
Deep Learning
,”
Nature
,
521
(
7553
), pp.
436
444
.
6.
Chawla
,
N. V.
,
Bowyer
,
K. W.
,
Hall
,
L. O.
, and
Kegelmeyer
,
W. P.
,
2002
, “
SMOTE: Synthetic Minority Over-Sampling Technique
,”
J. Artif. Intell. Res.
,
16
, pp.
321
357
.
7.
Chung
,
J.
,
Shen
,
B.
, and
Kong
,
Z. J.
,
2023
, “
Anomaly Detection in Additive Manufacturing Processes Using Supervised Classification With Imbalanced Sensor Data Based on Generative Adversarial Network
,”
J. Intell. Manuf.
8.
Yang
,
L.
,
Zhang
,
Z.
,
Song
,
Y.
,
Hong
,
S.
,
Xu
,
R.
,
Zhao
,
Y.
,
Zhang
,
W.
,
Cui
,
B.
, and
Yang
,
M.-H.
,
2023
, “
Diffusion Models: A Comprehensive Survey of Methods and Applications
,”
ACM
,
56
(
4
).
9.
Yang
,
S.
, and
Zhao
,
Y. F.
,
2015
, “
Additive Manufacturing-Enabled Design Theory and Methodology: A Critical Review
,”
Int. J. Adv. Manuf. Technol.
,
80
(
1
), pp.
327
342
.
10.
Fullington
,
D.
,
Bian
,
L.
, and
Tian
,
W.
,
2023
, “
Design De-Identification of Thermal History for Collaborative Process-Defect Modeling of Directed Energy Deposition Processes
,”
ASME J. Manuf. Sci. Eng.
,
145
(
5
), p.
051004
.
11.
Sanaei
,
N.
,
Fatemi
,
A.
, and
Phan
,
N.
,
2019
, “
Defect Characteristics and Analysis of Their Variability in Metal L-PBF Additive Manufacturing
,”
Mater. Des.
,
182
, p.
108091
.
12.
Bau
,
D.
,
Zhu
,
J.-Y.
,
Wulff
,
J.
,
Peebles
,
W.
,
Strobelt
,
H.
,
Zhou
,
B.
, and
Torralba
,
A.
,
2019
, “
Seeing What a GAN Cannot Generate
,” pp.
4502
4511
.
13.
Arora
,
S.
,
Ge
,
R.
,
Liang
,
Y.
,
Ma
,
T.
, and
Zhang
,
Y.
,
2017
, “
Generalization and Equilibrium in Generative Adversarial Nets (GANs)
,”
Proceedings of the 34th International Conference on Machine Learning
,
Sydney, Australia
,
Aug. 6–11
,
PMLR
, pp.
224
232
.
14.
Nagarajan
,
V.
,
Raffel
,
C.
, and
Goodfellow
,
I. J.
, “
Theoretical Insights Into Memorization in GANs
.”
15.
Gidel
,
G.
,
Berard
,
H.
,
Vignoud
,
G.
,
Vincent
,
P.
, and
Lacoste-Julien
,
S.
,
2020
, “
A Variational Inequality Perspective on Generative Adversarial Networks
.”
16.
Arjovsky
,
M.
,
Chintala
,
S.
, and
Bottou
,
L.
,
2017
, “
Wasserstein Generative Adversarial Networks
,”
Proceedings of the 34th International Conference on Machine Learning
,
Sydney, Australia
,
Aug. 6–11
,
PMLR
, pp.
214
223
.
17.
Xames
,
M. D.
,
Torsha
,
F. K.
, and
Sarwar
,
F.
,
2023
, “
A Systematic Literature Review on Recent Trends of Machine Learning Applications in Additive Manufacturing
,”
J. Intell. Manuf.
,
34
(
6
), pp.
2529
2555
.
18.
Shi
,
Z.
,
Mamun
,
A. A.
,
Kan
,
C.
,
Tian
,
W.
, and
Liu
,
C.
,
2022
, “
An LSTM-Autoencoder Based Online Side Channel Monitoring Approach for Cyber-Physical Attack Detection in Additive Manufacturing
,”
J. Intell. Manuf.
,
34
(
4
), pp.
1815
1831
.
19.
Ren
,
W.
,
Wen
,
G.
,
Zhang
,
Z.
, and
Mazumder
,
J.
,
2022
, “
Quality Monitoring in Additive Manufacturing Using Emission Spectroscopy and Unsupervised Deep Learning
,”
Mater. Manuf. Processes
,
37
(
11
), pp.
1339
1346
.
20.
Akbari
,
P.
,
Ogoke
,
F.
,
Kao
,
N.-Y.
,
Meidani
,
K.
,
Yeh
,
C.-Y.
,
Lee
,
W.
, and
Barati Farimani
,
A.
,
2022
, “
MeltpoolNet: Melt Pool Characteristic Prediction in Metal Additive Manufacturing Using Machine Learning
,”
Addit. Manuf.
,
55
, p.
102817
.
21.
Bappy
,
M. M.
,
Liu
,
C.
,
Bian
,
L.
, and
Tian
,
W.
,
2022
, “
Morphological Dynamics-Based Anomaly Detection Towards In Situ Layer-Wise Certification for Directed Energy Deposition Processes
,”
ASME J. Manuf. Sci. Eng.
,
144
(
11
), p.
111007
.
22.
Esfahani
,
M. N.
,
Bappy
,
M. M.
,
Bian
,
L.
, and
Tian
,
W.
,
2022
, “
In-Situ Layer-Wise Certification for Direct Laser Deposition Processes Based on Thermal Image Series Analysis
,”
J. Manuf. Process.
,
75
, pp.
895
902
.
23.
Kononenko
,
D. Y.
,
Nikonova
,
V.
,
Seleznev
,
M.
,
van den Brink
,
J.
, and
Chernyavsky
,
D.
,
2023
, “
An In Situ Crack Detection Approach in Additive Manufacturing Based on Acoustic Emission and Machine Learning
,”
Addit. Manuf. Lett.
,
5
, p.
100130
.
24.
Lyu
,
J.
, and
Manoochehri
,
S.
,
2021
, “
Online Convolutional Neural Network-Based Anomaly Detection and Quality Control for Fused Filament Fabrication Process
,”
Virtual Phys. Prototyp.
,
16
(
2
), pp.
160
177
.
25.
Valizadeh
,
M.
, and
Wolff
,
S. J.
,
2022
, “
Convolutional Neural Network Applications in Additive Manufacturing: A Review
,”
Adv. Ind. Manuf. Eng.
,
4
, p.
100072
.
26.
Banadaki
,
Y.
,
Razaviarab
,
N.
,
Fekrmandi
,
H.
, and
Sharifi
,
S.
,
2020
, “
Toward Enabling a Reliable Quality Monitoring System for Additive Manufacturing Process Using Deep Convolutional Neural Networks
,”
arXiv
. https://arxiv.org/abs/2003.08749
27.
Trinks
,
S.
, and
Felden
,
C.
,
2019
, “
Image Mining for Real Time Quality Assurance in Rapid Prototyping
,”
2019 IEEE International Conference on Big Data (Big Data)
,
Los Angeles, CA
,
Dec. 9–12
, pp.
3529
3534
.
28.
Patel
,
S.
,
Mekavibul
,
J.
,
Park
,
J.
,
Kolla
,
A.
,
French
,
R.
,
Kersey
,
Z.
, and
Lewin
,
G. C.
,
2019
, “
Using Machine Learning to Analyze Image Data From Advanced Manufacturing Processes
,”
2019 Systems and Information Engineering Design Symposium (SIEDS)
,
Charlottesville, VA
,
Apr. 26
, pp.
1
5
.
29.
Mahato
,
V.
,
Obeidi
,
M. A.
,
Brabazon
,
D.
, and
Cunningham
,
P.
,
2022
, “
Detecting Voids in 3D Printing Using Melt Pool Time Series Data
,”
J. Intell. Manuf.
,
33
(
3
), pp.
845
852
.
30.
Lyu
,
J.
,
Akhavan Taheri Boroujeni
,
J.
, and
Manoochehri
,
S.
,
2021
, “
In-Situ Laser-Based Process Monitoring and In-Plane Surface Anomaly Identification for Additive Manufacturing Using Point Cloud and Machine Learning
,”
ASME 2021 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
, Paper No: DETC2021-69436.
31.
Song
,
Y.
, and
Kingma
,
D. P.
,
2021
, “
How to Train Your Energy-Based Models
,”
arXiv
. https://arxiv.org/abs/2101.03288
32.
Kingma
,
D. P.
, and
Welling
,
M.
,
2019
, “
An Introduction to Variational Autoencoders
,”
MAL
,
12
(
4
), pp.
307
392
.
33.
Goodfellow
,
I.
,
Pouget-Abadie
,
J.
,
Mirza
,
M.
,
Xu
,
B.
,
Warde-Farley
,
D.
,
Ozair
,
S.
,
Courville
,
A.
, and
Bengio
,
Y.
,
2020
, “
Generative Adversarial Networks
,”
Commun. ACM
,
63
(
11
), pp.
139
144
.
34.
Zhang
,
Q.
, and
Chen
,
Y.
,
2021
, “Diffusion Normalizing Flow,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc.
, pp.
16280
16291
.
35.
Lee
,
K.-H.
, and
Yun
,
G. J.
,
2023
, “
Microstructure Reconstruction Using Diffusion-Based Generative Models
,”
Mech. Adv. Mater. Struct.
, pp.
1
19
.
36.
Ye
,
D.
,
Hong
,
G. S.
,
Zhang
,
Y.
,
Zhu
,
K.
, and
Fuh
,
J. Y. H.
,
2018
, “
Defect Detection in Selective Laser Melting Technology by Acoustic Signals With Deep Belief Networks
,”
Int. J. Adv. Manuf. Technol.
,
96
(
5
), pp.
2791
2801
.
37.
Ye
,
D.
,
Hsi Fuh
,
J. Y.
,
Zhang
,
Y.
,
Hong
,
G. S.
, and
Zhu
,
K.
,
2018
, “
In Situ Monitoring of Selective Laser Melting Using Plume and Spatter Signatures by Deep Belief Networks
,”
ISA Trans.
,
81
, pp.
96
104
.
38.
Wang
,
J.
,
Ma
,
Y.
,
Zhang
,
L.
,
Gao
,
R. X.
, and
Wu
,
D.
,
2018
, “
Deep Learning for Smart Manufacturing: Methods and Applications
,”
J. Manuf. Syst.
,
48
, pp.
144
156
.
39.
Song
,
Z.
,
Wang
,
X.
,
Gao
,
Y.
,
Son
,
J.
, and
Wu
,
J.
,
2023
, “
A Hybrid Deep Generative Network for Pore Morphology Prediction in Metal Additive Manufacturing
,”
ASME J. Manuf. Sci. Eng.
,
145
(
7
), p.
071005
.
40.
Ghayoomi Mohammadi
,
M.
,
Mahmoud
,
D.
, and
Elbestawi
,
M.
,
2021
, “
On the Application of Machine Learning for Defect Detection in L-PBF Additive Manufacturing
,”
Opt. Laser Technol.
,
143
, p.
107338
.
41.
Lew
,
A. J.
, and
Buehler
,
M. J.
,
2021
, “
Encoding and Exploring Latent Design Space of Optimal Material Structures via a VAE-LSTM Model
,”
Forces Mech.
,
5
, p.
100054
.
42.
Li
,
Y.
,
Shi
,
Z.
,
Liu
,
C.
,
Tian
,
W.
,
Kong
,
Z.
, and
Williams
,
C. B.
,
2022
, “
Augmented Time Regularized Generative Adversarial Network (ATR-GAN) for Data Augmentation in Online Process Anomaly Detection
,”
IEEE Trans. Autom. Sci. Eng.
,
19
(
4
), pp.
3338
3355
.
43.
Hertlein
,
N.
,
Buskohl
,
P. R.
,
Gillman
,
A.
,
Vemaganti
,
K.
, and
Anand
,
S.
,
2021
, “
Generative Adversarial Network for Early-Stage Design Flexibility in Topology Optimization for Additive Manufacturing
,”
J. Manuf. Syst.
,
59
, pp.
675
685
.
44.
He
,
X.
,
Chang
,
Z.
,
Zhang
,
L.
,
Xu
,
H.
,
Chen
,
H.
, and
Luo
,
Z.
,
2022
, “
A Survey of Defect Detection Applications Based on Generative Adversarial Networks
,”
IEEE Access
,
10
, pp.
113493
113512
.
45.
Saharia
,
C.
,
Ho
,
J.
,
Chan
,
W.
,
Salimans
,
T.
,
Fleet
,
D. J.
, and
Norouzi
,
M.
,
2023
, “
Image Super-Resolution via Iterative Refinement
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
45
(
4
), pp.
4713
4726
.
46.
Zhang
,
H.
,
Prasad Vallabh
,
C. K.
, and
Zhao
,
X.
,
2023
, “
Machine Learning Enhanced High Dynamic Range Fringe Projection Profilometry for In-Situ Layer-Wise Surface Topography Measurement During LPBF Additive Manufacturing
,”
Precis. Eng.
,
84
, pp.
1
14
.
47.
Ogoke
,
F.
,
Liu
,
Q.
,
Ajenifujah
,
O.
,
Myers
,
A.
,
Quirarte
,
G.
,
Beuth
,
J.
,
Malen
,
J.
, and
Farimani
,
A. B.
,
2023
, “
Inexpensive High Fidelity Melt Pool Models in Additive Manufacturing Using Generative Deep Diffusion
,”
arXiv
. https://arxiv.org/abs/2311.16168
48.
Zhao
,
S.
,
Ren
,
H.
,
Yuan
,
A.
,
Song
,
J.
,
Goodman
,
N.
, and
Ermon
,
S.
,
2018
, “Bias and Generalization in Deep Generative Models: An Empirical Study,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
49.
Gulrajani
,
I.
,
Ahmed
,
F.
,
Arjovsky
,
M.
,
Dumoulin
,
V.
, and
Courville
,
A. C.
,
2017
, “Improved Training of Wasserstein GANs,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
50.
Song
,
J.
,
Meng
,
C.
, and
Ermon
,
S.
,
2022
, “
Denoising Diffusion Implicit Models
.”
51.
Ho
,
J.
,
Jain
,
A.
, and
Abbeel
,
P.
,
2020
, “Denoising Diffusion Probabilistic Models,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc.
, pp.
6840
6851
.
52.
Bengio
,
Y.
,
Yao
,
L.
,
Alain
,
G.
, and
Vincent
,
P.
,
2013
, “Generalized Denoising Auto-Encoders as Generative Models,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
53.
Song
,
Y.
,
Sohl-Dickstein
,
J.
,
Kingma
,
D. P.
,
Kumar
,
A.
,
Ermon
,
S.
, and
Poole
,
B.
,
2021
, “
Score-Based Generative Modeling Through Stochastic Differential Equations
.”
54.
Song
,
Y.
, and
Ermon
,
S.
,
2019
, “Generative Modeling by Estimating Gradients of the Data Distribution,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
55.
Ronneberger
,
O.
,
Fischer
,
P.
, and
Brox
,
T.
,
2015
, “
U-Net: Convolutional Networks for Biomedical Image Segmentation
.”
56.
Sheikh
,
H. R.
, and
Bovik
,
A. C.
,
2006
, “
Image Information and Visual Quality
,”
IEEE Trans. Image Process.
,
15
(
2
), pp.
430
444
.
57.
Wang
,
Z.
, and
Bovik
,
A. C.
,
2002
, “
A Universal Image Quality Index
,”
IEEE Signal Process. Lett.
,
9
(
3
), pp.
81
84
.
58.
Salimans
,
T.
,
Goodfellow
,
I.
,
Zaremba
,
W.
,
Cheung
,
V.
,
Radford
,
A.
,
Chen
,
X.
, and
Chen
,
X.
,
2016
, “Improved Techniques for Training GANs,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
59.
Heusel
,
M.
,
Ramsauer
,
H.
,
Unterthiner
,
T.
,
Nessler
,
B.
, and
Hochreiter
,
S.
,
2017
, “GANS Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium,”
Advances in Neural Information Processing Systems
,
Curran Associates, Inc
.
60.
Bińkowski
,
M.
,
Sutherland
,
D. J.
,
Arbel
,
M.
, and
Gretton
,
A.
,
2021
, “
Demystifying MMD GANs
.”
61.
Betzalel
,
E.
,
Penso
,
C.
,
Navon
,
A.
, and
Fetaya
,
E.
,
2022
, “
A Study on the Evaluation of Generative Models
,”
arXiv
. https://arxiv.org/abs/2206.10935
62.
Loshchilov
,
I.
, and
Hutter
,
F.
,
2019
, “
Decoupled Weight Decay Regularization
.”
63.
Radford
,
A.
,
Metz
,
L.
, and
Chintala
,
S.
,
2016
, “
Unsupervised Representation Learning With Deep Convolutional Generative Adversarial Networks
.”
64.
Mao
,
X.
,
Li
,
Q.
,
Xie
,
H.
,
Lau
,
R. Y. K.
,
Wang
,
Z.
, and
Smolley
,
S. P.
, “
Least Squares Generative Adversarial Networks
.” https://ieeexplore.ieee.org/document/8237566
65.
Ridder
,
H. d.
,
1998
, “Psychophysical Evaluation of Image Quality: From Judgment to Impression,”
Human Vision and Electronic Imaging III
,
SPIE
, pp.
252
263
.
You do not currently have access to this content.