Abstract

The paper presents a novel approach to applying Bayesian Optimization (BO) in predicting an unknown constraint boundary, also representing the discontinuity of an unknown function, for a feasibility check on the design space, thereby representing a classification tool to discern between a feasible and infeasible region. Bayesian optimization is a low-cost black-box global optimization tool in the Sequential Design Methods where one learns and updates knowledge from prior evaluated designs, and proceeds to the selection of new designs for future evaluation. However, BO is best suited to problems with the assumption of a continuous objective function and does not guarantee true convergence when having a discontinuous design space. This is because of the insufficient knowledge of the BO about the nature of the discontinuity of the unknown true function. In this paper, we have proposed to predict the location of the discontinuity using a BO algorithm on an artificially projected continuous design space from the original discontinuous design space. The proposed approach has been implemented in a thin tube design with the risk of creep-fatigue failure under constant loading of temperature and pressure. The stated risk depends on the location of the designs in terms of safe and unsafe regions, where the discontinuities lie at the transition between those regions; therefore, the discontinuity has also been treated as an unknown creep-fatigue failure constraint. The proposed BO algorithm has been trained to maximize sampling toward the unknown transition region, to act as a high accuracy classifier between safe and unsafe designs with minimal training cost. The converged solution has been validated for different design parameters with classification error rate and function evaluations at an average of <1% and ∼150, respectively. Finally, the performance of our proposed approach in terms of training cost and classification accuracy of thin tube design is shown to be better than the existing machine learning (ML) algorithms such as Support Vector Machine (SVM), Random Forest (RF), and Boosting.

References

1.
Huo
,
J.
, and
Liu
,
L.
,
2018
, “
An Optimization Framework of Multiobjective Artificial Bee Colony Algorithm Based on the MOEA Framework
,”
Comput. Intell. Neurosci.
, pp.
1
26
. https://doi.org/10.1155/2018/5865168
Article ID 5865168
.
2.
Feng
,
J.
,
Shen
,
W. Z.
, and
Li
,
Y.
,
2018
, “
An Optimization Framework for Wind Farm Design in Complex Terrain
,”
Appl. Sci.
,
8
(
11
), p.
2053
. 10.3390/app8112053
3.
Li
,
Y.
,
Chen
,
C.-K.
, and
Cho
,
Y.-Y.
,
2006
, “
A Unified Optimization Framework for Microelectronics Industry
,”
Proceedings of the 8th Annual Conference on Genetic and Evolutionary Computation
,
Seattle, WA
,
July
.
4.
Isaac
,
B
, and
Allaire
,
D
,
2019
, “
Expensive Black-Box Model Optimization Via a Gold Rush Policy
,”
ASME. J. Mech. Des.
,
141
(
3
), p.
031401
. https://doi.org/10.1115/1.4042113
5.
Sharif
,
B.
,
Wang
,
G. G.
, and
ElMekkawy
,
T. Y.
,
2008
, “
Mode Pursuing Sampling Method for Discrete Variable Optimization on Expensive Black-Box Functions
,”
ASME J. Mech. Des.
,
130
(
2
), p.
021402
. 10.1115/1.2803251
6.
Tran
,
A.
,
Wildey
,
T.
, and
McCann
,
S.
,
2019
, “
sBF-BO-2CoGP: A Sequential Bi-Fidelity Constrained Bayesian Optimization for Design Applications
,”
The ASME 2019 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
. http://dx.doi.org/10.1115/DETC2019-97986
7.
Brochu
,
E.
,
Cora
,
V. M.
, and
de Freitas
,
N.
,
2010
, “
A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning
,”
arXivLabs
. arXiv:1012.2599v1
8.
Bree
,
J.
,
1967
, “
Elastic-Plastic Behaviour of Thin Tubes Subjected to Internal Pressure and Intermittent High-Heat Fluxes With Application to Fast-Nuclear-Reactor Fuel Elements
,”
J. Strain Anal.
,
2
(
3
), pp.
226
238
. 10.1243/03093247V023226
9.
Saranam
,
V. R.
, and
Paul
,
B. K.
,
2018
, “
Feasibility of Using Diffusion Bonding for Producing Hybrid Printed Circuit Heat Exchangers for Nuclear Energy Applications
,”
Procedia Manuf.
,
26
, pp.
560
569
. 10.1016/j.promfg.2018.07.066
10.
Musumeci
,
F.
,
Rottondi
,
C.
,
Nag
,
A.
,
Macaluso
,
I.
,
Zibar
,
D.
,
Ruffini
,
M.
, and
Tornatore
,
M.
,
2019
, “
An Overview on Application of Machine Learning Techniques in Optical Networks
,”
IEEE Commun. Surv. Tutor.
,
21
(
2
), pp.
1383
1408
. 10.1109/COMST.2018.2880039
11.
Binkhonain
,
M.
, and
Zhao
,
L.
,
2019
, “
A Review of Machine Learning Algorithms for Identification and Classification of Non-Functional Requirements
,”
Expert Syst. Appl. X
,
1
. https://doi.org/10.1016/j.eswax.2019.100001
Article 100001
.
12.
Sekeroglu
,
B.
,
Hasan
,
S. S.
, and
Abdullah
,
S. M.
,
2020
, “
Comparison of Machine Learning Algorithms for Classification Problems
,”
Advances in Computer Vision
,
Las Vegas
,
May 2–3
, pp.
491
499
. http://dx.doi.org/10.1007/978-3-030-17798-0_39
13.
Kurata
,
G.
,
Xiang
,
B.
, and
Zhou
,
B.
, 2016, “
Improved Neural Network-Based Multi-Label Classification with Better Initialization Leveraging Label Co-occurrence
,”
Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
,
San Diego, CA
,
June
, pp.
521
526
. 10.18653/v1/N16-1063
14.
Kanellopoulos
,
I.
, and
Wilkinson
,
G. G.
,
1997
, “
Strategies and Best Practice for Neural Network Image Classification
,”
Int. J. Remote Sens.
,
18
(
4
), pp.
711
725
. 10.1080/014311697218719
15.
Inan
,
O. T.
,
Giovangrandi
,
L.
, and
Kovacs
,
G. T. A.
,
2006
, “
Robust Neural-Network-Based Classification of Premature Ventricular Contractions Using Wavelet Transform and Timing Interval Features
,”
IEEE Trans. Biomed. Eng.
,
53
(
12
), pp.
2507
2515
. 10.1109/TBME.2006.880879
16.
Li
,
Y.
,
Xie
,
W.
, and
Li
,
H.
,
2017
, “
Hyperspectral Image Reconstruction by Deep Convolutional Neural Network for Classification
,”
Pattern Recognition
,
63
, pp.
371
383
. https://doi.org/10.1016/j.patcog.2016.10.019
17.
Nasierding
,
G.
,
Tsoumakas
,
G.
, and
Kouzani
,
A. Z.
,
2009
, “
Clustering Based Multi-Label Classification for Image Annotation and Retrieval
,”
2009 IEEE International Conference on Systems, Man and Cybernetics
,
San Antonio, TX
,
Oct. 11–14
, pp.
4514
4519
.
18.
Barros
,
R. C.
,
Cerri
,
R.
,
Freitas
,
A. A.
, and
de Carvalho
,
A. C. P. L. F.
,
2013
, “
Probabilistic Clustering for Hierarchical Multi-Label Classification of Protein Functions
,”
Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2013, Prague, Czech Republic
,
Sept. 23–27
,
Berlin, Heidelberg
, pp.
385
400
.
19.
Zhu
,
X.
,
Vondrick
,
C.
,
Fowlkes
,
C. C.
, and
Ramanan
,
D.
,
2016
, “
Do We Need More Training Data?
,”
Int. J. Comput. Vis.
,
119
(
1
), pp.
76
92
. 10.1007/s11263-015-0812-2
20.
Cho
,
J.
,
Lee
,
K.
,
Shin
,
E.
,
Choy
,
G.
, and
Do
,
S.
2020
, “
How Much Data is Needed to Train a Medical Image Deep Learning System to Achieve Necessary High Accuracy?
,”
ArXiv151106348 Cs
,
Jan. 2016, Accessed November 20, 2020
. http://arxiv.org/abs/1511.06348
21.
Lizotte
,
D.
,
Wang
,
T.
,
Bowling
,
M.
, and
Schuurmans
,
D.
, 2007, “
Automatic Gait Optimization with Gaussian Process Regression
,”
IJCAI'07: Proceedings of the 20th International Joint Conference on Artifical Intelligence
,
Hyderabad, India
,
Jan. 6–12
.
22.
Lizotte
,
D.
,
2008
,
Practical Bayesian Optimization
,
University of Alberta
,
Edmonton, Canada
.
23.
Cora
,
V. M.
,
2008
,
Model-Based Active Learning in Hierarchical Policies
,
University of British Columbia Library
,
Vancouver, Canada
. 10.14288/1.0051276
24.
Frean
,
M.
, and
Boyle
,
P.
,
2008
, “
Using Gaussian Processes to Optimize Expensive Functions
,”
AI 2008: Advances in Artificial Intelligence, Auckland, New Zealand
,
Dec. 1–5
,
Berlin, Heidelberg
, pp.
258
267
.
25.
Martinez-Cantin
,
R.
,
de Freitas
,
N.
,
Brochu
,
E.
,
Castellanos
,
J.
, and
Doucet
,
A.
,
Aug. 2009
, “
A Bayesian Exploration-Exploitation Approach for Optimal Online Sensing and Planning With a Visually Guided Mobile Robot
,”
Auton. Robots
,
27
(
2
), pp.
93
103
. 10.1007/s10514-009-9130-2
26.
Chu
,
W.
, and
Ghahramani
,
Z.
,
2005
, “
Extensions of Gaussian Processes for Ranking: Semisupervised and Active Learning
,”
The NIPS 2005 Workshop on Learning to Rank
,
Whistler, BC
,
Dec. 9
.
27.
Thurstone
,
L. L.
,
1927
, “
A Law of Comparative Judgment
,”
Psychol. Rev.
,
34
(
4
), pp.
273
286
. 10.1037/h0070288
28.
Mosteller
,
F.
,
2006
, “Remarks on the Method of Paired Comparisons: I. The Least Squares Solution Assuming Equal Standard Deviations and Equal Correlations,”
Selected Papers of Frederick Mosteller
,
S. E.
Fienberg
, and
D. C.
Hoaglin
, eds.,
Springer
,
New York, NY
, pp.
157
162
.
29.
Holmes
,
C. C.
, and
Held
,
L.
,
2006
, “
Bayesian Auxiliary Variable Models for Binary and Multinomial Regression
,”
Bayesian Anal.
,
1
(
1
), pp.
145
168
. 10.1214/06-BA105
30.
Shu
,
L.
,
Jiang
,
P.
,
Shao
,
X.
, and
Wang
,
Y.
,
2020
, “
A New Multi-Objective Bayesian Optimization Formulation With the Acquisition Function for Convergence and Diversity
,”
ASME J. Mech. Des.
,
142
(
9
), p.
091703
. 10.1115/1.4046508
31.
Sarkar
,
S.
,
Mondal
,
S.
,
Joly
,
M.
,
Lynch
,
M. E.
,
Bopardikar
,
S. D.
,
Acharya
,
R.
, and
Perdikaris
,
P.
,
2019
, “
Multifidelity and Multiscale Bayesian Framework for High-Dimensional Engineering Design and Calibration
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121001
. 10.1115/1.4044598
32.
Sexton
,
T.
, and
Ren
,
M. Y.
,
2017
, “
Learning an Optimization Algorithm Through Human Design Iterations
,”
ASME J. Mech. Des.
,
139
(
10
), p.
101404
. 10.1115/1.4037344
33.
Hutter
,
F.
,
Hoos
,
H. H.
, and
Leyton-Brown
,
K.
, “
Sequential Model-Based Optimization for General Algorithm Configuration
,”
Learning and Intelligent Optimization
,
Berlin, Heidelberg
,
2011
, pp.
507
523
. 10.1007/978-3-642-25566-3_40.
34.
Shahriari
,
B.
,
Swersky
,
K.
,
Wang
,
Z.
,
Adams
,
R. P.
, and
de Freitas
,
N.
,
2016
, “
Taking the Human Out of the Loop: A Review of Bayesian Optimization
,”
Proc. IEEE
,
104
(
1
), pp.
148
175
. 10.1109/JPROC.2015.2494218
35.
Andrianakis
,
I.
, and
Challenor
,
P.
,
2012
, “
The Effect of the Nugget on Gaussian Process Emulators of Computer Models
,”
Comput. Stat. Data Anal.
,
56
(
12
), pp.
4215
4228
. 10.1016/j.csda.2012.04.020
36.
Pepelyshev
,
A.
,
2010
, “
The Role of the Nugget Term in the Gaussian Process Method
,”
mODa 9—Advances in Model-Oriented Design and Analysis
,
Bertinoro, Italy
,
June 14–18
,
Heidelberg
, pp.
149
156
.
37.
Xing
,
W.
,
Elhabian
,
S. Y.
,
Keshavarzzadeh
,
V.
, and
Kirby
,
R. M.
,
2020
, “
Shared-Gaussian Process: Learning Interpretable Shared Hidden Structure Across Data Spaces for Design Space Analysis and Exploration
,”
ASME J. Mech. Des.
,
142
(
8
), p.
081707
. 10.1115/1.4046074
38.
Bostanabad
,
R.
,
Chan
,
Y.-C.
,
Wang
,
L.
,
Zhu
,
P.
, and
Chen
,
W.
,
2019
, “
Globally Approximate Gaussian Processes for Big Data With Application to Data-Driven Metamaterials Design
,”
ASME J. Mech. Des.
,
141
(
11
), p.
111402
. 10.1115/1.4044257
39.
Erickson
,
C. B.
,
Ankenman
,
B. E.
, and
Sanchez
,
S. M.
,
2018
, “
Comparison of Gaussian Process Modeling Software
,”
Eur. J. Oper. Res.
,
266
(
1
), pp.
179
192
. 10.1016/j.ejor.2017.10.002
40.
Kushner
,
H. J.
,
1964
, “
A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
,”
ASME J. Basic Eng.
,
86
(
1
), pp.
97
106
. 10.1115/1.3653121
41.
Jones
,
D. R.
,
2001
, “
A Taxonomy of Global Optimization Methods Based on Response Surfaces
,”
J. Glob. Optim.
,
21
(
4
), pp.
345
383
. 10.1023/A:1012771025575
42.
Cox
,
D. D.
, and
John
,
S.
, “
A Statistical Method for Global Optimization
,”
[Proceedings] 1992 IEEE International Conference on Systems, Man, and Cybernetics
,
Chicago, IL
,
Oct. 18–21
, Vol.
2
, pp.
1241
1246
.
43.
Bastos
,
L. S.
, and
O’Hagan
,
A.
,
2009
, “
Diagnostics for Gaussian Process Emulators
,”
Technometrics
,
51
(
4
), pp.
425
438
. 10.1198/TECH.2009.08019
44.
Chen
,
H.
,
Loeppky
,
J. L.
,
Sacks
,
J.
, and
Welch
,
W. J.
,
2016
, “
Analysis Methods for Computer Experiments: How to Assess and What Counts?
,”
Stat. Sci.
,
31
(
1
), pp.
40
60
. 10.1214/15-STS531
45.
Nielsen
,
H. B.
,
Lophaven
,
S. N.
, and
Søndergaard
,
J.
,
2002
, “
DACE—A Matlab Kriging Toolbox
.” https://orbit.dtu.dk/en/publications/dace-a-matlab-kriging-toolbox
46.
Lophaven
,
N. S.
,
Nielsen
,
B. H.
, and
Søndergaard
,
J.
,
2002
,
DACE – A Matlab Kriging Toolbox, Version 2.0.
,
DTU Orbit
, http://www.imm.dtu.dk/pubdb/p.php?3213
47.
Meyer
,
D.
,
Dimitriadou
,
E.
,
Hornik
,
K.
,
Leisch
,
F.
, and
Weingessel
,
A.
,
2020
,
Misc Functions of the Department of Statistics, Probability Theory Group (Formerly: E1071), TU Wien
.
48.
Breiman
,
L
,
2001
, “
Random Forests
,”
Machine Learning
,
45
, pp.
5
32
. https://doi.org/10.1023/A:1010933404324
49.
Greenwell
,
B.
,
Boehmke
,
B.
,
Cunningham
,
J.
, and
G. B. M. Developers
,
2020
,
Generalized Boosted Regression Models
, https://github.com/gbm-developers/gbm.
You do not currently have access to this content.