Abstract

The importance–performance analysis (IPA) is a widely used technique to guide strategic planning for the improvement of customer satisfaction. Compared with surveys, numerous online reviews can be easily collected at a lower cost. Online reviews provide a promising source for the IPA. This paper proposes an approach for conducting the IPA from online reviews for product design. Product attributes from online reviews are first identified by latent Dirichlet allocation. The performance of the identified attributes is subsequently estimated by the aspect-based sentiment analysis of IBM Watson. Finally, the importance of the identified attributes is estimated by evaluating the effect of sentiments of each product attribute on the overall rating using an explainable deep neural network. A Shapley additive explanation-based method is proposed to estimate the importance values of product attributes with a low variance by combining the effect of the input features from multiple optimal neural networks with a high performance. A case study of smartphones is presented to demonstrate the proposed approach. The performance and importance estimates of the proposed approach are compared with those of previous sentiment analysis and neural network-based method, and the results exhibit that the former can perform IPA more reliably. The proposed approach uses minimal manual operation and can support companies to take decisions rapidly and effectively, compared with survey-based methods.

References

1.
Martilla
,
J. A.
, and
James
,
J. C.
,
1977
, “
Importance–Performance Analysis
,”
J. Market.
,
41
(
1
), pp.
77
79
. 10.1177/002224297704100112
2.
Bi
,
J.-W.
,
Liu
,
Y.
,
Fan
,
Z.-P.
, and
Zhang
,
J.
,
2019
, “
Wisdom of Crowds: Conducting Importance-Performance Analysis (IPA) Through Online Reviews
,”
Tourism Manage.
,
70
, pp.
460
478
. 10.1016/j.tourman.2018.09.010
3.
Chu
,
R. K.
, and
Choi
,
T.
,
2000
, “
An Importance-Performance Analysis of Hotel Selection Factors in the Hong Kong Hotel Industry: A Comparison of Business and Leisure Travellers
,”
Tourism Manage.
,
21
(
4
), pp.
363
377
. 10.1016/S0261-5177(99)00070-9
4.
Deng
,
W.
,
2007
, “
Using a Revised Importance–Performance Analysis Approach: The Case of Taiwanese Hot Springs Tourism
,”
Tourism Manage.
,
28
(
5
), pp.
1274
1284
. 10.1016/j.tourman.2006.07.010
5.
Seng Wong
,
M.
,
Hideki
,
N.
, and
George
,
P.
,
2011
, “
The Use of Importance–Performance Analysis (IPA) in Evaluating Japan’s E-Government Services
,”
J. Theor. Appl. Electron. Commerce Res.
,
6
(
2
), pp.
17
30
. 10.4067/S0718-18762011000200003
6.
Izadi
,
A.
,
Jahani
,
Y.
,
Rafiei
,
S.
,
Masoud
,
A.
, and
Vali
,
L.
,
2017
, “
Evaluating Health Service Quality: Using Importance Performance Analysis
,”
Int. J. Health Care Qual. Assurance
,
30
(
7
), pp.
656
663
. 10.1108/IJHCQA-02-2017-0030
7.
Dahlgaard-Park
,
S. M.
,
Pezeshki
,
V.
,
Mousavi
,
A.
, and
Grant
,
S.
,
2009
, “
Importance–Performance Analysis of Service Attributes and Its Impact on Decision Making in the Mobile Telecommunication Industry
,”
Meas. Bus. Excell.
,
13
(
1
), pp.
82
92
.
8.
MacDonald
,
E.
,
Backsell
,
M.
,
Gonzalez
,
R.
, and
Papalambros
,
P.
,
2006
, “
The Kano Method’s Imperfections, and Implications in Product Decision Theory
,”
Proceedings of the 2006 International Design Research Symposium
,
Lisbon, Portugal
,
Nov. 1–4
, pp.
1
12
.
9.
Joung
,
J.
,
Jung
,
K.
,
Ko
,
S.
, and
Kim
,
K.
,
2019
, “
Customer Complaints Analysis Using Text Mining and Outcome-Driven Innovation Method for Market-Oriented Product Development
,”
Sustainability
,
11
(
1
), p.
40
. 10.3390/su11010040
10.
Ordenes
,
F. V.
,
Theodoulidis
,
B.
,
Burton
,
J.
,
Gruber
,
T.
, and
Zaki
,
M.
,
2014
, “
Analyzing Customer Experience Feedback Using Text Mining: A Linguistics-Based Approach
,”
J. Service Res.
,
17
(
3
), pp.
278
295
. 10.1177/1094670514524625
11.
Zhou
,
F.
,
Jiao
,
R. J.
, and
Linsey
,
J. S.
,
2015
, “
Latent Customer Needs Elicitation by Use Case Analogical Reasoning From Sentiment Analysis of Online Product Reviews
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071401
. 10.1115/1.4030159
12.
Zimmermann
,
M.
,
Ntoutsi
,
E.
, and
Spiliopoulou
,
M.
,
2015
, “
Discovering and Monitoring Product Features and the Opinions on Them With Opinstream
,”
Neurocomputing
,
150
, pp.
318
330
. 10.1016/j.neucom.2014.04.079
13.
Hou
,
T.
,
Yannou
,
B.
,
Leroy
,
Y.
, and
Poirson
,
E.
,
2019
, “
Mining Changes in User Expectation Over Time From Online Reviews
,”
ASME J. Mech. Des.
,
141
(
9
), p.
091102
. 10.1115/1.4042793
14.
Suryadi
,
D.
, and
Kim
,
H.
,
2018
, “
A Systematic Methodology Based on Word Embedding for Identifying the Relation Between Online Customer Reviews and Sales Rank
,”
ASME J. Mech. Des.
,
140
(
12
), p.
121403
. 10.1115/1.4040913
15.
Zhang
,
H.
,
Sekhari
,
A.
,
Ouzrout
,
Y.
, and
Bouras
,
A.
,
2016
, “
Jointly Identifying Opinion Mining Elements and Fuzzy Measurement of Opinion Intensity to Analyze Product Features
,”
Eng. Appl. Artif. Intell.
,
47
,
122
139
. 10.1016/j.engappai.2015.06.007
16.
Jeong
,
B.
,
Yoon
,
J.
, and
Lee
,
J.-M.
,
2019
, “
Social Media Mining for Product Planning: A Product Opportunity Mining Approach Based on Topic Modeling and Sentiment Analysis
,”
Int. J. Inform. Manage.
,
48
,
280
290
. 10.1016/j.ijinfomgt.2017.09.009
17.
Jiang
,
H.
,
Kwong
,
C.
, and
Yung
,
K.
,
2017
, “
Predicting Future Importance of Product Features Based on Online Customer Reviews
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111413
.
18.
Rai
,
R.
,
2012
, “
Identifying Key Product Attributes and Their Importance Levels From Online Customer Reviews
,”
ASME 2012 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Chicago, IL
,
Aug. 12–15
, pp.
533
540
.
19.
Decker
,
R.
, and
Trusov
,
M.
,
2010
, “
Estimating Aggregate Consumer Preferences From Online Product Reviews
,”
Int. J. Res. Market.
,
27
(
4
), pp.
293
307
. 10.1016/j.ijresmar.2010.09.001
20.
Chen
,
W.
,
Conner
,
C.
, and
Yannou
,
B.
,
2015
, “
User Needs and Preferences in Engineering Design
,”
ASME J. Mech. Des.
,
137
(
7
), p.
070301
.
21.
Wang
,
W.
,
Li
,
Z.
,
Tian
,
Z.
,
Wang
,
J.
, and
Cheng
,
M.
,
2018
, “
Extracting and Summarizing Affective Features and Responses From Online Product Descriptions and Reviews: A Kansei Text Mining Approach
,”
Eng. Appl. Artif. Intell.
,
73
, pp.
149
162
. 10.1016/j.engappai.2018.05.005
22.
Singh
,
A.
, and
Tucker
,
C. S.
,
2017
, “
A Machine Learning Approach to Product Review Disambiguation Based on Function, Form and Behavior Classification
,”
Decis. Support Syst.
,
97
,
81
91
. 10.1016/j.dss.2017.03.007
23.
Liu
,
Y.
,
Jin
,
J.
,
Ji
,
P.
,
Harding
,
J. A.
, and
Fung
,
R. Y.
,
2013
, “
Identifying Helpful Online Reviews: A Product Designer’s Perspective
,”
Comput. Aided Des.
,
45
(
2
), pp.
180
194
. 10.1016/j.cad.2012.07.008
24.
Chaklader
,
R.
, and
Parkinson
,
M. B.
,
2017
, “
Data-Driven Sizing Specification Utilizing Consumer Text Reviews
,”
ASME J. Mech. Des.
,
139
(
11
), p.
111406
. 10.1115/1.4037476
25.
Ferguson
,
T.
,
Greene
,
M.
,
Repetti
,
F.
,
Lewis
,
K.
, and
Behdad
,
S.
,
2015
, “
Combining Anthropometric Data and Consumer Review Content to Inform Design for Human Variability
,”
ASME 2015 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Boston, MA
,
Aug. 2–5
.
26.
Zhou
,
F.
,
Ayoub
,
J.
,
Xu
,
Q.
, and
Jessie Yang
,
X.
,
2020
, “
A Machine Learning Approach to Customer Needs Analysis for Product Ecosystems
,”
ASME J. Mech. Des.
,
142
(
1
), p.
011101
. 10.1115/1.4044435
27.
Suryadi
,
D.
, and
Kim
,
H. M.
,
2019
, “
A Data-Driven Approach to Product Usage Context Identification From Online Customer Reviews
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121104
.
28.
Wang
,
W.
,
Feng
,
Y.
, and
Dai
,
W.
,
2018
, “
Topic Analysis of Online Reviews for Two Competitive Products Using Latent Dirichlet Allocation
,”
Electron. Commer. Res. Appl.
,
29
,
142
156
. 10.1016/j.elerap.2018.04.003
29.
El Dehaibi
,
N.
,
Goodman
,
N. D.
, and
MacDonald
,
E. F.
,
2019
, “
Extracting Customer Perceptions of Product Sustainability From Online Reviews
,”
ASME J. Mech. Des.
,
141
(
12
), p.
121103
. 10.1115/1.4044522
30.
Wang
,
L.
,
Youn
,
B.
,
Azarm
,
S.
, and
Kannan
,
P.
,
2011
, “
Customer-Driven Product Design Selection Using Web Based User-Generated Content
,”
ASME 2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Washington, DC
,
Aug. 28–31
, pp.
405
419
.
31.
Nasim
,
Z.
, and
Haider
,
S.
,
2017
, “
Absa Toolkit: An Open Source Tool for Aspect Based Sentiment Analysis
,”
Int. J. Artif. Intell. Tools
,
26
(
6
), p.
1750023
. 10.1142/S0218213017500233
32.
Mikulić
,
J.
, and
Prebežac
,
D.
,
2012
, “
Accounting for Dynamics in Attribute-Importance and for Competitor Performance to Enhance Reliability of BPNN-Based Importance–Performance Analysis
,”
Expert Syst. Appl.
,
39
(
5
), pp.
5144
5153
. 10.1016/j.eswa.2011.11.026
33.
Garver
,
M. S.
,
2003
, “
Best Practices in Identifying Customer-Driven Improvement Opportunities
,”
Ind. Mark. Manage.
,
32
(
6
), pp.
455
466
. 10.1016/S0019-8501(02)00238-9
34.
Myers
,
J. H.
, and
Alpert
,
M. I.
,
1977
, “
Semantic Confusion in Attitude Research: Salience Vs. Importance Vs. Determinance
,”
ACR North Am. Adv.
,
4
, pp.
106
110
.
35.
Deng
,
W.-J.
,
Chen
,
W.-C.
, and
Pei
,
W.
,
2008
, “
Back-Propagation Neural Network Based Importance–Performance Analysis for Determining Critical Service Attributes
,”
Exp. Syst. Appl.
,
34
(
2
), pp.
1115
1125
. 10.1016/j.eswa.2006.12.016
36.
Joung
,
J.
, and
Kim
,
H. M.
,
2020
, “
Automated Keyword Filtering in LDA for Identifying Product Attributes From Online Reviews
,”
ASME J. Mech. Des.
, pp.
1
10
. 10.1115/1.4048960
37.
Blei
,
D. M.
,
Ng
,
A. Y.
, and
Jordan
,
M. I.
,
2003
, “
Latent Dirichlet Allocation
,”
J. Mach. Learn. Res.
,
3
(
Jan.
), pp.
993
1022
.
38.
Mimno
,
D.
,
Wallach
,
H. M.
,
Talley
,
E.
,
Leenders
,
M.
, and
McCallum
,
A.
,
2011
, “
Optimizing Semantic Coherence in Topic Models
,”
Proceedings of the Conference on Empirical Methods in Natural Language Processing
,
Edinburgh, Scotland, UK
,
July 27–31
, pp.
262
272
.
39.
Miller
,
G. A.
,
1995
, “
Wordnet: A Lexical Database for English
,”
Commun. ACM
,
38
(
11
), pp.
39
41
. 10.1145/219717.219748
40.
Mikolov
,
T.
,
Sutskever
,
I.
,
Chen
,
K.
,
Corrado
,
G. S.
, and
Dean
,
J.
, “
Distributed representations of words and phrases and their compositionality
,”
Advances in Neural Information Processing Systems 26 (NIPS 2013)
,
Stateline, CA
,
Dec. 5–10
, pp.
3111
3119
.
41.
Rehurek
,
R.
, and
Sojka
,
P.
,
2010
, “
Software Framework for Topic Modelling With Large Corpora
,”
Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks
,
Valletta, Malta
,
May 22
.
42.
Ramage
,
D.
, and
Rosen
,
E.
,
2011
,
Stanford Topic Modeling Toolbox, http://nlp.stanford.edu/software/index.shtml
.
43.
Miller
,
G. F.
,
Todd
,
P. M.
, and
Hegde
,
S. U.
,
1989
, “
Designing Neural Networks Using Genetic Algorithms.
ICGA
,
Morgan Kaufmann, Palo Alto, CA
, Vol.
89
, pp.
379
384
.
44.
Lundberg
,
S. M.
, and
Lee
,
S. -I.
, “
A Unified Approach to Interpreting Model Predictions
,”
Advances in Neural Information Processing Systems 30 (NIPS 2017)
,
Long Beach, CA
,
Dec. 4–9
, pp.
4765
4774
.
45.
Delen
,
D.
,
Sharda
,
R.
, and
Kumar
,
P.
,
2007
, “
Movie Forecast Guru: A Web-Based DSS for Hollywood Managers
,”
Decis. Support Syst.
,
43
(
4
), pp.
1151
1170
. 10.1016/j.dss.2005.07.005
46.
Friedman
,
J.
,
Hastie
,
T.
, and
Tibshirani
,
R.
,
2001
,
The Elements of Statistical Learning
, Vol.
1
,
Springer Series in Statistics
,
New York
.
47.
McLachlan
,
G. J.
,
Do
,
K. -A.
, and
Ambroise
,
C.
,
2005
,
Analyzing Microarray Gene Expression Data
, Vol.
422
,
John Wiley & Sons
,
Hoboken, NJ
.
48.
Pedamonti
,
D.
,
2018
,
Comparison of Non-Linear Activation Functions for Deep Neural Networks on MNIST Classification Task. ArXiv180402763 Cs Stat, http://arxiv.org/abs/1804.02763
49.
Kingma
,
D. P.
, and
Ba
,
J.
,
2014
, “
Adam: A method for stochastic optimization
,”
3rd International Conference on Learning Representations (ICLR 2015)
,
San Diego, CA
,
May 7–9
.
50.
Arifovic
,
J.
, and
Gencay
,
R.
,
2001
, “
Using Genetic Algorithms to Select Architecture of a Feedforward Artificial Neural Network
,”
Phys. A: Stat. Mech. Appl.
,
289
(
3–4
), pp.
574
594
. 10.1016/S0378-4371(00)00479-9
51.
Davis
,
L.
,
1991
,
Handbook of Genetic Algorithms
,
Van Nostrand Reinhold
,
New York
.
52.
Batchelor
,
R.
, and
Dua
,
P.
,
1995
, “
Forecaster Diversity and the Benefits of Combining Forecasts
,”
Manage. Sci.
,
41
(
1
), pp.
68
75
. 10.1287/mnsc.41.1.68
53.
Azzopardi
,
E.
, and
Nash
,
R.
,
2013
, “
A Critical Evaluation of Importance–Performance Analysis
,”
Tourism Manage.
,
35
, pp.
222
233
. 10.1016/j.tourman.2012.07.007
54.
Eskildsen
,
J. K.
, and
Kristensen
,
K.
,
2006
, “
Enhancing Importance–Performance Analysis
,”
Int. J. Product. Perform. Manage.
,
55
(
1
), pp.
40
60
. 10.1108/17410400610635499
55.
Box
,
G. E.
, and
Meyer
,
R. D.
,
1986
, “
An Analysis for Unreplicated Fractional Factorials
,”
Technometrics
,
28
(
1
), pp.
11
18
. 10.1080/00401706.1986.10488093
56.
Bekkar
,
M.
,
Djemaa
,
H. K.
, and
Alitouche
,
T. A.
,
2013
, “
Evaluation Measures for Models Assessment Over Imbalanced Data Sets
,”
J. Inf. Eng. Appl.
,
3
(
10
), pp.
27
38
.
57.
Bi
,
J.-W.
,
Liu
,
Y.
,
Fan
,
Z.-P.
, and
Cambria
,
E.
,
2019
, “
Modelling Customer Satisfaction From Online Reviews Using Ensemble Neural Network and Effect-Based Kano Model
,”
Int. J. Product. Res.
,
57
(
22
), pp.
7068
7088
. 10.1080/00207543.2019.1574989
58.
Goodfellow
,
I.
,
Bengio
,
Y.
, and
Courville
,
A.
,
2016
,
Deep Learning
,
MIT Press
,
Cambridge, MA
.
You do not currently have access to this content.