Abstract

Risk analyses require proper consideration and quantification of the interaction between humans, organization, and technology in high-hazard industries. Quantitative human reliability analysis approaches require the estimation of human error probabilities (HEPs), often obtained from human performance data on different tasks in specific contexts (also known as performance shaping factors (PSFs)). Data on human errors are often collected from simulated scenarios, near-misses report systems, and experts with operational knowledge. However, these techniques usually miss the realistic context where human errors occur. The present research proposes a realistic and innovative approach for estimating HEPs using data from major accident investigation reports. The approach is based on Bayesian Networks used to model the relationship between performance shaping factors and human errors. The proposed methodology allows minimizing the expert judgment of HEPs, by using a strategy that is able to accommodate the possibility of having no information to represent some conditional dependencies within some variables. Therefore, the approach increases the transparency about the uncertainties of the human error probability estimations. The approach also allows identifying the most influential performance shaping factors, supporting assessors to recommend improvements or extra controls in risk assessments. Formal verification and validation processes are also presented.

References

1.
Ramos
,
M. A.
,
Utne
,
I. B.
,
Vinnem
,
J. E.
, and
Mosleh
,
A.
,
2018
, “
Accounting for Human Failure in Autonomous Ship Operations
,”
Safety and Reliability–Safe Societies in a Changing World, Proceedings of ESREL 2018
, Trondheim, Norway, June 17–21, pp.
355
363
.
2.
Zio
,
E.
,
2018
, “
The Future of Risk Assessment
,”
Reliab. Eng. Syst. Saf.
,
177
, pp.
176
190
.10.1016/j.ress.2018.04.020
3.
Henderson
,
J.
, and
Embrey
,
D.
,
2012
, “
Guidance on Quantified Human Reliability Analysis
,” Energy Institute, London.
4.
Kirwan
,
B.
, and
Ainsworth
,
L. K.
,
1992
,
A Guide to Task Analysis: The Task Analysis Working Group
,
CRC Press
,
Boca Raton, FL
.
5.
Kirwan
,
B.
,
1997
, “
Validation of Human Reliability Assessment Techniques—Part 1: Validation Issues
,”
Saf. Sci.
,
27
(
1
), pp.
25
41
.10.1016/S0925-7535(97)00049-0
6.
Swain
,
A. D.
, and
Guttmann
,
H. E.
,
1983
, “
Handbook of Human-Reliability Analysis With Emphasis on Nuclear Power Plant Application
s,” Sandia National Labs, Washington, DC, Report. No. NUREG/CR–1278.
7.
Gertman
,
D. I.
,
Blackman
,
H. S.
,
Byers
,
J.
,
Haney
,
L.
,
Smith
,
C.
, and
Marble
,
J.
,
2005
, “
The SPAR-H Method
,” U.S. Nuclear Regulatory Commission, Washington, DC, Report No. NUREG/CR-6883.
8.
Williams
,
J. C.
,
1988
, “
A Data-Based Method for Assessing and Reducing Human Error to improve operational performance
,”
Proceedings of IEEE Forth Conference on Human factors in Power Plants
Monterey, CA, June 6–9, pp.
436
450
.
9.
Hollnagel
,
E.
,
1998
,
Cognitive Reliability and Error Analysis Method (CREAM)
,
Elsevier
,
Oxford, UK
.
10.
Cooper
,
S. E.
,
Ramey-Smith
,
A. M.
,
Wreathall
,
J.
, and
Parry
,
G. W.
,
1996
, “
A Technique for Human Error Analysis (ATHEANA)
,” Nuclear Regulatory Commission, Washington, DC, Report No. NUREG/CR-6350; BNL-NUREG-52467.
11.
Bye
,
A.
,
2018
, “
Informing HRA by Empirical Data, Halden Reactor Project Lessons Learned and Future Direction
,”
Probabilistic Safety Assessment and Management (PSAM 14 2018)
, Los Angeles, CA, Sept.
16
21
12.
Kirwan
,
B.
,
1997
, “
Validation of Human Reliability Assessment Techniques—Part 2: Validation Results
,”
Saf. Sci.
,
27
(
1
), pp.
43
75
.10.1016/S0925-7535(97)00050-7
13.
Zio
,
E.
,
2009
, “
Reliability Engineering: Old Problems and New Challenges
,”
Reliab. Eng. Syst. Saf.
,
94
(
2
), pp.
125
141
.10.1016/j.ress.2008.06.002
14.
Bell
,
J.
, and
Holroyd
,
J.
,
2009
, “Review of Human Reliability Assessment Methods,” Vol.
78
,
Buxton, UK
:
Health and Safety Executive
.
15.
Mosleh
,
A.
,
Bier
,
V. M.
, and
Apostolakis
,
G.
,
1988
, “
A Critique of Current Practice for the Use of Expert Opinions in Probabilistic Risk Assessment
,”
Reliab. Eng. Syst. Saf.
,
20
(
1
), pp.
63
85
.10.1016/0951-8320(88)90006-3
16.
Mkrtchyan
,
L.
,
Podofillini
,
L.
, and
Dang
,
V. N.
,
2016
, “
Methods for Building Conditional Probability Tables of Bayesian Belief Networks From Limited Judgment: An Evaluation for Human Reliability Application
,”
Reliab. Eng. Syst. Saf.
,
151
, pp.
93
112
.10.1016/j.ress.2016.01.004
17.
Shirazi
,
C. H.
,
2009
, “
Data-Informed Calibration and Aggregation of Expert Judgment in a Bayesian Framework
,” Ph.D. dissertation, Digital Repository at the University of Maryland, College Park, MD.
18.
Lin
,
S.-W.
, and
Bier
,
V. M.
,
2008
, “
A Study of Expert Overconfidence
,”
Reliab. Eng. Syst. Saf.
,
93
(
5
), pp.
711
721
.10.1016/j.ress.2007.03.014
19.
Laumann
,
K.
,
Blackman
,
H. S.
, and
Rasmussen
,
M.
,
2018
, “
Challenges With Data for Human Reliability Analysis
,”
Safety and Reliability–Safe Societies in a Changing World, Proceedings of ESREL 2018
, Trondheim, Norway, June 17–21, pp.
315
321
.
20.
Chang
,
Y. J.
,
Bley
,
D.
,
Criscione
,
L.
,
Kirwan
,
B.
,
Mosleh
,
A.
,
Madary
,
T.
,
Nowell
,
R.
,
Richards
,
R.
,
Roth
,
E. M.
,
Sieben
,
S.
, and
Zoulis
,
A.
,
2014
, “
The SACADA Database for Human Reliability and Human Performance
,”
Reliab. Eng. Syst. Saf.
,
125
, pp.
117
133
.10.1016/j.ress.2013.07.014
21.
Lois
,
E.
,
2009
,
International HRA Empirical Study–Phase 1 Report: Description of Overall Approach and Pilot Phase Results From Comparing HRA Methods to Similar Performance Data
,
Office of Nuclear Regulatory Research, US Nuclear Regulatory Commission
,
Washington, DC
.
22.
Kim
,
Y.
,
Park
,
J.
, and
Jung
,
W.
,
2017
, “
A Classification Scheme of Erroneous Behaviors for Human Error Probability Estimations Based on Simulator Data
,”
Reliab. Eng. Syst. Saf.
,
163
, pp.
1
13
.10.1016/j.ress.2017.01.022
23.
Park
,
J.
, and
Jung
,
W.
,
2007
, “
OPERA—A Human Performance Database Under Simulated Emergencies of Nuclear Power Plants
,”
Reliab. Eng. Syst. Saf.
,
92
(
4
), pp.
503
519
.10.1016/j.ress.2006.01.007
24.
Gibson
,
W. H.
, and
Megaw
,
T. D.
,
1999
,
The Implementation of CORE-DATA, a Computerised Human Error Probability Database
,
HSE Books
,
Norwich, UK
.
25.
Park
,
J.
,
Kim
,
Y.
, and
Jung
,
W.
,
2017
, “
Use of a Big Data Mining Technique to Extract Relative Importance of Performance Shaping Factors From Event Investigation Reports
,”
International Conference on Applied Human Factors and Ergonomics
, Cham, Switzerland, July 17–21, pp.
230
238
.
26.
Preischl
,
W.
, and
Hellmich
,
M.
,
2016
, “
Human Error Probabilities From Operational Experience of German Nuclear Power Plants—Part II
,”
Reliab. Eng. Syst. Saf.
,
148
, pp.
44
56
.10.1016/j.ress.2015.11.011
27.
Preischl
,
W.
, and
Hellmich
,
M.
,
2013
, “
Human Error Probabilities From Operational Experience of German Nuclear Power Plants
,”
Reliab. Eng. Syst. Saf.
,
109
, pp.
150
159
.10.1016/j.ress.2012.08.004
28.
Kletz
,
T.
,
2011
, “
Some Common Errors in Accident Investigations
,”
Saf. Reliab.
,
31
(
1
), pp.
4
13
.10.1080/09617353.2011.11690926
29.
Moura
,
R.
,
Beer
,
M.
,
Patelli
,
E.
,
Lewis
,
J.
, and
Knoll
,
F.
,
2016
, “
Learning From Major Accidents to Improve System Design
,”
Saf. Sci.
,
84
, pp.
37
45
.10.1016/j.ssci.2015.11.022
30.
Kyriakidis
,
M.
,
Majumdar
,
A.
, and
Ochieng
,
W. Y.
,
2015
, “
Data Based Framework to Identify the Most Significant Performance Shaping Factors in Railway Operations
,”
Saf. Sci.
,
78
, pp.
60
76
.10.1016/j.ssci.2015.04.010
31.
API, ANSI,
2010
, “
API Recommended Practice 754
,”
Process Safety Performance Indicators for the Refining and Petrochemical Industries
, 1st ed.,
American Petroleum Institute
,
Washington, DC
.
32.
Tolo
,
S.
,
Patelli
,
E.
, and
Beer
,
M.
,
2018
, “
An Open Toolbox for the Reduction, Inference Computation and Sensitivity Analysis of Credal Networks
,”
Adv. Eng. Software
,
115
, pp.
126
148
.10.1016/j.advengsoft.2017.09.003
33.
Mkrtchyan
,
L.
,
Podofillini
,
L.
, and
Dang
,
V. N.
,
2015
, “
Bayesian Belief Networks for Human Reliability Analysis: A Review of Applications and Gaps
,”
Reliab. Eng. Syst. Saf.
,
139
, pp.
1
16
.10.1016/j.ress.2015.02.006
34.
Tolo
,
S.
,
Patelli
,
E.
, and
Beer
,
M.
,
2015
, “
Enhanced Bayesian Network Approach to Sea Wave Overtopping Hazard Quantification
,”
25th European Safety and Reliability Conference
, ESREL, Zurich, Switzerland, Sept. 7–10, pp.
1983
1990.
35.
Chen
,
S. H.
, and
Pollino
,
C. A.
,
2012
, “
Good Practice in Bayesian Network Modelling
,”
Environ. Modell. Software
,
37
, pp.
134
145
.10.1016/j.envsoft.2012.03.012
36.
Tolo
,
S.
,
Patelli
,
E.
, and
Beer
,
M.
,
2017
, “
Risk Assessment of Spent Nuclear Fuel Facilities Considering Climate Change
,”
ASCE-ASME J. Risk Uncertainty Eng. Syst., Part A: Civ. Eng.
,
3
(
2
), p.
G4016003
.10.1061/AJRUA6.0000874
37.
Groth
,
K. M.
, and
Mosleh
,
A.
,
2012
, “
Deriving Causal Bayesian Networks From Human Reliability Analysis Data: A Methodology and Example Model
,”
Proc. Inst. Mech. Eng., Part O
,
226
(
4
), pp.
361
379
.10.1177/1748006X11428107
38.
Yang
,
Z. L.
,
Bonsall
,
S.
,
Wall
,
A.
,
Wang
,
J.
, and
Usman
,
M.
,
2013
, “
A Modified CREAM to Human Reliability Quantification in Marine Engineering
,”
Ocean Eng.
,
58
, pp.
293
303
.10.1016/j.oceaneng.2012.11.003
39.
Sundaramurthi
,
R.
, and
Smidts
,
C.
,
2013
, “
Human Reliability Modeling for the Next Generation System Code
,”
Ann. Nucl. Energy
,
52
, pp.
137
156
.10.1016/j.anucene.2012.07.027
40.
Nielsen
,
T. D.
, and
Jensen
,
F. V.
,
2009
,
Bayesian Networks and Decision Graphs
,
Springer
,
New York
.
41.
Fenton
,
N.
, and
Neil
,
M.
,
2012
,
Risk Assessment and Decision Analysis With Bayesian Networks
,
CRC Press
,
Boca Raton, FL
.
42.
Jentsch
,
F. G.
,
1993
, “
Problems of Systematic Safety Assessments: Lessons Learned From Aircraft Accidents
,”
Verification and Validation of Complex Systems: Human Factors Issues
,
Springer
,
Berlin
, pp.
251
259
.
43.
Trucco
,
P.
,
Cagno
,
E.
,
Ruggeri
,
F.
, and
Grande
,
O.
,
2008
, “
A Bayesian Belief Network Modelling of Organisational Factors in Risk Analysis: A Case Study in Maritime Transportation
,”
Reliab. Eng. Syst. Saf.
,
93
(
6
), pp.
845
856
.10.1016/j.ress.2007.03.035
44.
Samaniego
,
F. J.
,
1985
, “
On Closure of the IFR Class Under Formation of Coherent Systems
,”
IEEE Trans. Reliab.
,
34
(
1
), pp.
69
72
.10.1109/TR.1985.5221935
45.
Barlow
,
R. E.
, and
Wu
,
A. S.
,
1978
, “
Coherent Systems With Multi-State Components
,”
Math. Oper. Res.
,
3
(
4
), pp.
275
281
.10.1287/moor.3.4.275
46.
Kirwan
,
B.
,
Kennedy
,
R.
,
Taylor-Adams
,
S.
, and
Lambert
,
B.
,
1997
, “
The Validation of Three Human Reliability Quantification Techniques—THERP, HEART and JHEDI—Part II: Results of Validation Exercise
,”
Appl. Ergon.
,
28
(
1
), pp.
17
25
.10.1016/S0003-6870(96)00045-2
47.
Pirie
,
W.
,
1988
, “
Spearman Rank Correlation Coefficient
,”
Encyclopedia of Statistical Sciences
,
Wiley
,
New York
.
48.
Abdi
,
H.
,
2007
, “
The Kendall Rank Correlation Coefficient
,”
Encyclopedia of Measurement and Statistics
,
N.
Salkind
, ed.,
Sage Publications
,
Thousand Oaks, CA
, pp.
508
510
.
49.
Morais
,
C.
,
Moura
,
R.
,
Beer
,
M.
, and
Patelli
,
E.
,
2018
, “
Attempt to Predict Human Error Probability in Different Industry Sectors Using Data From Major Accidents and Bayesian Networks
,”
Probabilistic Safety Assessment and Management (PSAM 14)
, Los Angeles, CA, Sept. 16–21.
50.
Moura
,
R.
,
Beer
,
M.
,
Patelli
,
E.
,
Lewis
,
J.
, and
Knoll
,
F.
,
2017
, “
Learning From Accidents: Interactions Between Human Factors, Technology and Organisations as a Central Element to Validate Risk Studies
,”
Saf. Sci.
,
99
, pp.
196
214
.10.1016/j.ssci.2017.05.001
51.
Authority, Civil Aviation,
2016
, “CAP 737 Flight-Crew Human Factors Handbook,”
Civil Aviation Authority
,
London
.
52.
BayesFusion, LLC
,
2017
, “
GeNIe Modeler
,” BayesFusion, LLC, Pittsburgh, PA, accessed Nov. 30, 2017, http://www.bayesfusion.com
53.
Patelli
,
E.
,
Tolo
,
S.
,
George-Williams
,
H.
,
Sadeghi
,
J.
,
Rocchetta
,
R.
,
de Angelis
,
M.
, and
Broggi
,
M.
,
2018
, “
OpenCossan 2.0: An Efficient Computational Toolbox for Risk, Reliability and Resilience Analysis
,”
Joint ICVRAM ISUMA UNCERTAINTIES Conference
, Florianopolis, SC, Brazil, Apr. 8–11, 2018, http://icvramisuma2018.org/cd/web/PDF/ICVRAMISUMA2018-0022.PDF
54.
Murphy
,
K.
,
2007
, “
Software for Graphical Models: A Review
,”
Int. Soc. Bayesian Anal. Bull.
,
14
(
4
), pp.
13
15
.https://www.cs.ubc.ca/~murphyk/Software/bnsw.pdf
You do not currently have access to this content.