RE: [sc] Fukushima, the Tsunami Hazard, and the Engineering Record



RE: [sc] Fukushima, the Tsunami Hazard, and the Engineering Record

From: Carl Sandom <carl_at_xxxxxx>
Date: Tue, 29 Mar 2011 13:41:58 +0100
Message-ID: <!&!AAAAAAAAAAAYAAAAAAAAAAXMX+u/Tb1BtCSTIqbutD/CgAAAEAAAAOfc5nR1CwhGqEzfwdktXSgBAAAAAA==@xxxxxx>
Nancy,

I don't share your view that: "...the argument for safety [within a safety
case] then goes on [from the hazard analysis] to assume that because the
hazard analysis did not find a problem, then the system is safe". 

The approach that I've seen used this side of the pond is to make a
defensible argument, with supporting evidence, that the residual risk (based
upon the hazard analysis) is ALARP. If there is a "great leap in logic and
an unjustified one" then the safety argument is not defensible; I'm sure
that does happen but it's not a general symptom of using a safety case
approach.

I genuinely don't see that a safety case approach is any different from the
MIL-STD 882D approach that requires the SSHAR to document the "reduction of
mishap risk to an acceptable level". 

I do however, agree with you on the topic of 'confirmation bias' or as I've
called it many times the self-fulfilling safety prophesy, that's the basic
problem with using a fundamentally inductive approach.

Best Regards
Carl
____________________
Carl Sandom 
iSys Integrity Limited
+44 (0) 7967 672560
Carl@xxxxxx
www.iSys-Integrity.com
____________________

-----Original Message-----
From: safety-critical-request@xxxxxx
[mailto:safety-critical-request@xxxxxx] On Behalf Of Nancy Leveson
Sent: 29 March 2011 12:41 PM
To: martyn@xxxxxx
Cc: safety-critical@xxxxxx
Subject: Re: [sc] Fukushima, the Tsunami Hazard, and the Engineering Record

"
Content-Type: multipart/mixed;
	boundary="000e0cd28d84b03b05049f9d8859
"
X-YCS-Spam-Score: 0.0 (/)

--000e0cd28d84b03b05049f9d8859

Content-Type: text/plain; charset=ISO-8859-1

Martyn, the problem is that they do not stop with the hazard analysis (which
may well be in the "safety case" but that the argument for safety then goes
on to assume that because the hazard analysis did not find a problem, then
the system is safe. This is a great leap in logic and an unjustified one.

In almost every safety case I have seen, there have been instances of
confirmation bias.

*Confirmation bias* is a tendency for people to favor information that
confirms their preconceptions or hypotheses regardless of whether the
information is true. People will focus on and interpret evidence in a way
that confirms the goal they have set for themselves. If the goal is to prove
the system is safe, they will focus on the evidence that shows it is safe
and create an argument for safety. If the goal is to show the system is
unsafe, the evidence used and the interpretation of available evidence will
be quite different.

The SafetyAssessment Report used in MIL-STD-882, for example, is very
different than, say, the safety case as defined by the U.K. HSE. The safety
assessment report does not argue that the system is safe, it only presents
the hazards identified and the controls and mitigations provided for those
hazards.


So the problems start not in the hazard analysis (if one is done), but the
argument *after* the hazard analysis.


Nancy


On Tue, Mar 29, 2011 at 5:49 AM, Martyn Thomas <
martyn@xxxxxx> wrote:

> On 29/03/2011 10:28, Nancy Leveson wrote:
>
>> On another topic, however, I would not, however, require a "safety case"
>> but
>> instead a hazard analysis. When doing a safety case, one looks for
>> evidence
>> that a system is safe. Such arguments are subject to confirmation
>> bias---as
>> I have seen in many safety cases that have been published as examples of
>> how
>> to do it. On the other hand, a hazard analysis looks for hazards and the
>> ways that the system is potentially dangerous and is therefore more
likely
>> to find them. The safest industries that I am familiar with require
hazard
>> analyses. Some of the more dangerous ones rely on safety cases. The
>> results
>> of the two are different.
>>
> Nancy
>
> Every safety case that I have ever seen contains a hazard analysis as an
> essential starting point, so there may be some differing uses of
terminology
> clouding this discussion.
>
> Regards
>
> Martyn
>
>


-- 
Dr. Nancy Leveson
Professor, Aeronautics and Astronautics
Professor, Engineering Systems
MIT

http://sunnyday.mit.edu

--000e0cd28d84b03b05049f9d8859

Content-Type: text/plain
X-Original-Content-Type: text/html; charset=ISO-8859-1


[The content of this part has been removed by the mailing list software]

--000e0cd28d84b03b05049f9d8859
--


__________ Information from ESET NOD32 Antivirus, version of virus signature
database 5996 (20110329) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com
Received on Tue 29 Mar 2011 - 13:42:06 BST