Re: Which tools or software are most critical to the integrity of the product? [safety]



Date view Thread view Subject view Author view

Peter B. Ladkin (ladkin(at)rvs.uni-bielefeld.de)
Sat, 24 Oct 1998 20:15:54 +0200


Martyn Thomas wrote: > But the original questioner, lost in the mists, asked a sensible > question and could well have been totally frustrated by the discussion > he provoked. I hope your rhetorical licence is still valid, Martyn, because I might have to ask you to present it for inspection. Paul asked more than one question, and those questions have different answers. First: paul.edwards(at)roverpte.demon.co.uk wrote: > In particular is there any existing statistical or experimental > evidence which suggests that some classes of software are more likely > to lead to errors in code than others [...] The answer to which, if measurement techniques can't be justified with precision, is no. But Paul got a positive answer none the less. Second: > The question is… is it possible to place any sort of risk ranking on > the above software items? [...] The second question depends on what one means by the art of the possible. Let's assume one is building a safety case, and wants to say that: one used COTS software of class X and COTS software of class Y; and that software of class X is riskier than software of class Y. Are there any instances of X and Y in the list he gave for which that would constitute acceptable reasoning in a safety case? The answer is yes. Let class Y be a frequently-used building tool (compiler or other code generator for software; or CATIA for real hardware) that has been used in other safety-critical applications, none of whom have gone catastrophic. Let class X be the code performing the concurrency control in your flight management system. Then such an argument, beefed up a little would be acceptable in many safety cases. But that doesn't mean that software of class X is in fact riskier than software of class Y. Just because the reasoning is acceptable in a safety case, it doesn't mean that it's correct. If one cannot evaluate the risk, one really has no idea how likely it is that a problem will occur (this is a tautology). And one indeed cannot, according to standard notions of risk. Suppose one is trying to build a case for a 1 in 10^9 failure rate. No math would currently support a case that your compiler is more reliable than your OS concurrency control, on this level. The reason one can try to make the case nevertheless is through (at)best practice', which can be thought of as a euphemism for piggy-backing on someone else's safety case. Which is the reason why (at)X is riskier than Y' assertions are acceptable in such cases. A question of my own: Is this sort of safety-case reasoning appropriate, even though invalid? Third: > [...] Are there any items which are more likely to > lead to errors in the product than others? Now, that's a question of what the world is like, rather than what we know or can justify. A matter of fact rather than epistomology. Paul got an answer: you can mostly trust your compiler (well, one or two of them) more than the code you write with it. Is this true? According to Darren, there are whole classes of errors in the product generated by a compiler that don't count as compiler bugs. That is, one can have a provably correct program (say), compile it, use it on a machine whose machine code is provably correct (say), and have your program fail, and still not have a (at)compiler bug'. Well, that makes me suspicious about what it's supposed to mean to trust my compiler. Note that the clarification of (at)compiler bug' you attribute to Brian does not help with this question, because that notion could agree with Darren's (let's say) and the same argument would apply. Finally, your question: > What usable advice would we, as a community, give to a practising > engineer who asks for it? Don't accept anecdotes. Define a hierarchy of levels of software in the product, from specification down to hardware, look at the specific tools you use to generate that hierarchy. See if anybody has built safety reasoning before around some of those tools. If they have, see whether it's valid, whether it applies to your case, and steal it if the answers are yes and yes. Else build your custom argument. In particular, do not accept or use the following reasoning: My program fulfils the specification (we proved it); My compiler's correct (Fred says he proved it); The definition of the object code semantics is correct (the manufacturer proved it) ---------- Therefore my computer fulfils the specification. The reason is that it is invalid, under the interpretation that to say the compiler is correct is to say that there are no compiler bugs in the sense of Brian or Darren. Now, if there are no errors in your program, the compiler has no errors, and there aren't any errors in the design of the machine code, you would imagine that concatenating error-free operations from an error-free source should yield an error-free result. But that is not necessarily the case, under this interpretation of (at)error-free compilation'. So if you want to use the reasoning that error-free source and the concatenation of error-free operations yield an error-free result (and reasoning of this sort is essential in making safety cases, if you want to finish before the universe burns up), then you cannot use Brian or Darren's notion of error-free compiler. Now, if that's not usable advice, I don't know what is. Cheers, PBL ---------------------- --------------------- Peter Ladkin, Professor fuer Rechnernetze und Verteilte Systeme ladkin(at)rvs.uni-bielefeld.de http://www.rvs.uni-bielefeld.de Snailmail: Universitaet Bielefeld, Technische Fakultaet, Postfach 10 01 31, D-33501 Bielefeld, Germany Tel: +49 (0)521 106-5326/5325/2952, Fax (new): +49 (0)521 106-6440 ---------------------- ---------------------


Date view Thread view Subject view Author view