Re: [sc] Testing



Re: [sc] Testing

From: David Haworth <dh_at_xxxxxx>
Date: Mon, 27 Jun 2011 15:43:50 +0200
Message-ID: <20110627134350.GA5170@xxxxxx>
Peter,

you might want to try looking for texts about adequacy of testing
or test coverage and how to measure it.
There are quite a few papers of varying quality.

Basically the idea is to try to measure how much of your system is
actually being tested, and to design your test cases to increase
the proportion that is tested.

How you design your test cases isn't so important. You can start
with requirement based testing (or muddle-based, as someone has
joked). If you're lucky you might reach 50% coverage using some
of the more generous measurement methods. After that, you can refine your
tests, add more test cases, whatever, to try to achieve the
elusive 100% coverage (whatever that means). You can also
use the coverage results (with extreme caution) to weed out
unnecessary code.

Just my 3 ha'porth

Dave


On 2011-06-27 14:33:00 +0200, Peter Bernard Ladkin wrote:
> On 6/27/11 1:15 PM, Nancy Leveson wrote:
> >There is an enormous amount of literature on testing.
> 
> Yes, I know. 99% of it is reinventing some part of the wheel. (Like
> a similar proportion of the literature on specification - the
> difference being that there are a few good texts on specification,
> from any one of which people can learn state-of-the-practice
> techniques.)
> 
> We all think testing is key. But if there aren't a few key texts,
> then hardly anyone (let alone a student) is going to learn how to
> test SW. Because very few practitioners, and no students, are going
> to wade through 30 years of conference proceedings and IEEE TSE
> papers to try to get an idea, especially with all that repetition.
> 
> I am not content with this situation. I suspect it is a sign of how
> arbitrary the activity known as testing still is. That's certainly
> the impression I got twenty years ago.
> 
> In stark contrast, I can pick up one book, Intro to Flight by
> Anderson, which will tell me most of what I want to know for intro
> aerodynamics, sufficient that I can talk to, even argue with,
> aerodynamicists. Or Fundamentals of Flight, by Shevell.
> 
> > you don't really say what you want to know.
> 
> Indeed I don't. But I could ask the same question in aerodynamics
> and I would be reasonably referred to at most a half-dozen books,
> any couple of which would suffice. Or specification, for that
> matter. Say, Somerville for an overview, Barnes for state of the
> practice, Lamport for state of the theory, and Holzmann for checking
> (actually, one could consider checking as part of testing, so maybe
> that is part of "testing" covered).
> 
> >If what you want to know is about testing for safety, then that can't be done.
> 
> I want to know about methods of determining through experiment that
> a system is fit for purpose. Safety is, I hope you agree, part of
> being fit for purpose.
> 
> But thanks for giving me the chance to nit-pick :-) You may be right
> in general, if by "can't be done" you mean that full coverage in
> general cannot be guaranteed. But of course *some*
> demonstrably-sufficient testing for safety can be done. Boeing's
> video of the 777 wing-strength destruction test is on YouTube. Wings
> don't break off commercial airplanes for many decades now, and that
> test is part of the reason for it.
> 
> PBL
> 
> Peter Bernard Ladkin, Professor of Computer Networks and Distributed Systems,
> Faculty of Technology, University of Bielefeld, 33594 Bielefeld, Germany
> Tel+msg +49 (0)521 880 7319  www.rvs.uni-bielefeld.de
> 
> 
> 
> 
> 

-- 
David Haworth B.Sc.(Hons.), OS Kernel Developer         Tel: +49 9131 7701-6154
david.haworth@xxxxxx                            Fax: +49 9131 7701-6333
Elektrobit Automotive GmbH           Am Wolfsmantel 46, 91058 Erlangen, Germany
Geschäftsführer: Otto Fößel, Gregor Zink             Amtsgericht Fürth HRB 4886


----------------------------------------------------------------
Please note: This e-mail may contain confidential information
intended solely for the addressee. If you have received this
e-mail in error, please do not disclose it to anyone, notify
the sender promptly, and delete the message from your system.
Thank you.
Received on Mon 27 Jun 2011 - 14:43:57 BST