 | | From: | jrefactors at hotmail.com | | Subject: | Can Every Software Defects Be Reproduced? | | Date: | 11 Jan 2005 21:55:10 -0800 |
|
|
 | I posted similar question regarding the software defects reproduction.
Actually my intended question is: Can Every Software Defects Be
Reproduced? Yes or No answer? I don't see that happens in every case in
real development environment.
Please advise and discuss this argument. thanks!!
|
|
| | From: | Alexei Polkhanov | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 13:29:19 -0800 |
|
|
| jrefactors@hotmail.com wrote:
> I posted similar question regarding the software defects reproduction.
> Actually my intended question is: Can Every Software Defects Be
> Reproduced? Yes or No answer? I don't see that happens in every case in
> real development environment.
> Please advise and discuss this argument. thanks!!
>
Perhaps there is a term confusion here. "Bug" can be reproduced and it
is an appearance of DEFECT not defect itself. The defect which appear in
one or another form to external user of system usually called "Bug". By
removing single or more defects you can resolve several bugs. Defect
exists by itself and only its appearance can or cannot be reproduced.
Say for example you have 2 defects and side effects of second defect
hiding the first defect. This situation usually happen when someone is
trying to fix problem without knowing exactly what is causing it. My
favorite example given in Steve Mc Connels "Code complete" when
programmer trying to fix loop by changing exit condition by +1 or -1 and
rerunning it.
Theoretically every defect can be discovered by trying all possible user
inputs in couple million years, but it will be unfeasible. So speaking
further on this topic we will end up defining software quality metrics
such as mean time to failure etc.
Cheers,
---
Alexei Polkhanov
Sr. Consultant/Software Systems Analyst
Tel: (604) 719-2515
E-mail: usenet@monteaureus.com
http://www.monteaureus.com/
|
|
| | From: | Phlip | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 15:28:28 GMT |
|
|
| jrefactors wrote:
> I posted similar question regarding the software defects reproduction.
> Actually my intended question is: Can Every Software Defects Be
> Reproduced? Yes or No answer? I don't see that happens in every case in
> real development environment.
The absolute answer is No, for the same reason as Turing's Halting Problem.
You can't prove software doesn't have a bug, because even if you apply a
math proof to your source code (an NP-hard problem), somewhere out in space
there is a cosmic ray with your CPU's serial number written on it.
In practice, given a hard-to-reproduce bug, your packages and modules should
all have unit tests, should all be decoupled, and should all have a kind of
independent user interface in acceptance tests. So given a strange bug
report, you should be able to seek reproduction via divide-and-conquer.
Estimate the intermediate values for the bug, and test for these in each
package. If a package raises alarms, repeat in each module, until you find
problems that could have caused the bug.
Such a development strategy also makes the odds of all defects go down, so
the non-reproducibles go down too.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
|
|
| | From: | Peter Amey | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 16:32:09 +0000 |
|
|
|
Phlip wrote:
> jrefactors wrote:
>
>
[snip]
>
> In practice, given a hard-to-reproduce bug, your packages and modules should
> all have unit tests, should all be decoupled, and should all have a kind of
> independent user interface in acceptance tests. So given a strange bug
> report, you should be able to seek reproduction via divide-and-conquer.
> Estimate the intermediate values for the bug, and test for these in each
> package. If a package raises alarms, repeat in each module, until you find
> problems that could have caused the bug.
>
It's nothing like as easy as that. Consider the case where a coding
error allows memory to be overwritten (perhaps by writing oustide an
array's bounds). The effect of this may not be visible until some time
later, when something else runs that depends on the corrupted memory.
Whether this can be detected by unit tests is highly problematic - the
unit doing the corrupting may pass its unit test alright - the unit that
gets hurt will run ok if the unit causing the corruption hasn't been run
first (with the special data that triggers the overwriting).
Similar effects occur with the use of uninitialized variables -
returning a random value might give the right results until something
else (such as running tests in a different order) causes the random
value to change. This is especially problematic with Booleans in the
languages where 0 is interpreted as false and anything else as true. If
you return a random byte (which is more likely to be non zero than zero)
and if true is the right answer then your tests will pass, by luck.
To have a real chance of making bugs reproducible you need to invest a
lot of effort in making the computing environment deterministic by
eliminating such things as uninitialized variables and potential buffer
overflows (that is one of reasons we put so much effort into eliminating
all of these things from SPARK). That only leaves you with logic errors
to find and, by and large, they should be reproducible.
[snip]
Peter
|
|
| | From: | Phlip | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Thu, 13 Jan 2005 05:27:59 GMT |
|
|
| Peter Amey wrote:
> Phlip wrote:
> > In practice, given a hard-to-reproduce bug, your packages and modules
should
> > all have unit tests, should all be decoupled, and should all have a kind
of
> > independent user interface in acceptance tests. So given a strange bug
> > report, you should be able to seek reproduction via divide-and-conquer.
> > Estimate the intermediate values for the bug, and test for these in each
> > package. If a package raises alarms, repeat in each module, until you
find
> > problems that could have caused the bug.
> >
>
> It's nothing like as easy as that.
Sorry. Thought I was on the games dev group there...
> Consider the case where a coding
> error allows memory to be overwritten (perhaps by writing oustide an
> array's bounds). The effect of this may not be visible until some time
> later, when something else runs that depends on the corrupted memory.
> Whether this can be detected by unit tests is highly problematic - the
> unit doing the corrupting may pass its unit test alright - the unit that
> gets hurt will run ok if the unit causing the corruption hasn't been run
> first (with the special data that triggers the overwriting).
>
> Similar effects occur with the use of uninitialized variables -
> returning a random value might give the right results until something
> else (such as running tests in a different order) causes the random
> value to change. This is especially problematic with Booleans in the
> languages where 0 is interpreted as false and anything else as true. If
> you return a random byte (which is more likely to be non zero than zero)
> and if true is the right answer then your tests will pass, by luck.
>
> To have a real chance of making bugs reproducible you need to invest a
> lot of effort in making the computing environment deterministic by
> eliminating such things as uninitialized variables and potential buffer
> overflows (that is one of reasons we put so much effort into eliminating
> all of these things from SPARK). That only leaves you with logic errors
> to find and, by and large, they should be reproducible.
Greetings from the opposite side of the industry. In game development, you
often pass the system clock time into your physics engine and your random
number generator. This leads to highly non-reproducible situations.
Now evaluate my paragraph again. By applying my techniques, or their
equivalents, one can push determinism in games as far as, for example,
rewind and replay features that run the game backwards, without storing all
its frames or something miserable like that.
Still a long way from your level of determinism.
--
Phlip
http://industrialxp.org/community/bin/view/Main/TestFirstUserInterfaces
|
|
| | From: | David Lightstone | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 13:34:55 GMT |
|
|
|
wrote in message
news:1105509310.383841.173590@z14g2000cwz.googlegroups.com...
> I posted similar question regarding the software defects reproduction.
> Actually my intended question is: Can Every Software Defects Be
> Reproduced? Yes or No answer? I don't see that happens in every case in
> real development environment.
> Please advise and discuss this argument. thanks!!
Every software defect can be reproduced. Unfortunately it may take several
lifetimes .
How many times in your lifetime will all the planets be aligned into that
simple Euclidean figure called a line?
>
|
|
| | From: | Wayne Woodruff | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 06:17:34 -0500 |
|
|
| On 11 Jan 2005 21:55:10 -0800, jrefactors@hotmail.com wrote:
>I posted similar question regarding the software defects reproduction.
>Actually my intended question is: Can Every Software Defects Be
>Reproduced? Yes or No answer? I don't see that happens in every case in
>real development environment.
>Please advise and discuss this argument. thanks!!
No.
Uninitialized variables/pointers, or stray pointers can cause random
crashes.
There are mant others.
Wayne Woodruff
http://www.jtan.com/~wayne
|
|
| | From: | Thomas Lorenz | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | 12 Jan 2005 08:29:36 GMT |
|
|
| jrefactors@hotmail.com wrote in
news:1105509310.383841.173590@z14g2000cwz.googlegroups.com:
> I posted similar question regarding the software defects reproduction.
> Actually my intended question is: Can Every Software Defects Be
> Reproduced? Yes or No answer? I don't see that happens in every case in
> real development environment.
> Please advise and discuss this argument. thanks!!
>
>
Defects in a multithreaded process can be nearly impossible to reproduce.
|
|
| | From: | Jonathan Allan | | Subject: | Re: Can Every Software Defects Be Reproduced? | | Date: | Wed, 12 Jan 2005 07:02:10 -0600 |
|
|
| jrefactors@hotmail.com wrote:
> I posted similar question regarding the software defects reproduction.
> Actually my intended question is: Can Every Software Defects Be
> Reproduced? Yes or No answer? I don't see that happens in every case in
> real development environment.
> Please advise and discuss this argument. thanks!!
These machines are deterministic, so yes, all defects can be reproduced.
The environment the machine works in is NOT deterministic and in many
cases cannot be reproduced reliably. Because the environment affects
the machine, many defects cannot be reproduced.
--
Jonathan Allan
Neither Mayo Clinic nor I speak for each other unless we explicitly
say so. You should assume I am speaking only for myself.
Please remove the antispam ".6809" to reply direct to me. Thanks!
|
|
