Planning and Executing Credible Experiments. Robert J. Moffat

Чтение книги онлайн.

2 The Nature of Experimental Work

It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong. ¹

      Richard Feynman

      Science, medicine, and engineering depend heavily on experiments. Business does too, where product design and marketing experiments must account for the added complexity of human nature and taste. In every case in any field, an experiment must be credible or it wastes time and resources.

      Engineering problems are solved using three tools: insight, analysis, and experiment. Usually all three are brought to bear on any given problem; each complementing the others. A sudden insight or inspiration may be enough to suggest what must be done next, a new analysis or a new experiment, but to actually get an answer takes work.

2.1 Tested Guide of Strategy and Tactics

      This book is about the strategy and tactics of experimental work – the techniques whereby one plans and executes an experiment which insight or inspiration has suggested. Our strategies and tactics are a superset which includes the concepts of design of experiments (DoE) and extends beyond.

      Have you heard how designed experiments improve results while reducing the amount of data needed? In our experience, few engineering students have learned DoE in their undergraduate labs, so we are including DoE concepts. Many DoE techniques were pioneered by Ronald Fisher (1890–1962) for agricultural experiments in the early 1900s. As the advantages of DoE become known, we all benefit.

This book offers a tested guide so that you can design effective experiments. We teach the techniques by which your insight can result in a high‐impact experiment. You know of a need that requires a credible solution. We hope to launch from your creativity and reinforce it. Could something with absolutely no creativity (e.g. a computer) be successful in either analysis or experiment? It seems not. As humans, our needs often inspire our inventiveness and creativity. Creativity is boosted by seeing successful and failed solutions. The art and science of engineering advances through decisions – decisions based on experience gained through experimental results.

      The purpose of an experiment is to get provably accurate, relevant, and credible data – data that are reliable enough to serve as the basis for answering questions and making decisions. Most experiments arise out of questions which must be answered, such as “Does this device behave the way it is supposed to?” or “How much cooling do we need?” or “What is the relationship between X and Y?” or “Which of these designs performs best?” In many cases, such questions lead to experiments, and the data from those experiments lead to decisions.

      The three key words are “accurate,” “relevant,” and “credible.” One point to remember is that when the work is all done, it will be your signature on the report – and that report will be around for a long time! It is not enough for you to be personally convinced that your results are accurate. You must be able to establish the credibility of the work “beyond a reasonable doubt” or at least well enough so that a prudent engineer would be willing to accept your results as valid when you are no longer around to answer his or her questions.

      The process of establishing credibility begins with the experiment plan and only ends when the results have been presented in such form that they can easily be understood. The experiment plan must make provision for the appropriate checks and balances: baseline checking, repeatability tests, and the other diagnostics which guard against error. Showing agreement with a baseline dataset is one of the most convincing pieces of evidence that can be offered to support the credibility of an experiment. The data presentation must include a quantitative description of the residual uncertainty in the results.

      In large measure, that is what this book is all about: designing and executing experiments for credibility. Before we get to the main issue, however, there are some key points to consider about experiments in general.

2.2 What Can Be Measured and What Cannot?

      A quantitative property can be measured. A categorical property can be recorded but not measured.

      The act of measurement is an ordering in a scalar system involving a “less than, equal to, or greater than” test. We assign a value to the measurand by comparing it with a standard interval and counting the number of intervals equal to the measurand. The only attributes of any system that can be measured are those which can be put into one‐to‐one correspondence with points on the real number line. Since only the real number system has the order property, only real numbers (scalars) can be measured.

      2.2.1 Examples Not Measurable

      Only the simplest attributes of systems can be directly measured – the rest are inferred.

For example, tensors, complex numbers,² and vectors cannot be ordered and, therefore, cannot be “measured.” These can be described by ordered sets of scalars (dyads, triads, and two or three‐dimensional arrays of scalars), but those are simply the components of the entity, not the entity itself.

      Many times people seem to recognize this problem but don't know how to describe their malaise. They want information but find themselves talking about measurements. For example, a former governor of СКАЧАТЬ