Scientific Theories must be Elegant.
Since there are always infinitely many different hypotheses which fit any set of data, there must be some prior beliefs which we use to decide between them. Any hypothesis which has an excessive number of entities or postulates is unappealing, and gives rise to the suspicion that it works because of special pleading or force-fitting the data rather than because it has any deep connection with Nature. So all else being equal, scientists prefer hypotheses which are simple, uniform, common-sensical and aesthetically pleasing.
At least part of this requirement is captured in the principle known as Occam's razor, which in the original form proposed by Occam translates to “Entities are not to be multiplied without necessity”. Of course, one may be forced to postulate complexities if the data rules out any simpler hypothesis, but even here one must pick among the simplest of an infinite number of possible explanations for the same data.
This criterion of elegance is informed by previous scientific work as well as by a priori considerations. It also varies from field to field: a particle physicist should be much more reluctant to postulate a new force of nature than a cellular biologist is to postulate a new kind of organelle.
Because many important scientific theories have greatly defied prior expectations, it is best not to turn these a priori expectations into hard and fast rules which would prevent too many hypotheses from being considered altogether. Instead, scientists mainly use intuition and rules-of-thumb to judge which theories are worth considering.
There are many famous cases where the elegance of a new theory was used to predict confidently the results of an experiment. Einstein once quipped about Planck that
...he did not really understand physics, during the eclipse of 1919 he stayed up all night to see if it would confirm the bending of light by the gravitational field [as predicted by Einstein]. If he had really understood the general theory of relativity, he would have gone to bed the way I did.
Nevertheless, ultimately the criterion of elegance is subordinate to observations. It doesn't matter how beautiful or simple your theory is, if it gets the facts wrong. To be sure, sometimes experiments turn out to be wrong too, especially when they go against fundamental principles of theory (like the recent supposedly faster-than-light neutrinos thing). But if, in the long run, experimental observation can't correct our prejudices, then there's no point in doing science. Nature may be beautiful but that doesn't mean that she (or her Creator) cares about our personal aesthetic of how things should be run. In the greatest popularized physics lectures of all time, Feynman advises that:
Finally, there is this possibility: after I tell you something, you just can't believe it. You can't accept it. You don't like it. A little screen comes down and you don't listen anymore. I'm going to describe to you how Nature is—and if you don't like it, that's going to get in the way of your understanding it. It's a problem that physicists have learned to deal with: They've learned to realize that whether they like a theory or they don't like a theory is not the essential question. Rather, it is whether or not the theory gives predictions that agree with experiment. It is not a question of whether a theory is philosophically delightful, or easy to understand, or perfectly reasonable from the perspective of common sense. The theory of quantum electrodynamics describes Nature as absurd from the point of view of common sense. And it agrees fully with experiment. So I hope you can accept Nature as She is—absurd.
I'm going to have fun telling you about this absurdity, because I find it delightful. Please don't turn yourself off because you can't believe Nature is so strange. Just hear me all out, and I hope you will be as delighted as I am when we're through.
Excellent advice for anyone who wants to see the world scientifically. Perhaps you can already see some implications for religious views, but we'll go into that some other time.
