Interesting question and I think important in addressing in all areas of study. It's almost like we're asking about the study of study. It makes sense that we should know how and why we study before we set out upon a certain goal.

To me theory is sort of like story-telling. When a new theorist comes about or a new theory, it takes an older story and rewrites it. It's almost like an adaptation. It takes what it considers significant and transfers it to a new system.

Theories attempt to be universal. In this way they have to address certain universal properties. In gradeschool they taught these to us as the Who, What, Where, Why, and How. All theories address all of these points.

When a new theory comes about it retains some of these properties and alters others. Therefore it connects to an older model and to an older world while making concessions to new modes.

For instance, with the example you gave Patrick of evolution, Darwin goes ahead and changes the How of it all. Before his theory man had always been, but with it he gave us another option, another choice as you said, for how we came about.

The world is a constantly changing place. It would suck if it weren't. But because things change, old models will always need to be altered to fit the times.

Just how exactly theory takes over the public imagination, well, I think that is an altogether trickier question...

What we want theories to do, more than anything else, is to cohere, to be consistent with one another. As ridiculous as this may sound, we would ideally want our theory of the sublimation of carbon dioxide to at some point be consistent with feminism. But in order for this to happen, we would have to discover the nature of the vast number of theories that would lie in between these two areas of inquiry. As we would proceed from one to the other, we would be confronted with the essential difference between the two. The natural sciences have the advantage of being able to demonstrate reproducibility of results to a very high degree of precision; the theory that water boils at 212 degrees Fahrenheit, given certain other parameters, can be (re) discovered by anyone. Alas, such is not the case with the social sciences, in which are found such theories as feminism. The social sciences are, perhaps, forever consigned to the circle of hell that hold middle-range theories, being neither able to explain very general things, like the meaning of it all, nor very specific things, like why John Smith of 123 Mockingbird Lane likes vanilla ice cream better than chocolate (with apologies to any Smiths possibly living at any such address). They're pretty good at explaining how violence on television affects youngsters in general, but again, not any given young person, nor why violence is always a bad thing (if indeed it is).
Recently, I have noticed the ascent of two disciplines in theory-making - economics and neurophysiology. I am hearing eminence grises in both fields happily straying pretty far from their traditional haunts and asserting theories thereunto. For example, there was the recent example of economists expounding on how dating between the sexes works (to me, this was REALLY a stretch). And more than ever, I am hearing that we enjoy poetry, fall in love, prefer certain music or swing on swings because certain processes in our brains switch on and off. Not our minds, mind you: but our brains.
What is important is not even whether such theories are plausible, but what their implications are for us as ethical beings. For example, if there is no such thing as mind, or if we believe that minds are nothing more than the present function point of our brains, then it is difficult to see how or why we would ever believe that there was an ethical component to anything. There would be no "should"; there would be only "is." As for an economic point of view, if rational self-gain is the only point to anything, then any other consideration is either incommensurable with that, or immensurable in and of itself.