There was a reason I decided, after my undergraduate studies in literature, to switch to philosophy. I found that a well-argued point, no matter how contentious the point itself, could get you points in a literature department. In philosophy, that was no longer quite as easy to do. My master's in philosophy was my only "real" education in the sense that, although everything else was important, only the MA forced me to consider my words very carefully.
"Philosophy" as a concept means many different things and it's used in such a diluted form that we need to define it a little more clearly if we want a serious debate about whether it matters or not — but as a preliminary move, just so that I can make it clear where I'm coming from, I would define philosophy as a method more than anything else. To do philosophy would mean using a method of critical reasoning that always calls itself back into question. I'm particularly interested in the work of Hegel because he managed (or convinced himself that he'd managed) to create a whole method based on complete presuppositionlessness. He started out by taking absolutely nothing for granted, and saw where that could take him. That, I think, is the core of what has become the philosophical discipline.
You ask if our ideas matter to us — well, yes, I would say that they do. But what do you mean by ideas? The way you have represented the world to yourself? The abstractions that help you to make sense of the flux of activity everywhere around you? These would be basic questions that require answering if you wanted to take the first difficult step towards questioning your own words. Another type of philosopher might ask you why "philosophical ideas" should "make the world a better place" — not because philosophy needs to indulge in nihilism, but because that in itself is a pretty big prejudice that should be analyzed.
Still another approach would be to adopt a kind of ironic interrogative tone and ask how, exactly, you managed to attain "the secret knowledge that the whole thing is tied together with these fantastic little threads of thought" and what would happen if you DID tug at the whole thing. Would your whole life collapse in fact? Is philosophy that powerful? I'd say that it is, but only when it's done with care.
I think this is a great topic.