What an excellent question - I often feel the same & end up repeating myself all over. My preference for comments would be on my blog rather than say Facebook or Twitter but then more people probably see the last two rather than my blog. To me a blog is a more 'permanent' feature and more searchable, I think, by Google.
I agree that the best place to comment is the blog. That's where the content is, and everywhere else is just a link to it. Comments on the blog are visible to everyone who reads afterwards. Having said that I'm happy for comments wherever they are, and I often get them on Facebook instead.
It gets even worse when you are trying to provide customer support. People ask questions on your blog, on your forum, on Twitter, on Facebook, through email, the phone, fax and direct contact. You have to answer and support them all. The "marketing" answer to this is that everyone has their preferred mediums. Some people may not see the comment on your blog, but may read your Tweets. So unfortunately, we have to choose whether to satisfy them all, or choose the media and thus type of people who we are able to handle. Maybe one day this will all converge again into a single info flow - at least we can hope.
Good question. I prefer to have the comments on my blog for all the reasons Carole mentions but appreciate feedback from any source. One of the issues today is information overload. There has never been so much great content and value given away and freely shared as is happening today. The problem is keeping up with it all. I truly appreciate bloggers like Randy Seaver and others who do "Best of" posts. I also appreciate the content wikis that are appearing. Keeping up with it all is a challenge.