Well, it doesn't scale, that's what's wrong. Ever been at a really big party, where a couple of hundred people were chattering away? You might not be able to hear every word every person is saying - but you sure as hell can hear the combined babble assaulting your eardrums, making it hard to hear what the really cute guy you're trying to chat up is saying...
That's one reason why Mike doesn't like people using Twitter to send messages to an individual. To me, it suggests one reason why Mike probably doesn't like parties, either.
Yes, sending unicast messages over a broadcast medium is inefficient; but it's also the core of a great party. You're drifting towards the punch table when you hear Scoble telling SoulSailor about a new feature that you didn't know about on your favorite site.
Yes, the broadcast nature means you get hit with a lot of messages that aren't directed at you. If you don't like that, go back to your unicast-only media.
The fact that conversations that start as one-on-one can grow to encompass the whole group; the fact that you can learn interesting things because you hear snatches of conversation flying past - that's what makes a good party fun (that, and, aforementioned cute guys and the snogging in the corner that happens later); they're what made IRC great, back before unicast IM systems stole the users away. It's what makes Twitter great, too.
Admittedly, some way to deal with the scaling is needed - have I ranted about channels enough yet?) - but if you think that using a broadcast medium for unicast messages is "abuse" - you'd be better off not coming to the party.