Frozen and Unfrozen Chicken: what's the difference?

Yes, there is a clear difference in never-frozen vs. defrosted meat.

The cell plasma in your chicken freezes in the freezer, turning into ice crystals. Since cell plasma is over 90% water, it expands instead of shrinking in the 0 to -4°C range. The crystals cut the cell walls. When you defrost the meat and cook it, the cell plasma flows out into the pan, leaving you with dried out meat. This is the first, major problem with freezing, and occurs even when the meat is frozen for a very short time.

If you keep meat stored frozen for longer time, there are two other effects. The unavoidable one is rancidity. The cell plasma in frozen meat has a high mineral concentration, and given time, it will oxidize the fat in meat, changing its taste for the worse. The avoidable one is freezer burn - it occurs if the meat is not wrapped properly in the freezer. Freezer-burned meat is really unpleasant to eat, most people throw out affected parts.

Of course, this is not the only factor affecting meat quality. SAJ14SAJ's comment is true in the sense that bad meat (from mass-produced young animals grown in bad conditions) will not have much taste of its own, and even if you prevent it from drying out, it still won't be as nice as good meat. But what is more pertinent in this case is that the cooking process can more impact on drying out than a freezing-defrosting cycle. If you cook defrosted meat carefully, you will minimize the dryness effect. It won't be the perfect morsel, but it can still taste pretty well. On the other hand, if you have never-frozen meat and overcook it, it can get to shoe-leather consistency, so that you lose all advantages from non-freezing.

References: McGee on Food and Cooking, p.146 (reputable, dead-paper book) or http://www.wedlinydomowe.com/sausage-making/freezing-meat - (random Google hit, one click away).

Frozen chicken can often have a different texture to chicken that has never been frozen... Although I couldn't personally say I'd be able to tell the difference in a blind taste test, I find chicken that has been frozen tends to be a bit flakier... This may just be my experience from when I've bought frozen fish, but I find it breaks a bit more easily in a manner reminiscent of fish, although the texture is still primarily chickeny.