Coffee begins to lose its flavour and freshness as soon as the roasting procedure is complete. Whole beans are best used within a month of roasting. The best way at looking at ground coffee is that it is similar to the whole bean, only with a whole lot more (pardon the poor english) surface area. That means that any of the breakdown that occurs to the bean will occur exponentially faster with a grind. You should always grind beans for each use, if you wish to have maximum flavour. I am no expert, but I can tell the difference between freshly ground beans (like my wife and I do), or not so freshly ground (like my in-laws do). I am no scientist, but the older the grind, the 'flatter' the flavour. It is definitely noticeable.

Posted as answer by request of @BaffledCook:

Here's an slightly informal blog post outlining changes in taste between varying degrees of freshness in the grind of a coffee.

http://investigationsblog.wordpress.com/2010/03/03/do-i-really-have-to-grind-coffee-right-before-brewing-it/

The short version is that the coffee starts losing freshness as soon as it is ROASTED. The longer it sits the faster it goes stale. The more surface area it has (ie: you've ground it up; also, the finer the grind), the faster it goes stale. The more you expose it to air, the faster it goes stale. So, seal your coffee in an air tight container at room temperature. And grind as close to the time you add water to it as possible.

How much of a difference detected depends on the the actual coffee been, the roast, the taster's taste buds. If you buy a bean that's been sitting on the shelf for 3 months already, you will probably notice less of a difference than a bean that was roasted last week. I buy beans that are roasted and sold within a week, and I DO notice a difference if I leave the grounds for a day or two before drinking.

I commonly drink espresso, french press, and stove-top (Moka-pot) coffee. Here's my personal experience:

Freshness

There are (at least) three different stages during which to measure freshness, and the length of time before the coffee goes stale changes at each stage.

Green Coffee

After the coffee cherry has been processed, but before roasting. Coffee in this stage will last months.

Roasted Whole Bean

There is some contention about how long coffee in this stage can be considered fresh, so your mileage may vary. I find that roasted beans last 1-2 weeks. I notice the change in flavor starting at about 1 week after roasting, and I'm ready to throw out old beans after 2 weeks.

Ground Coffee

Freshness lasts minutes (at best). Espresso will demonstrate this the most dramatically, but other coffee drinks will benefit from grinding immediately before brewing.

Flavor Differences

Flavor differences will depend on the specific coffee and the brew method, but in general fresh coffee is rich and tastes more like dark chocolate, while stale coffee is bland and tastes more like dirt. In my experience the ability to distinguish is learned, and it's hard to unlearn.

I don't think coffee goes stale very quickly if stored in an airtight container. The beans are very dry, there is not much to go off

Some people like the aroma of freshly roasted coffee, some people just like it fresh ground. Some people just like coffee made from grounds in a French press

This is all personal and a subjective thing. There is certainly a difference between fresh ground and stored ground, but ground coffee does not go off

To me fresh roasted and fresh ground is great. But once it's an old roast, fresh or old ground makes little difference as far as I can taste in a cup of coffee

Yes, there are noticeable differences with storage, as others have noted. The coffee after roasting is pretty sterile and too dry to support microbiological growth, and the flavour changes are linked with chemical reactions with oxygen in the air. These are basically loss of flavour at first becoming flat and dull, followed by development of off flavours, becoming rancid and unpleasant, and may be less obvious with milked coffee. Roasted coffee beans are protected from oxygen by carbon dioxide evolved during roasting and probably last some weeks if transferred rapidly to an airtight container. After grinding the carbon dioxide is quickly released and the coffee is more vulnerable to oxidation - a good taster can detect flavour differences within hours. If the ground coffee is protected from oxygen, by vac-packing, valved packaging or flushing with an inert gas in air-tight packaging, it will still last some weeks, but deteriorate once the pack is opened. At any stage after roasting they benefit from protection from light and preferably storage in the fridge or even frozen. How soon all these changes are detectable or unacceptable, depends on many factors, especially the sensitivity of the taster - I've known people who happily drink coffee that makes me almost sick.

Yes, I will also add to the above comment. As soon as you grind coffee (when it is reasonably fresh) it starts to oxidize and loose the gases trapped within the whole bean. The finer the grind, the more surface area present, and the faster this happens. For an espresso grind, the window you have to work with is around 30 seconds to a minute. When you are referring to "grocery store" coffee it is likely up to a year old and has long lost most of these gases so I'm sure the effect of grinding would make less of a noticeable difference. (However it's been years since I've experimented with less than great specialty coffee).