Why can food be left out to cool (140F to 40F in < 6h) but not to defrost?

The very document you link to clearly explains the reason:

It is important to use methods that will allow the entire mass to thaw evenly. Any method that allows one part, for example, the outside surface, to defrost before the inner portion is not acceptable, because the portion that thaws out first will be in the danger zone before the other portion is thawed.

Bacteria grow and produce toxins while in the danger zone. If you thaw on the counter then the exterior will be crawling with them long before the interior is thawed.

The other methods are acceptable because:

1. Keeping the food refrigerated never permits any part of the food to enter the danger zone.
2. Very cold water keeps the exterior cold, which is close enough to refrigeration as long as the food will be cooked immediately.
3. Microwaves partially penetrate the interior and cook the entire portion very quickly; the food will be in the danger zone but for too short a time period to matter.

It's also very important to note that the cooling rules are not about simply "watching the clock". If it were that simple, they wouldn't bother with rules, because room temperature is in a pretty consistent range no matter where you are. Cooling hot, potentially-dangerous food on a countertop is every bit as inappropriate as defrosting it there. Food service professionals will hold the food above 140° F until it is ready to serve or store, and if storing, they'll cool it very quickly by dividing into smaller portions and/or using an ice water bath or even specialized equipment (e.g. flash freezing).

There's really no practical difference here. The extra 1 hour permitted to cool from 140° F to 70° F is really only because you've just cooked it, so (a) it's been pasteurized and (b) temperatures very close to 140° F are not very hospitable for bacteria should the food become contaminated again. When defrosting from frozen, it's assumed that the food (cooked or raw) is already contaminated either from factory handling or from prior cooling in the danger zone, so you need to strictly follow the 2-hour rule.

Cooked foods are starting with a very low bacterial count, because the cooking process killed everything. Defrosting foods are starting from an unknown count, because they picked up some during the cooling process before they were frozen. Bacteria multiplies exponentially, so the starting point matters greatly.

I'm glad you asked for theories because I don't have anything to back this up.

When defrosting there could be a big difference in temperature between the outside and middle of the item. The middle could still be rock hard when the outside is warmed into the danger zone on a summer's day. Thus the recommendation is to defrost within a refrigerator as the max temperature any part gets to will be controlled. Defrosting in running water or microwave are also acceptable as they are fast and so little has time to grow and go bad.

We do seem to be missing an important point here... cooling is not allowed to happen at "room temperature" at product temperatures below 140 F; only until reaching 140 F from some higher temp. In fact, no food that supports bacterial growth is allowed to be kept at "room temperature", or any temperature inside the danger zone (40 - 140 F) for any period of time except in preparation for chilling, cooking, or service, and that only very carefully under sanitary conditions. Check the USDA guidelines (I just did, courtesy of NDSU: http://www.ag.ndsu.edu/pubs/yf/foods/fn572-1.htm#Safe )