Why doesn't mayonnaise taste like pure oil?

Starting from the basics: Mayonnaise, as you know, is a combination of water-based liquids, water-soluble ingredients, and lipids (fats/oils). Since water and lipids are immiscible, that makes mayonnaise an emulsion.

Because the droplets (of fat) suspended in an emulsion are not actually dissolved, the properties of that emulsion depend entirely on the size of those droplets and their dispersion. The most likely reason that your mayonnaise tasted like oil is that it actually was pure oil in spots.

The technical term for this is flocculation.

(source: Cube Cola)

This is probably what happened to you - it's possible that if you had really poor dispersion, you might have even been closer to the "coalescence" stage.

To use a more tangible example, consider what happens when you dissolve flour or corn starch in cold water, then heat it. The starch gelatinizes and you end up with a fairly uniform, thick paste. Now think of what happens if you toss it into hot water; you'll tend to end up with something that isn't uniform, instead you'll end up with big globs of cooked flour floating around in thin, cloudy water.

Keep in mind that the chemistry is completely different with an emulsion - in fact, there technically is no chemistry happening with an emulsion until emulsifiers come into the picture - but the concept is the same. You might not be able to see those globs of oil floating around in the water as well as you can see the globs of flour, but if you didn't get proper dispersion and suspension, they're there, and they will taste exactly how you'd expect a glob of pure oil to taste.

Traditional mayonnaise uses raw egg yolk (containing lecithin) and mustard (containing mucilage), both of which act as emulsifiers. These are called "emulsifiers" mainly because they help the emulsion to stay stable, which is why the store-bought mayonnaise doesn't separate (it also probably has a few extra additives). However, they aren't all that helpful for getting that initial dispersion; the most efficient way to do that is to let small drops of oil into a liquid that is being constantly and uniformly agitated.

You can do this by stirring, but an even better way is to use an immersion blender with emulsifying blade. Note that this is not the flat aerating blade that is often confused with the emulsifying blade, nor is it the star-shaped liquefying blade that is the default on most sticks and many manufacturers confusingly call an "emulsifying" blade. The one you want looks a bit like a hubcap; it's flat with several slits or holes and is sometimes also called a "smoothie blade" or "whisk blade":

or

(the one I'm talking about is the bottom left)

These things are perfect for preparations like mayonnaise, but if you don't have one, you can get halfway decent results with a wire whisk. You'll just need to use a lot of elbow grease.

If you get really good dispersion, and use sufficient emulsifiers such that the emulsion doesn't separate too fast, then I promise you, your mayo won't have that "fatty" taste and it will be 1000 times better than the store-bought goop.

Aaronut's answer can't be beat if your issue was primarily textural.

As a matter of flavor, though, if your flavor, rather than texture, was too oily, perhaps you should have added some additional acid (lemon juice or some form of vinegar) to your base. You may also find that additional sugar more closely approximates the mayonnaise you're accustomed to. (Japanese mayonnaise, for example, is even sweeter than the American or French counterpart, in my experience). You'll probably need more sugar as you add acidity, unless you're looking for something for artichokes or something similar.

I've found neutral oils most closely match my expectations of the taste of mayonnaise. Olive oil works, but it has a strong taste and almost a different animal in my mind. But there are better quality neutral oils, too. Perhaps you need fresher oil, or something with a more pleasant smell.

One thing I haven't seen mentioned is why you should absolutely avoid using extra virgin olive oil. Not for any of the reasons mentioned so far, but because it will become very bitter due to it's unique chemical properties. I discovered it the hard way and The Bitter Truth About Olive Oil explains why. I cringe every time I see someone like Jamie Oliver or another famous cook put EVOO in the blender.

Note that you can get away with using plain (non-virgin) olive oil, but I agree with @JasonTrue that the more neutral oils better match my expectations of home made mayo flavor.

Extract from the article:

According to Cook’s Illustrated, extra-virgin olive oil is the only kind of oil susceptible to becoming bitter. Even pure olive oil can handle blending better than the extra-virgin kind. The reason is because extra-virgin olive oil contains a high percentage of molecular compounds called polyphenols (thought to be cancer-fighters), which are normally coated in fatty acids. Under standard conditions, the fatty acids in the oil prevent polyphenols from dispersing in an aqueous environment. This is because oil and water do not mix.

When these fat molecules are broken into droplets in an emulsion, however, the polyphenols are distributed into the solution and their bitter taste can become apparent. When the emulsion is only lightly blended, the bitterness is not perceptible. But a blender or food processor breaks the droplets down into smaller sizes, increasing polyphenol dispersal. These suspended polyphenols can ruin an otherwise delicious recipe.

The easiest way to avoid this problem is to use either pure olive oil or a different kind of oil altogether, such as canola or safflower oil. Alternatively, if you would like to keep the rich taste of extra-virgin olive oil you can hand whisk your emulsion rather than using a blender. Just be careful not to over work the mixture. You can also start your recipe by blending a small amount of stable oil (e.g. canola), then hand whisking your extra-virgin olive oil in at the end.

Your mayo will only taste as good as the ingredients you use, so using a nice olive oil will certainly be better than canola oil (and one definite advantage to making it yourself). The main key, though, is to add the oil slowly... so it's not so much more whisking or blending, as thoroughly incorporating the oil a tiny bit at a time. The other main ingredient in homemade mayo is egg yolk, so some of your flavour and mouth-feel should come from that.

You might find the detail given in this recipe helpful to compare to the technique you used.