Egg whites contain a significant amount of protein, which is an excellent binder when cooked. This protein is what makes a cooked egg white somewhat rubbery.

Egg yolks contain lecithin, which is an emulsifier. The lecithin in an egg yolk is what combines with oil to make the emulsion we know as mayonnaise.

The white (with its protein) and yolk (with its lecithin and fat) are both critical to the chemistry when baking. If you eliminate just the yolk, you may get a tough and flat result. Substitutes like silken tofu, yogurt, applesauce, or mashed banana can all be used to replace either the yolk or a whole egg.

The best substitute will depend on the exact recipe and desired result. Applesauce and banana tend to impart added flavor, which can be a consideration as well.

It depends what binding you're looking for. In an emulsification, the yolk is the relevant part as it contains both polar and non-polar components. One of these components is lecithin, which can be extracted from both vegetal and animal sources and is commonly used on its own. It's a powerful ingredient that prevents fats from separating from watery substances.

The white contains no fat and a significant amount of protein, which provides a binding effect in that it can be worked into another liquid and cooked to provide solidity. Proteins can also be whipped into and maintain an aerated structure, binding air to the finished dish.

I'm primarily restating AMtwo's perfectly acceptable answer because I can't comment and I would like to provide the real source for moodboom's link to a pop-science article, which does in fact describe a correlation between cardiovascular events and egg consumption, but only when overall cholesterol intake is not accounted for.