How to Make the perfect French Custard?

Custard should taste eggy. This is a feature, not a bug. However, from your mention of flour, I think what you're talking about is a souffle, not something like a creme brulee or along those lines. Would that be correct?

Edited based on subsequent clarifications:

Ah. Your method is sorely lacking, and your ingredients are off. You need to temper the yolks first, you need to use yolks only, and use cream instead of milk. Here is a smaller version of what I use at work for creme brulee: - 1L 35% cream - 11 egg yolks - 1C sugar - 1 vanilla bean, scraped

Bring cream to a simmer with the vanilla seeds and pod (you can use approx 1.5tsp vanilla extract, pure not artificial, instead). While it is heating, beat the yolks and sugar together until just incorporated. Pour the cream into the eggs whisking briskly to prevent curdling. Pour mixture through a fine chinois. Skim off foam, pour into ramekins. Place ramekins in a pan, add water to 1/2way up the sides. Cover pan with foil. Bake at 275 (convection) or 325 (non convection) for approx 40 minutes, until wobbly in the centre but set. Chill until set, eat.

The deflation is caused by a souffle effect coming from including the egg whites, which are never used in a custard--custards, creme anglaise, etc, are always yolks only. When you include the whites, air will be trapped inside temporarily, and will escape/collapse when the mixture cools. If you bake until fully stiff with my method, you will get hard and rubbery custard by the time it is set in the fridge.

Also, save the whites--they freeze well--for meringues or souffles.

Is the milk / cream too hot (/or cold) when you whisk it with the egg yolks? Do you return the mixture to the pan and cook it for long enough (slowly enough?)

Which recipe are you using?

EDIT

As roux mentioned in the comments to your question, the recipe you're using doesn't sound like it's up to the job.

I've used this recipe for Crème Anglaise with success in the past. Maybe it could be adapted to your requirements?

http://www.epicurious.com/recipes/food/views/Creme-Anglaise-4984

If you want to try another recipe, here's a great one from Delia Smith (author, food critic and British institution).

http://www.deliaonline.com/recipes/type-of-dish/sweet/traditional-english-custard.html

PS. Interesting to note that in America you call it French custard, while in France its known as Crème Anglaise (English cream). Here in Britain, its just 'custard' ;-)

First, do not use exclusively whole eggs, you got to have more yolk than whole eggs in your mixture otherwise the texture is compromised.

Never forget that you're dealing with a specialization of a base mixture. The ingredient you provided would make a somewhat eggy crème anglaise, whereas you're seeking the perfect crème patissière.

Crème anglaise is simply eggs / yolks mixed with vanilla extract and a bit of sugar on which you pour milk and heat slowly until the yolk starts to thicken the sauce, you have to take it off the heat as soon as it thickens and immediately cool it off, it would be wise to have a batch of ice in the sink to lay the saucepan on it, dont forget to stir rapidly so the heat will evaporate thus preventing the egg yolk to coagulate more.

Crème patissière is a variant of crème anglaise as you need to add a little flour either with a roux or a technique called singer, which is basically pouring flour lightly over the mixture as it heat. This preparation requires less care as the flour will prevent chemically the yolk from forming lump as you stir and the sauce thickens.

From crème anglaise you can cook crème brulé, crème caramel and serve it on dessert / breakfast items.

From this base recipe you can derive it and add flour and it will turn into a creme patissiere, which is basically the ultimate custard item, it is rather simple to execute , you just gotta make sure you dont use whole eggs exclusively and don't overpour flour in the mixture.