How does one invent a cookie recipe?

Yes, the original recipes involved an enormous amount of trial and error. People baked, swapped recipes, and the good ones were desirable and became widespread, while the bad ones died out. The advanced knowledge of food chemistry wasn't even available at the time - Hannah Glasse published her cookbook more then 100 years before Mendeleev published the periodic table.

Luckily, you don't have to go through the same process if you want to invent a good recipe now. There are only a few basic ingredients in baking - eggs, sugar, flour, water - and you can be sure that all possible ways to combine them, as well as most secondary additions and substitutions, have been thoroughly explored. People have distilled the knowledge of this exploration in books, and you can use this basic knowledge to build new recipes which will function with a high probability.

Before you develop a baking recipe, there are four things you have to know.

1. Technique. If you want to create a cookie recipe, you will have to use creaming. If you want to make a new kind of eclair, you have to know how to prepare a basic pate a choux. There are cookbooks teaching these techniques, some of them are explained in questions here on SA.

2. Base ratio. For most baked goods, there are ratios which give you the best results. For example, for a crepe you want 1 part flour, two parts egg and two parts liquid. As long as you keep that, you can let your imagination run free by using different liquids, or adding spices, or even plopping pieces of fruit in the pan and pouring the batter over them. You can learn about this from Ruhlman's book Ratio, or use a known-good recipe for a plain variant of the good you are trying to make as your starting point.

3. Flavor combinations. You can add any ingredients you want, but there is no guarantee they will work well together. Choosing the right ones is a combination of talent (being able to imagine what a combination will taste like before you have had it) and experience. You should try to be more analytical about the flavors of things you eat - which tastes can you distinguish? Which aromas? What makes them go well together? Is it their similarity, or the contrast? - and when you have done that long enough, you will be able to predict the goodness of a combination. A book to help you along is The Flavor Bible, which describes good combinations. You can also take popular combinations and transfer them across types of food. For example, if you like apple-cinnamon pie, you can try to make cinnamon cookies glued in pairs with apple butter.

4. Ingredients' structure and their role in the baked good. This knowledge is maybe the least usual among home cooks. When you create new recipes, you will always dilute the base formula, or use substitutions. If you don't have this knowledge, it will be hit-or-miss whether your new recipe will work or not. It is a bit like rebuilding a house's interior without knowing which walls are load-bearing and which aren't. If you know what each ingredient does in a recipe, you will know when a substitution is possible and when it isn't. For example, many people will tell you that applesauce is a substitute for eggs. And you can indeed bake a cake with applesauce instead of eggs, but don't try to whip a mayonnaise with it. This is because eggs have a different role in cake and mayonnaise. Learning about these roles requires a lot of effort, and a curiosity about these things. Most people will find it easier to just try whatever they feel like, and live with the occasional failed new-recipe attempt. If you would rather learn about them, I can recommend two sources. One is Cooking for geeks by Jeff Potter, which is short and doesn't go into the depth of things, but offers lots of useful information and is easy to read, or the really exhaustive book called On Food and Cooking by Harold McGee, which runs to about 800 pages and goes into detail most people don't want to know about, but is a must-read if you are fascinated by food science.

And, of course you can always just run into the kitchen and experiment. Don't forget to document along the way so you can reproduce your results! Basically, you can always have luck and hit a good new recipe, the knowledge I listed above (combined with some baking experience) helps you increase your first-try success rate from about 2-3% as a full amateur to somewhere about 80% or more. It is your decision how to divide your time between reading and baking, any combination can function, depending on your learning style.

Alton Brown dicusses how to change cookie recipes in the Good Eats episode "Three Chips For Sister Marsha" (S3E6P1)

The transcript can be found here: http://www.goodeatsfanpage.com/Season3/Cookie/CookieTranscript.htm

From scene 14, we learn that

If you:

• add soda
• replace 1 egg with milk
• high ratio white:brown sugar
• use butter

You get: Thin & Crisp cookies

If you use:

• cake flour
• baking powder (instead of soda)

You get: Soft cookies

If you use:

• melted butter
• bread flour

You get: Chewy cookies

The long version of Scene 14:

Now take your favorite chocolate chip cookie recipes, add a little extra soda, replace some or all of eggs with milk, up the ratio of white to brown sugar and use butter rather than shortening and thin, crisp cookies will result.

If your tastes run to the soft and cakey, use cake flour, baking powder rather than soda, and shortening instead of butter. Chilling the batter and scooping on the small side will add to the puff factor, too.

Now finally, chewiness calls for melting the butter, holding back on the egg whites and using more, if not all, brown sugar.

These things need lots of experience, and trial and error of the recipes. A recipe might not taste good the first time around, but with time you can improve the recipe.

As for inventing your own recipes, try mixing things that match with each other (for example: instead of putting water, try adding pineapple juice with some pineapples on the top to get a pineapple cookie). Take into consideration the amounts of water and flour (to get a perfect dough) used in the recipe.

You do need the experience and the talent of cooking so that you can consciously group the ingredients.You can also learn something from the recipe you can find and have a try over and over.

Another thing is that you need someone to check out the effects of your cooking and give you some suggestion.