Coconut milk/cream vs cow milk in recipes

If you are not expecting a dairy-specific reaction, you can generally substitute fairly freely. Caveats are things like cheese- or yogurt-making, or whipping cream -- you can make yogurt or whipped cream but the process is not identical; non-dairy cheeses exist but I have never tasted one that approximates a flavor or texture that I associate with cheese; on the other hand, I don't think I've ever had one made from coconut, either.

One issue you may have, though, is in the terminology of defining coconut milk and cream. :-)

Coconut Milk #1 - In the US, you will find something called coconut milk in the refrigerated dairy section of well-stocked grocery stores in places where they cater to vegans and the lactose-intolerant. This is a good 1-to-1 substitute for regular dairy milk in most cases. I personally find that coconut milk is a lot closer to dairy milk than most soy or almond milks, and I think they have to work less hard to get it that way, meaning potentially fewer ingredients and stabilizers to mess with the chemistry of what you're cooking.

Coconut Milk #2 - In the baking section and/or the Asian food section, you may find something called coconut milk in cans. This is a good direct substitute for dairy cream as long as you're not trying for whipped cream (if you are, you need to chill it first to separate the actual fatty cream from the liquid). Check the ingredients. There should be no added sugar, it should look something like this:

Coconut Cream #1 - In the US, most products called "coconut cream" are cans of a combination of coconut milk with various stabilizers and flavorings (to make it taste more coconutty -- real coconut milk has a distinctive but mild flavor) and a lot of sugar. These are generally intended as mixers for tropical drinks, but they are sometimes found in the baking section as well. Check the ingredients. If it has a lot of sugar, it's not meant for baking and it is not a good direct substitute. (you'd have to adjust other ingredients, and depending on the brands and what else is in there you could get some weird flavors.) If you see an American recipe that calls for coconut cream, you may need to use some judgment to determine what is actually intended, unless it gives a brand name like Coco Lopez. Not good for substitutions:

If you have a good Asian market nearby, you have some other options.

Coconut Milk Powder - In addition to tins of Coconut Milk #2 from above, you may also find Coconut Milk Powder (Maggi brand is common in Indian markets) which is an acceptable substitute for either dairy or coconut milk or cream in baking or curries (adjusting the liquid to powder ratio gives you the distinction), but it is a little harder to get fully reconstituted to a smooth texture, so while I do like to have some on hand and I do use it in my baking, it isn't really quite as simple a substitution.

Coconut Cream #2 - You may find pressed blocks of semi-dry coconut cream, the fatty part that was separated out when you chilled your can of coconut milk to make whipped coconut cream. This is another one that is great to have on hand and can work but it's not as easy to work with as a direct substitution.

EDIT: I didn't answer all of the question, I think, regarding what coconut milk is like.

Coconut milk has a very mild taste. It is mostly fairly neutral (reasonably similar to cow's milk) and not what you think of for "coconut" flavor (or at least, for me, that is more the flavor of the toasted coconut flesh). It does impart a distinctive flavor to foods, but it is still a mild flavor and can be overpowered by other ingredients and may not be as pronounced as you might expect at first.

The drinking style of coconut milk (non-dairy cow's milk substitute) is pretty much just like cow's milk in texture, or as close as they can manage to get it.

The baking/cooking style of tinned coconut milk is a little fattier than heavy cream, and tends to separate, but mixes back up to a similar consistency to dairy cream. (It mixes better at room temperature, or when just a little bit warm.)