Vanilla Extract with and without alcohol

According to the FDA vanilla extract must contain a minimum of 35 percent alcohol ("TITLE 21--FOOD AND DRUGS"). That's roughly on par with vodka.

There's no such thing as flavorless alcohol. Ethanol has a flavor ("Bitter and sweet components of ethanol taste in humans") and alcohol (at least, when the dilution is greater than 35 percent) produces a burning sensation in the mouth and throat. Perhaps this is what you've identified as the harsh flavor.

There are pretty good alternatives, fortunately.

One option is vanilla flavoring. This is similar to vanilla extract, in that it's made from real vanilla beans, but it contains no or very low amounts of alcohol. Instead, the solvent will often be propylene glycol.¹

The other very common option is what's sometimes called imitation vanilla, which is not made from vanilla beans, but instead from artificially produced vanillin. Such imitation flavorings may or may not contain alcohol.

Vanillin is an organic compound that's responsible for the most prominent flavor in vanilla beans. It's produced naturally in the beans, and artificially through a number of different processes (which some people think aren't safe, but most of the concern I seen seems not to be grounded in science, but in a fear of "chemicals"). Vanilla beans also contain a ton of other flavors, which is why imitation vanilla may be thought of as inferior. But, on the other hand, a lot of these other flavors can either be denatured or simply overpowered, especially when baked. The general advice I've heard (which may have come from America's Test Kitchen) is to use imitation vanilla in baking and vanilla extract for making things like whipped cream, where the vanilla is practically all you taste.

Adding imitation vanilla to tea may not improve the flavor (over vanilla extract, that is), but it costs very little, so it wouldn't hurt to try.

1: As with so much in the online world, there's a lot of bad information out there about vanilla flavoring. For instance, look at what Eden Foods says about vanilla flavoring:

Natural Vanilla Flavor is an extract of many things using animal glycerin or propylene glycol (antifreeze) and a googol of other synthetic and/or toxic chemicals.

https://www.edenfoods.com/articles/view.php?articles_id=83

Thirty seconds of Googling will tell you that antifreeze is ethylene glycol, not propylene glycol and that the FDA considers propylene glycol to be "generally recognized as safe" ("PART 184 -- DIRECT FOOD SUBSTANCES AFFIRMED AS GENERALLY RECOGNIZED AS SAFE")

The Center For Disease Control has put together a "Public Health Statement for Propylene Glycol", which you can read for even more information.