Is there a water kettle or dispenser that can keep the water lukewarm?

The easiest tool I can think of is a thermos - fill it with water at or slightly above your desired temperature and enjoy your supply for a few hours.

On a larger scale, a tankless heater can give you a basically unlimited supply of water, but is almost certainly overkill.

Please be careful with all devices that keep water at around 100F / 37C, this lukewarm water will encourage the growth of microorganisms, aka a biofilm. This is also an issue with all water dispensers / coolers, which should also be cleaned regularly.

Why don't you grab a bottle warmer for baby food? Smaller quantities, but you can set the temperature just fine.

There are kettles on the market which have an adjustable electronic thermostat, some of them allowing very low temperature settings. Example (review article of some such kettles, some of them appropriate for the task): http://www.independent.co.uk/extras/indybest/food-drink/best-variable-temperature-kettles-adjustable-electric-glass-no-plastic-2016-tea-a7179216.html.

Mixing boiling and cold water is very feasible. Two parts cold to one part boiling will give about the temperature you want. I use this method for the rapid bake program in the breadmaker, which expects water at an optimal temperature for yeast (actually slightly higher as the flour is at room temperature). For breadmaking I measure by volume, but for drinking you can do it by eye in a glass. There's no risk in this method: if it turns out too cold, at a drop more hot, and if it turns out too warm, wait a moment or add a drop of cold.

An alternative is to work out how long you need to heat a glass of water in a microwave from cold. Try 30s to start with. It will need a stir afterwards.

An alternative is to put a thermostatic mixing valve below your sink and add a warm water tap.

A thermostatic mixing valve accepts cold and hot water inputs, and can be set to output a set temperature. It automatically adjusts the ratio so the output is the set temperature.

The valve is inexpensive, and combined with a single valve faucet above the sink would fulfill your needs without requiring cords and appliances on the counter.

https://www.amazon.com/Fyeer-3-Way-Thermostatic-Mixing-Connections/dp/B01MS3X57O/

There are devices for keeping coffee or tea warm at your desk called mug warmer. If you only want to keep a single mug worth of water warm, I think it will work well. I have linked one that has a temp you can set.

There are three things come to mind. Point of use water heaters. For a shower or tap. Set the temp. turn on the switch, turn on the tap. electric. Or water heaters for pets. Set the temp. hold water at a high of about 100f. Found in pet stores. For 2 1/2 gal. or 5 gal. Set on counter. Building supply stores or the pet shop. I see them in Asia under baby needs. Were women want warm water for a child on hand. But look on line there for one. May not be in America. Most of Asia is 220v so may not work in America.