Do I need to grind Ceylon Cinnamon sticks for coffee/tea, or can I add them whole?

You can definitely put a whole stick of cinnamon in with your coffee grounds or tea leaves. If your cinnamon stick is very tightly coiled, the hot water may not reach the inside of the coil. Split the stick lengthwise so the water can more easily reach the inside of the bark (that way you get flavor out of both sides of the cinnamon bark instead of just one side.

You're correct that you won't get all the cinnamon flavor out of a cinnamon stick in a single use. However, it doesn't have to be wasteful - you can reuse the cinnnamon stick multiple times. After your coffee is brewed, pull the cinnamon stick pieces out of the grounds, tap off any remaining coffee grounds, and lay them out to dry. It helps if you do this while the grounds are still hot, so the residual heat in the cinnamon stick helps it to dry fully. Make sure the cinnamon stick doesn't end up sitting in water. You can put it on a dry towel, or prop it up on the edge of a clean mug, or put it in a mesh strainer.

Re-use the cinnamon stick as often as you make coffee. When you notice it starting to lose flavor, break it into more pieces. Eventually, the parts will run out of cinnamon flavor and/or get too small to be worth the effort of saving, and you'll start a new stick.

Note: this works because the cinnamon stick has only been exposed to coffee grounds (or tea leaves) and hot water. Don't save a cinnamon stick that you've used to stir a cup of coffee with milk or cream. You won't be able to get all the milk or cream off of the cinnamon stick, and it won't be food safe for your next use, and it might taste bad. (Of course you can reuse it if you make your next cuppa immediately; just don't save it for tomorrow.)

Most people just use ready-ground cinnamon.
It's much cheaper & just stirs in with no chewy, twiggy bits.

Of course, it looks a whole lot better with en entire stick of cinnamon bark sticking out of a tall, elegant glass/mug, but the amount of flavour you're going to get from it is going to be minimal in comparison.
If you're cooking with cinnamon, it will simmer for several hours, after which you pick out the bits & throw them away. You could see how much flavour you get from one steeped in hot water for 15 minutes or so & it might be enough, but I'd save the sticks for a long-cook, or for when you need the look, rather than when you just need a hint of cinnamon in a tea or coffee.

I have never actually tried it myself [I keep both in the cupboard, ground & sticks], but I doubt you'll get a smooth-enough powder from any home-grinder if you try to DIY.

Generally agreed with Tetsujin, but I'd personally recommend that for coffee or tea specifically, grind it into coarse chips and add it to the tea leaves or coffee grounds. In my experience that's the best way to get a nice, noticeable, though not strong, cinnamon flavor in your drink with no unpleasant textural experiences.

Sticks look nice, but impart little to no flavor. They're there for the look and the scent. Those two aspects will be enough to convince some people they're tasting cinnamon as well. But one thing about adding a whole stick to the drink is that you must A) take it back out before drinking, B) hold it in place whole you drink, or C) have a stick smack or poke you in the face while you drink. I usually don't think and end up with option C personally.

The first problem with ground cinnamon is that sometimes it turns slimy, clumps together, and refuses to mix in. But you still get a nice flavor. The other problem is that when the powder does hydrate and sink in the drink it becomes much coarser and noticeably grainy. I never measured to see how much ground cinnamon swelled up in the bottom of my drink, but unlike the fine sludge that will settle in the bottom of a cup of French Press Coffee, to me the cinnamon sludge was undrinkably coarse.

Of coarse I could just be buying the wrong type of ground cinnamon, but I think a very coarse grind might actually be ideal for coffee and tea.

Yes, You can put a whole stick of cinnamon if you like the taste in your coffee or tea. But but grind cinnamon powder will give you perfect flavor. The reason is stick is very tightly coiled, the hot water may not reach the inside of the coil.