How do I identify a bad sharpener?

All sharpeners remove steel. When sharpening a knife, you ideally want to remove as little steel as possible to get a very specific angle on the blade edge.

If you were getting a blade sharpened professionally, they would typically go through different grits to get it smoother and smoother and eventually a polished edge. For a home pull through sharpener, you're using a single grit that will do most of the work. Ideally it would be something in the middle that will maybe take a little longer to remove nicks in your blade & grind the edge down, but not pull off too much steel.

There are two dangers with a cheap sharpener, that I can think of:

1. It removes too much. If the grit is too rough, it'll burn through your knife, destroying your blade quicker than necessary.
2. It sharpens unevenly/angle is wrong. If the angle is wrong, it could either make the blade more prone to chipping by making it too narrow, or not sharp by making it too wide, thus wasting your time. If it's really bad, I guess it could produce an uneven edge too.

Hard to say what is exactly happening in your case. That all said, if it's a cheap blade and your knife is sharper, then use it maybe? Without knowing what it was, I don't know that I'd gamble with a good blade. "Cheap plastic board with a in-built sharpener" is probably a clue that it's cheap though.

Test how long the knife stays sharp, with and without stropping. If it will be dull after a few trivial board impacts, what seems to be sharp was actually just a burr - bad sharpener. If it quickly loses sharpness but can be stropped back again and again, there might be a reasonably stable wire edge, or the sharpener (and the angle it sharpens at) is just ill chosen for that particular blade - lackluster sharpener. If the edge seems stable - above average sharpener.

The problem with almost all such devices is that, even if they sharpen correctly, they still sharpen only a small area at the edge, there being a quite defined line where the sharpened area ends. While that looks good and crisp, it is a disaster in the making when such sharpeners are regularly used on a blade (worst if the edge does not last and the sharpener is utilized more often). Blades are usually either gradually getting thicker towards the spine, or there is a distinct area of a cm or two that steeply thins towards the edge. The border between the sharpened area and the remaining blade will be on a thicker section of the blade after each sharpener use. If the sharpener forces a v shape so firmly that this border will always be ground equally thin, you end up with a geometry similar to a hollow grind instead. Both are bad - you get a knife that, while being good at penetrating into ingredients at the edge, will bind in ingredients, split or plough instead of cut, and take unsafe levels of force to use. Experienced whetstone users often grind off a bit of the metal above the edge line in a steep angle each time they sharpen - even if this looks less "factory new" or might even scratch the blade.

Even with good whetstones, repairing a knife where that transition line has become far too thick easily turns into a difficult job for an hour or more.

Rule of thumb, keep non-wheeled or metal-wheel pull throughs away from knives worth more than $20, and wheeled (ceramic wheels) ones from anything above $50 (MAYBE excepting matched sharpeners that are made for and sold with a knife series).