when should you reconsider your real-time strategy? like, maybe when its not working?

ive been working on a few projects recently that require real-time updates. Weve handled some of it with standard polling, but its starting to feel inefficient and clunky for certain features. Were thinking about moving towards WebSockets or maybe even Server-Sent Events SSE. For those who have made this transition, what were the biggest challenges you faced? Are there specific types of applications or user interactions where WebSockets or SSE are a clear win over polling, beyond just frequent updates?