So, how do you make sure messages aren't processed more than once in distributed systems?

hey everyone, ive been running into a bit of a headache lately with message queues in our microservices. were using Kafka, and while its great for decoupling, ensuring that messages are processed exactly once, or at least handled gracefully if they arrive more than once, is proving tricky. weve got several critical services that shouldnt perform an action twice, like charging a customer. what strategies or patterns are you using to guarantee idempotency when producing and consuming messages? im looking for practical advice and common pitfalls to avoid.