When is making your own ORM a mistake?

Starting a new project, and we're debating whether to use an existing ORM or just build our own. I've seen custom ORMs really shine, making things super specific and fast for certain situations. But, man, I've also seen them turn into total headaches, sucking up developer time just to keep them afloat and add stuff that standard ORMs already do. What have you all experienced? When do you think the perks of a custom ORM are worth the long-term pain and possible bugs, especially when you could just use a proven, ready-made solution?