When does database indexing become a performance problem?

We all know indexes speed up reads, but I've been noticing in some of our older systems that too many indexes, or ones that are too complex, seem to be slowing down writes a lot. Is there a general rule of thumb, or a specific metric you look at, to know when you've gone too far with indexing? What are the trade-offs you consider when deciding to add or remove an index?