From your description, I'm guessing that you're talking about the battery in the wearlink transmitter (that attaches to the chest strap).
It is a stupid design, with the points of maximum leverage (the outer edges) having least depth to the groove, and with it being made of soft plastic it...