I think you need to leave. As in, tell him you need a day off and then go away. Leave him alone with your daughter. Check into a motel, turn off your phone and take a nap. Read novels. Take a bubble bath. Do NOT call him, text him, email him... Control yourself.

Your partner will be forced to swim. He will not sink. Your child may be upset because you are not there but she's with her father. Maybe they will start to bond, which would be great. That would encourage your partner to do more.

Your partner may not take care of your child as well as you could, but your child will be fine. You will come home the next day rested. And your partner may have a new appreciation for you.