4 year controlling relationship? but i just cant keep away. any advice?....i am so sad and just sit and cry?

Ive become aware of just how often my partner picks on me and tells me im wrong. yesterday i was distraught over some bad news, i wasnt very talkative and was to upset to try and get my brain in gear when he asked me something. he went quiet. and then said i was not answering simple questions and this is what he has had to deal with for 4 years. he swung the whole thing round to be about what i was doing wrong. i was upset and not able to really talk. he could see that clearly. he then moaned at me for the rest of the night telling me all i do wrong. that he cant deal with me etc! i really needed him to just be there. but instead made me feel worse. this happens alot. im finding that i listen to him, i sit and take all those bad words. if i shout back he ends it and then i beg him to take me back! he thinks i cheat- i never have ever. i feel i cant let go. he has ended it again and im fighting myself. its a pattern. why cant i let go? whats up with me. im constantly trying to prove to him that im good person