Hm... I admit that it's quite difficult for me to relate to your childhood (and, for that matter, your current life, as I'm a young woman and much more at peace with being single, rather than in a committed relationship with children). While I would not claim to have experienced a harsher life in general, in fact I know plenty of people that were or still are worse off than I ever will be, I faced my own set of struggles, including mentally-absent parents but also some long-lasting ostracism and a general sense of being misunderstood by nearly all the people I used to be surrounded by. It took me years – and several rather unhelpful therapists repeating this old pattern and my rare moments of de-realization – to come to the conclusion that it wasn't me responsible for many misunderstandings and that some people, well, simply suck. Because no matter what I did, it always triggered a negative response from someone.
While all of this was incredibly painful and certainly damaging, I've grown accustomed to it and even embraced it to a certain degree, as it is often much better to feel something (even years later, when such a a realization hits you all of the sudden), rather than nothing the entire time, which I did for a few years. Due to this, on the other hand, I've made the decision to never have kids. I turned out to be capable to make it through and process certain hardships, yet my potential child may not be as mentally strong – and I've had to work with traumatized children dealing with broken homes and misunderstandings on behalf of teachers, (adoptive) parents and other authorities, so seeing them and how all of their issues stem from adults that base their actions on some sort of "good intentions" (even those that clearly were abusive) left an additional mark.
No parent is perfect, of course, and it's clear you worry about how you treat your children. This already makes you a better parent than those that see no wrong in ~any~ of their actions. Continue to make the best our of your situation, even if it may be difficult at times. Just remember that some concerns may be solved with some calm and clear communication.