First, I am very sorry that you have had such a difficult childhood. Your thought about affection now being a challenge for you is very insightful. A key is to start, when you are ready, to forgive your mother. Let a sense of compassion for your mother come to you, even if this develops slowly. Try to see how emotionally wounded your mother was to have not given you affection.
As you see her woundedness, try to be aware of even a small amount of compassion building in your heart for her. This compassion, emerging out of forgiving your mother, can be the building-block for compassion toward other adults now in your life. That compassion will help you to build stronger, more trusting relationships. If you think about it, you now have the opportunity to be a deeply compassionate person because of your past pain.