We have missed the point. Our sin isn't actually against the people around us or even ourselves-although the consequences of our choices can be damaging to both. We have sinned against God and it is our rebellion against Him that causes us the most grief. Even though we know it isn't right to cheat on our spouses, to get drunk every weekend, to gossip about others, to lie, to be divisive in church and to steal, it isn't until we realize that these are transgressions against God and not each other that we stand a chance of true repentance and finding forgiveness and cleansing from Him.
Psalm 51:3-4 "For I know my transgressions, and my sin is always before me. Against You, You only, have I sinned and done what was evil in Your sight"