Yes, God did put within each of us a sense of right and wrong — what we usually call our conscience.
The problem is that our conscience — that inner sense of right and wrong — has been corrupted by sin.
Our conscience can alert us to right and wrong. But it’s not always a dependable guide — and it can even deceive us if we aren’t careful. That’s why we need Christ, who lives within us when we welcome him into our hearts. It’s also why we need the Bible, which God has given to guide us.