The question is do you feel regret when people talk bad things about you? If yes then you do not understand certain things in life. God in his own wisdom created us differently. We all have different purpose in life.
What you can do in life is quite different from what other person can do. Therefore if you take a step in doing something good for yourself and you are being criticized, you have no reason in feeling regret.
It might be that the one criticizing you envy you, yes he/she want to be the one doing what you are capable of doing and not you. So if you are doing something legally good then don't care about what bad people might say to bring you down.
Just believe in yourself, put any bad comment behind you and work hard for yourself and you shall surely achieve your aim.