Just like most good qualities and habits, empathy is a double edged sword.
The ugly part of empathy is that it makes things messy. Instead of being able to stay inside your own heart and mind and operate based on that, it forces you outside of that bubble and into other peoples’ ideas, opinions, and feelings. This means instead of brushing someone off as just wrong, you start to see where they’re coming from, which makes it a lot harder to hate your enemies. Instead, you’re stuck relating to them, which means there has to be a lot more conversation and figuring stuff out. (You’re right, this isn’t ugly — but it forces you to confront ugly ideas, and ugly parts of yourself.)
The bad part is, this can make it hard to cut destructive people off. When you understand the reasoning, however twisted, behind someone’s abusive, manipulative, or otherwise harmful behavior, it’s harder to just leave them behind. Because you can empathize a bit, you want to be able to help them somehow, but sometimes that just can’t happen. People make their own decisions, and sometimes being empathetic with those people makes their bad decisions hurt more.
The good part, however, far outweighs any bad. The good part is that it makes any relationship you build stronger, because you are able to base it on an emotional connection. It makes arguments less heated, because you are able to reach out to the other person with understanding rather than hate, acknowledgement rather than dismissal.