When the new person in your life still has feelings for their ex

What happens when you fall for someone who might still have feelings for the person they were with? Is it a relationship that can survive? Or will the fear of them returning to the person they cared about before you be too much for you to keep it going?