While the infidelity would surely hurt, anyone that would change their opinion of the children they raised because of something like this was a shit parent from the start.