If a point exists in your network design, where it's failure will cause the network to go down you have a Single Point of Failure
The idea is to eliminate or at least mitigate Single Points of Failure
The most effective solution to the Single Point of Failure issue is the use of Redundant components