Internet Traffic Stops

I have a Fortinet 100D firewall with two Internet connections connected to the firewall with the following port setup.    Port 16 is the fiber connection to fiber provider at 1GB.   On Wan1 I have a 100MB cable modem connection.   In my static route settings I have the following.

Fiber = 0.0.0.0/0.0.0.0 Default Gateway X.X.X.X Admin Distance = 1 and Priority = 0

Cable = 0.0.0.0/0.0.0.0 Default Gateway X.X.X.X Admin Distance = 1 and Priority = 10

Randomly we loose internet connection incoming and outgoing for approximately 10-20 seconds and it just starts to go again. If I have a continuous ping to 8.8.8.8 when this happens I get a request time out a couple of times and then reply from 172.16.XX.XX (Internal IP of Firewall) Destination unreachable a couple of times and then request times out and then back with pings.  

Pinging the outside IP addresses of the firewall and some server from a different location and Internet provider when we loose internet I get request times out on the other end.  I also have a ping going to the backup cable modem link and I don't loose a ping at all while the fiber connection goes up and down.  

I believe this might be a  firewall problem because I have a laptop with an outside IP address from the internet provider and have a continuous ping going to 8.8.8.8 and it never drops a ping when we loose the internet connection.

I have already replaced the cables from the Internet connection to the firewall and from the firewall to the switch and still the same problem.    

To get VIP's working on the fiber was had to set gratuitous arp on the VIP to make them work with the fiber connection, but not on the cable connection.    

It seems all of this started when we added the fail-over link.  

Any help in this matter would be appreciated.