Having to reboot ISP DSL/Cable Modem to allow VPN Traffic??

We have ran into this situation many times. We have sites where their Fortigate 60D's (firmware 5.6.3) just stop passing traffic through VPN Tunnels. At first we would rebuild the tunnels on both sides (No, I have inherited established direct site to site VPN, no BGP/ADVPN), but even after rebuilding the tunnels on both sides, this wouldn't always work. So what we found was that simply restarting the ISP's modem (most of our smaller branches have DSL) the tunnels would miraculously start communicating again. The site would never have internet issues, just VPN issues. Why? What is it about these modems, that these fortigates have trouble with? 

Thank you.