Separate WAN for each VLAN

Hi All,

I'm having an issue with a Fortigate 80F running 6.4.7. The setup we are trying to achieve is something like below:

WAN1 - Primary cable modem

WAN2 - Secondary cable modem (both same provider)

VLANs 1-4 go out through WAN1 only

VLAN 5 goes out through WAN2 only

Both WAN have their static routes programmed from 0.0.0.0 to their primary gateways.

I'm trying to figure out if we need to adjust the Administrative Distance or the Priority to make them work, or if we need to make static routes specific to each VLAN subnet. Usually we would set these two separate WANs in load balancing but this client specifically wants the one VLAN only on the secondary WAN and everything else on the primary.