HA with Load balance with 2 ISPs

Hi not sure if anyone can give me some pointers 

But trying to get a HA solution running 

Before going ahead could anyone let me know if the following drawing would work

we Have

2 ISPs

2 Juniper routers

2 Fortigate 300D FWs

My intention is to connect each FW to have a connection to each router and use LB to push the data to the primary circuit if the primary router fails LB to allow data to goto the secondary router

In case on a Primary FW failure we intend to use HA (Active Passive) to fail over to the seconadry FW and connect to the primary Router if again this fail it will router through secondary router 

Cheers for any advice on insights