transparent fortigate firewall best practice with two ASA configured in failover mode

i have a setup where i have two ASA firewalls facing the wan in failover mode, and between the core switch and the ASAs i have fortigate running 5.4.5 OS in transparent mode, i had ipv4 virtual pairs but they consume a lot of memory, so i wanted  to minimize the load over the memory and changed to IPv4 policies only, but a lot of rules sometimes match implicit deny cause traffic is sent to mgmt port.

"ASA1 on interface 12 on fgt , ASA2 on interface 10 on fgt , core switch is connected on interface 9 and 11 traffic coming from vpn users from ASAs they match implicit deny if i don't chose in the destination interface any. can i somehow force all the traffic coming from 12 and 10 to go only to 9 and 11 without using virtual pair and vice versa.