Slow Backups and File transfers with Fortigate

Hello, we have noticed backups and file transfers over SSH are taking quite a bit long since moving from Cisco. 20min before, now 2 hours.

I noticed we have SSL-Inspection turned on with one vdom but it's only inspecting 443. We have IPS monitoring only. I'm wondering if anyone has some ideas what to check here. I'm fairly new to these. The switchports and ports on the Fortigates have no errors.

Thanks