Filter Groups Radius Policy

I have multiple groups on my FAC that contain LDAP users, I have the filter turned on on Radius Policies Under Groups, and selected all the groups, this works fine as they have a Radius Attribute for the SSLVPN portal that looks for remote user groups,

The FAC is limited to 10 groups, so I have now turned this filter off, so it shouldnt filter at all, and let users and groups in? but it has stopped allowing eveything when I toggle this off? why is that? I am going to have to create remote users instead of groups because of this limit.