Troubleshooting Tip: Fix the 'Datasource cannot be used by compliance policy' error when creating custom policies

[vschmitt_FTNT]

Description	This article describes how to investigate and remediate the error 'Datasource cannot be used by compliance policy'. During the creation of a custom compliance policy, the administrator needs to define the datasource that will serve as a source for the query. The resources that are queried for an AWS compliance policy are read from AWS public APIs using multiple API calls to retrieve. Lacework then put each API call into its own datasource and link them via joins to honor the query. Lacework Administrators may sometimes encounter the error 'Datasource cannot be used by compliance policy' upon compliance policy validation.
Scope	Compliance policy, LQL, datasource.
Solution	One reason is that the datasource could not be assessed because the data are missing. This may be for the following reasons: The rate-limit was reached during the data query. Lacework ingestion data has a retry mechanism, but the rate-limit can be hit. Security groups configured on AWS may prevent Lacework to access some data. Another possible reason is that the policy may not be able to assess the compliance policy. For example, if Lacework hit a problem listing all S3 buckets, Lacework will be able to assess a policy only for the S3 buckets that were collected, but not for the others. Consequently, therefore Lacework will be unable to globally assess the policy. These 'primary' data sources (s3 buckets, for instance) represent the top-level list of 'all the resources'. But when a policy's query does not use the primary data source, Lacework cannot apply that logic that determines whether or not the policy itself is assess-able across all the resources.  This is why Lacework validates that the policy does use a primary source. If the administrator is not using a primary source in the query, the administrator will encounter the 'Datasource cannot be used by compliance policy' error. To fix this error, the administrator will need to link the non-primary source with a primary source. Example when not using a primary source, which will lead to the 'Datasource cannot be used by compliance policy' error: { source { LW_CFG_AWS_EC2_IMAGES image, array_to_rows(image.RESOURCE_CONFIG:BlockDeviceMappings) as (volume) } filter { volume:Ebs.Encrypted = "false" // Filter images owned by current account only and image.RESOURCE_CONFIG:OwnerId = image.ACCOUNT_ID } return distinct { image.ACCOUNT_ALIAS, image.ACCOUNT_ID, image.RESOURCE_CONFIG:ImageId as RESOURCE_KEY, image.RESOURCE_REGION, image.RESOURCE_TYPE, image.SERVICE, 'AMIUnencrypted' as COMPLIANCE_FAILURE_REASON } } As LW_CFG_AWS_EC2_IMAGES is not a primary source, the Lacework administrator will have to include the primary source LW_CFG_AWS_EC2_INSTANCES: { source { LW_CFG_AWS_EC2_INSTANCES instance with LW_CFG_AWS_EC2_IMAGES image, array_to_rows(image.RESOURCE_CONFIG:BlockDeviceMappings) as (volume) } filter { volume:Ebs.Encrypted = "false" // Filter images owned by current account only and image.RESOURCE_CONFIG:OwnerId = image.ACCOUNT_ID } return distinct { instance.ACCOUNT_ALIAS, instance.ACCOUNT_ID, instance.RESOURCE_CONFIG:ImageId as RESOURCE_KEY, instance.RESOURCE_REGION, "ec2:image" as RESOURCE_TYPE, instance.SERVICE, 'AMIUnencrypted' as COMPLIANCE_FAILURE_REASON } }