Mock Output limitation

[MohamedYasser]

Hello Community, Is there any limitation on the mock output in the playbooks. I was using the results of couple of SIEM queries as a mock output but the playbook didn't work.

Appreciate your assistance.

Thank you.

[anarula]

@MohamedYasser , could you please share some examples/details of the problems you faced?

[MohamedYasser]

@anarula , I was using couple of SIEM queries each returning a lot of events ranging between 5k-30k event, trying to do some operations with those events in the playbook, so after running searches and getting results I took the output to be used as mock output later on. That's when I noticed the playbook couldn't run.

[anarula]

Ok, so far we have established that event data size is on higher side and perhaps causing a failure. We need to further know, what is the failure/error that you are seeing .. and perhaps share the playbook (with the mock output embedded) so that we can attempt to replicate the problem.

[MohamedYasser]

There is no failure/error showing but when I trigger the playbook it loads then nothing happens the playbook does not run. Unfortunately sharing mock output isn't doable as the data returned is confidential. But, if possible, I can share dummy data with the same amount of results from each query.

[anarula]

yes, dummy data is fine. Share the PB with along with dummy mock data

[MohamedYasser]

I have attached zipped file with dummy mock from 6 queries used with the same amount of data returning from each query, and the playbook used.

[tkanade]

Can you check sealab.log at /var/log/cyops/cyops-workflow/sealab.log, do we see any max data size exceptions ? 

[MohamedYasser]

I have seen this message repeated multiple times but it's not showing what may be the cause:

[2024-10-14 07:06:19] ERROR [django.security.RequestDataTooBig:124] Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.
Traceback (most recent call last):
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view
    return view_func(*args, **kwargs)
  File "sealab/workflow/views.py", line 392, in workflow.views.StartView.dispatch
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
    raise exc
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 497, in dispatch
    self.initial(request, *args, **kwargs)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 414, in initial
    self.perform_authentication(request)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 324, in perform_authentication
    request.user
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 231, in user
    self._authenticate()
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 384, in _authenticate
    user_auth_tuple = authenticator.authenticate(self)
  File "sealab/auth_ogre/schemes.py", line 183, in auth_ogre.schemes.OgreAuthentication.authenticate
  File "sealab/auth_ogre/schemes.py", line 71, in auth_ogre.schemes.HmacAuthenticationScheme.authenticate
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 421, in __getattr__
    return getattr(_request, attr)
  File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/http/request.py", line 354, in body
    raise RequestDataTooBig(
django.core.exceptions.RequestDataTooBig: Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.

[tkanade · Accepted solution]

Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.

This error indicates that the data being passed is too large for automation framework to handle hence causing no execution of playbook. Can you reduce mock data to single query output and try.
There is way to increase this limit as well, you can check this -- 
https://community.fortinet.com/t5/FortiSOAR-Knowledge-Base/Technical-Tip-Playbooks-fail-to-run-on-he...