Skip to main content
MohamedYasser
MohamedYasser
New Member
October 14, 2024
Solved

Mock Output limitation

  • October 14, 2024
  • 5 replies
  • 3031 views

Hello Community, Is there any limitation on the mock output in the playbooks. I was using the results of couple of SIEM queries as a mock output but the playbook didn't work.

Appreciate your assistance.

Thank you.

    Best answer by tkanade
    Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.

    This error indicates that the data being passed is too large for automation framework to handle hence causing no execution of playbook. Can you reduce mock data to single query output and try.
    There is way to increase this limit as well, you can check this -- 
    https://community.fortinet.com/t5/FortiSOAR-Knowledge-Base/Technical-Tip-Playbooks-fail-to-run-on-heavy-records/ta-p/190194

    5 replies

    anarula
    Staff
    anarula
    Staff
    October 14, 2024

    @MohamedYasser , could you please share some examples/details of the problems you faced?

    MohamedYasser
    MohamedYasserAuthor
    New Member
    October 14, 2024

    @anarula , I was using couple of SIEM queries each returning a lot of events ranging between 5k-30k event, trying to do some operations with those events in the playbook, so after running searches and getting results I took the output to be used as mock output later on. That's when I noticed the playbook couldn't run.

    anarula
    Staff
    anarula
    Staff
    October 14, 2024

    Ok, so far we have established that event data size is on higher side and perhaps causing a failure. We need to further know, what is the failure/error that you are seeing .. and perhaps share the playbook (with the mock output embedded) so that we can attempt to replicate the problem.

    MohamedYasser
    MohamedYasserAuthor
    New Member
    October 14, 2024

    There is no failure/error showing but when I trigger the playbook it loads then nothing happens the playbook does not run. Unfortunately sharing mock output isn't doable as the data returned is confidential. But, if possible, I can share dummy data with the same amount of results from each query.

    anarula
    Staff
    anarula
    Staff
    October 14, 2024

    yes, dummy data is fine. Share the PB with along with dummy mock data

    MohamedYasser
    MohamedYasserAuthor
    New Member
    October 14, 2024

    I have attached zipped file with dummy mock from 6 queries used with the same amount of data returning from each query, and the playbook used.

    tkanade
    Staff
    tkanade
    Staff
    October 14, 2024

    Can you check sealab.log at /var/log/cyops/cyops-workflow/sealab.log, do we see any max data size exceptions ? 

    MohamedYasser
    MohamedYasserAuthor
    New Member
    October 14, 2024

    I have seen this message repeated multiple times but it's not showing what may be the cause:

    [2024-10-14 07:06:19] ERROR [django.security.RequestDataTooBig:124] Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE. Traceback (most recent call last):   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/core/handlers/exception.py", line 55, in inner     response = get_response(request)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/core/handlers/base.py", line 197, in _get_response     response = wrapped_callback(request, *callback_args, **callback_kwargs)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view     return view_func(*args, **kwargs)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/generic/base.py", line 104, in view     return self.dispatch(request, *args, **kwargs)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/utils/decorators.py", line 46, in _wrapper     return bound_method(*args, **kwargs)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/views/decorators/csrf.py", line 56, in wrapper_view     return view_func(*args, **kwargs)   File "sealab/workflow/views.py", line 392, in workflow.views.StartView.dispatch   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch     response = self.handle_exception(exc)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception     self.raise_uncaught_exception(exc)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception     raise exc   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 497, in dispatch     self.initial(request, *args, **kwargs)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 414, in initial     self.perform_authentication(request)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/views.py", line 324, in perform_authentication     request.user   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 231, in user     self._authenticate()   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 384, in _authenticate     user_auth_tuple = authenticator.authenticate(self)   File "sealab/auth_ogre/schemes.py", line 183, in auth_ogre.schemes.OgreAuthentication.authenticate   File "sealab/auth_ogre/schemes.py", line 71, in auth_ogre.schemes.HmacAuthenticationScheme.authenticate   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/rest_framework/request.py", line 421, in __getattr__     return getattr(_request, attr)   File "/opt/cyops-workflow/.env/lib64/python3.9/site-packages/django/http/request.py", line 354, in body     raise RequestDataTooBig( django.core.exceptions.RequestDataTooBig: Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.
    tkanade
    Staff
    tkanadeAnswer
    Staff
    October 14, 2024
    Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.

    This error indicates that the data being passed is too large for automation framework to handle hence causing no execution of playbook. Can you reduce mock data to single query output and try.
    There is way to increase this limit as well, you can check this -- 
    https://community.fortinet.com/t5/FortiSOAR-Knowledge-Base/Technical-Tip-Playbooks-fail-to-run-on-heavy-records/ta-p/190194

      MohamedYasser
      MohamedYasserAuthor
      New Member
      October 17, 2024

      This article was really useful, but I still can't figure the limit to use in this playbook already tried increasing the limit but didn't work I guess the mock is much more than I thought. So we will try to find a way to lower the amount of mock we are using at the moment.

      Thanks for you help

      Terms & ConditionsAccessibility statement