[Bug]: EWF dumps can't be mounted by worker

Question

[Bug]: EWF dumps can't be mounted by worker

Slaynot opened this issue 3 months ago · comments

What steps will reproduce the bug?

Build the docker images from the master branch
Submit a request for processing a .E01 dump

turbinia-client submit ewfdisk --source_path /path/to/dump.E01

What is the expected behavior?

The .E01 should be mounted sucessfully with ewfmount by the worker and processed.

What do you see instead?

The worker fails to mount the .E01 image because it can't find the ewfmount command.

2024-02-18 20:59:55 [INFO] Turbinia version: 20231116.2
2024-02-18 20:59:57 [INFO] Disabling non-allowlisted jobs configured to be disabled in the config file: dfdeweyjob, volatilityjob, hindsightjob
2024-02-18 20:59:57 [INFO] Registering job timeouts.
2024-02-18 20:59:57 [INFO] Disabling non-allowlisted jobs configured to be disabled in the config file: dfdeweyjob, volatilityjob, hindsightjob
2024-02-18 20:59:57 [INFO] Performing docker dependency check.
2024-02-18 20:59:57 [WARNING] The job dfdeweyjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job fileartifactextractionjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job hindsightjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job volatilityjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [INFO] Performing system dependency check.
2024-02-18 20:59:57 [WARNING] The job dfdeweyjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job fileartifactextractionjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job hindsightjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [WARNING] The job volatilityjob was not found or has been disabled. Skipping dependency check...
2024-02-18 20:59:57 [INFO] Registering job timeouts.
2024-02-18 20:59:57 [INFO] Dependency check complete. The following jobs are enabled for this worker: binaryextractorjob,bulkextractorjob,containerdenumerationjob,dockercontainersenumerationjob,filesystemtimelinejob,finalizerequestjob,fsstatjob,grepjob,httpaccesslogextractionjob,httpaccessloganalysisjob,jenkinsanalysisjob,jupyterextractionjob,jupyteranalysisjob,linuxaccountanalysisjob,yaraanalysisjob,partitionenumerationjob,photorecjob,plasojob,postgresacctanalysisjob,psortjob,redisanalysisjob,redisextractionjob,linuxsshanalysisjob,sshdanalysisjob,sshdextractionjob,stringsjob,tomcatextractionjob,tomcatanalysisjob,windowsaccountanalysisjob,wordpresscredsanalysisjob,statjob
2024-02-18 20:59:57 [INFO] Running Turbinia Celery Worker.
 
 -------------- celery@bd567dcc870c v5.3.6 (emerald-rush)
--- ***** ----- 
-- ******* ---- Linux-6.1.0-18-amd64-x86_64-with-glibc2.35 2024-02-18 20:59:58
- *** --- * --- 
- ** ---------- [config]
- ** ---------- .> app:         turbinia:0x7f8c82967be0
- ** ---------- .> transport:   redis://redis:6379//
- ** ---------- .> results:     redis://redis/
- *** --- * --- .> concurrency: 1 (solo)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** ----- 
 -------------- [queues]
                .> turbinia-instance1 exchange=turbinia-instance1(direct) key=turbinia-instance1
                

[tasks]
  . task_runner

[2024-02-18 20:59:58,463: INFO/MainProcess] Connected to redis://redis:6379//
[2024-02-18 20:59:58,480: INFO/MainProcess] celery@bd567dcc870c ready.
[2024-02-18 21:00:37,656: INFO/MainProcess] Task task_runner[7b8129cd-c6f3-462d-b88c-ee110e00cb44] received
2024-02-18 21:00:37 [INFO] Updating task BinaryExtractorTask in Redis
2024-02-18 21:00:37 [INFO] Starting Task BinaryExtractorTask 6183112248b24d158fb1f5d97b28284d
2024-02-18 21:00:37 [INFO] Getting evidence size via ['blockdev', '--getsize64', '/path/to/dump.E01']
2024-02-18 21:00:37 [INFO] Starting preprocessor for evidence /path/to/dump.E01
2024-02-18 21:00:37 [INFO] Running: sudo ewfmount -X allow_other /path/to/dump.E01 /tmp/turbinia-mounts/turbinia6xb63l9h
sudo: ewfmount: command not found
2024-02-18 21:00:37 [ERROR] Error running preprocessor for /path/to/dump.E01: Could not mount directory Command '['sudo', 'ewfmount', '-X', 'allow_other', '/path/to/dump.E01', '/tmp/turbinia-mounts/turbinia6xb63l9h']' returned non-zero exit status 1.
2024-02-18 21:00:37 [INFO] Preprocessing evidence /path/to/dump.E01 is complete, and evidence is in state [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]
2024-02-18 21:00:37 [ERROR] BinaryExtractorTask Task failed with exception: [Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.]
2024-02-18 21:00:37 [ERROR] Traceback (most recent call last):
  File "/home/turbinia/turbinia/workers/__init__.py", line 1064, in run_wrapper
    self.evidence_setup(evidence)
  File "/home/turbinia/turbinia/workers/__init__.py", line 573, in evidence_setup
    raise TurbiniaException(
turbinia.TurbiniaException: Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.

2024-02-18 21:00:37 [ERROR] BinaryExtractorTask Task failed with exception: [Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.]
2024-02-18 21:00:37 [INFO] Traceback (most recent call last):
  File "/home/turbinia/turbinia/workers/__init__.py", line 1064, in run_wrapper
    self.evidence_setup(evidence)
  File "/home/turbinia/turbinia/workers/__init__.py", line 573, in evidence_setup
    raise TurbiniaException(
turbinia.TurbiniaException: Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.

2024-02-18 21:00:37 [INFO] Result check: Successful
2024-02-18 21:00:37 [WARNING] Trying last ditch attempt to close result
2024-02-18 21:00:37 [INFO] Trying last ditch attempt to close result
2024-02-18 21:00:37 [INFO] Task Result was auto-closed from task executor on bd567dcc870c likely due to previous failures.  Previous status: [BinaryExtractorTask Task failed with exception: [Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.]]
2024-02-18 21:00:37 [INFO] Task Result was auto-closed from task executor on bd567dcc870c likely due to previous failures.  Previous status: [BinaryExtractorTask Task failed with exception: [Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being processed by Task BinaryExtractorTask requires Evidence to be in state ATTACHED, but earlier pre-processors may have failed.  Current state is [MOUNTED: False, ATTACHED: False, DECOMPRESSED: False, CONTAINER_MOUNTED: False]. See previous logs for more information.]]
2024-02-18 21:00:37 [INFO] Starting postprocessor for evidence /path/to/dump.E01
2024-02-18 21:00:37 [INFO] Result check: Successful
[2024-02-18 21:00:37,697: INFO/MainProcess] Task task_runner[7b8129cd-c6f3-462d-b88c-ee110e00cb44] succeeded in 0.04092840100929607s: {'closed': True, 'evidence': [], 'evidence_size': None, 'input_evidence': None, 'id': 'c0e001455b4f454f9214ef40388fe319', 'job_id': 'c8883ae34caf4723b822ad332787eb80', 'base_output_dir': '/evidence', 'request_id': 'be2348a9cf06460cb98d47ea8f1ae269', 'task_id': '6183112248b24d158fb1f5d97b28284d', 'task_name': 'BinaryExtractorTask', 'requester': 'user_unspecified', 'output_dir': '/evidence/be2348a9cf06460cb98d47ea8f1ae269/1708290037-6183112248b24d158fb1f5d97b28284d-BinaryExtractorTask', 'report_data': None, 'report_priority': Priority.MEDIUM, 'run_time': None, 'saved_paths': ['/evidence/be2348a9cf06460cb98d47ea8f1ae269/1708290037-6183112248b24d158fb1f5d97b28284d-BinaryExtractorTask/worker-log.txt'], 'successful': False, 'status': 'Task Result was auto-closed from task executor on bd567dcc870c likely due to previous failures.  Previous status: [BinaryExtractorTask Task failed with exception: [Evidence EwfDisk:/path/to/dump.E01:/path/to/dump.E01 being...', , ...}
...

Additional information

No response

Aaron Peterson · Answer 1 · Thu Feb 22 2024 14:55:48 GMT+0800 (China Standard Time)

@Slaynot Thanks for fixing this!