Scans failing can cause a soft-lock
J-Stuff opened this issue · comments
Describe the bug
Scans often fail and softlock Fireshare until fireshare.lock is deleted or the container is restarted.
To Reproduce
Steps to reproduce the behavior:
- Wait for a scan to initiate
- Sometimes (for what seems to be no reason) the scan will raise an exception and softlock the app
Expected behavior
The scan continues as normal
Exception
2023-06-06 07:41:03.064413+00:002023-06-06 19:41:03,064 INFO schedule.fireshare_scan:12 | Starting scheduled scan...
2023-06-06 07:41:03.649948+00:002023-06-06 19:41:03,649 INFO __init__.update_config:27 | Validating configuration file...
2023-06-06 07:41:03.673969+00:002023-06-06 19:41:03,670 INFO __init__.update_config:27 | Validating configuration file...
2023-06-06 07:41:03.683975+00:002023-06-06 19:41:03,683 INFO cli.scan_videos:50 | Scanning /videos for .mp4, .mov, .webm video files
2023-06-06 07:41:04.228177+00:002023-06-06 19:41:04,227 INFO cli.scan_videos:78 | Adding new Video d76bbe5d3276a41d08fff725f5f7e66f at SCP Secret Laboratory/SCP Secret Laboratory 2023.06.06 - 19.36.19.06.mp4 (created 2023-06-06T19:41:03.801988, updated 2023-06-06T19:41:03.808220)
2023-06-06 07:41:05.194111+00:002023-06-06 19:41:05,193 INFO cli.scan_videos:96 | Linking ../../videosSCP Secret LaboratorySCP Secret Laboratory 2023.06.06 - 19.36.19.06.mp4 --> /processed/video_links/d76bbe5d3276a41d08fff725f5f7e66f.mp4
2023-06-06 07:41:05.221758+00:002023-06-06 19:41:05,221 INFO cli.scan_videos:106 | Verifying 143 video files still exist...
2023-06-06 07:41:05.232449+00:002023-06-06 19:41:05,232 INFO __init__.update_config:27 | Validating configuration file...
2023-06-06 07:41:05.252232+00:002023-06-06 19:41:05,252 INFO cli.sync_metadata:145 | Found 1 videos without metadata
2023-06-06 07:41:05.430619+00:00--- Logging error ---
2023-06-06 07:41:05.432046+00:00Traceback (most recent call last):
2023-06-06 07:41:05.432226+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/util.py", line 47, in get_media_info
2023-06-06 07:41:05.432277+00:00data = json.loads(sp.check_output(cmd.split()).decode('utf-8'))
2023-06-06 07:41:05.432288+00:00File "/usr/local/lib/python3.9/subprocess.py", line 424, in check_output
2023-06-06 07:41:05.432295+00:00return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
2023-06-06 07:41:05.432302+00:00File "/usr/local/lib/python3.9/subprocess.py", line 528, in run
2023-06-06 07:41:05.432310+00:00raise CalledProcessError(retcode, process.args,
2023-06-06 07:41:05.432317+00:00subprocess.CalledProcessError: Command '['ffprobe', '-v', 'quiet', '-print_format', 'json', '-show_entries', 'stream', '/processed/video_links/d76bbe5d3276a41d08fff725f5f7e66f.mp4']' returned non-zero exit status 1.
2023-06-06 07:41:05.432366+00:002023-06-06T07:41:05.432366309Z
2023-06-06 07:41:05.432393+00:00During handling of the above exception, another exception occurred:
2023-06-06 07:41:05.432414+00:002023-06-06T07:41:05.432414324Z
2023-06-06 07:41:05.432423+00:00Traceback (most recent call last):
2023-06-06 07:41:05.432430+00:00File "/usr/local/lib/python3.9/logging/__init__.py", line 1083, in emit
2023-06-06 07:41:05.432437+00:00msg = self.format(record)
2023-06-06 07:41:05.432444+00:00File "/usr/local/lib/python3.9/logging/__init__.py", line 927, in format
2023-06-06 07:41:05.432451+00:00return fmt.format(record)
2023-06-06 07:41:05.432458+00:00File "/usr/local/lib/python3.9/logging/__init__.py", line 663, in format
2023-06-06 07:41:05.432465+00:00record.message = record.getMessage()
2023-06-06 07:41:05.432471+00:00File "/usr/local/lib/python3.9/logging/__init__.py", line 367, in getMessage
2023-06-06 07:41:05.432487+00:00msg = msg % self.args
2023-06-06 07:41:05.432496+00:00TypeError: not all arguments converted during string formatting
2023-06-06 07:41:05.432504+00:00Call stack:
2023-06-06 07:41:05.433741+00:00File "/usr/local/bin/fireshare", line 8, in <module>
2023-06-06 07:41:05.433772+00:00sys.exit(cli())
2023-06-06 07:41:05.433786+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
2023-06-06 07:41:05.433794+00:00return self.main(*args, **kwargs)
2023-06-06 07:41:05.433801+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
2023-06-06 07:41:05.433808+00:00rv = self.invoke(ctx)
2023-06-06 07:41:05.433815+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
2023-06-06 07:41:05.433822+00:00return _process_result(sub_ctx.command.invoke(sub_ctx))
2023-06-06 07:41:05.433828+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
2023-06-06 07:41:05.433877+00:00return ctx.invoke(self.callback, **ctx.params)
2023-06-06 07:41:05.433887+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2023-06-06 07:41:05.433905+00:00return __callback(*args, **kwargs)
2023-06-06 07:41:05.433923+00:00File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
2023-06-06 07:41:05.433930+00:00return f(get_current_context(), *args, **kwargs)
2023-06-06 07:41:05.433938+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/cli.py", line 265, in bulk_import
2023-06-06 07:41:05.433945+00:00ctx.invoke(sync_metadata)
2023-06-06 07:41:05.433951+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2023-06-06 07:41:05.433958+00:00return __callback(*args, **kwargs)
2023-06-06 07:41:05.433965+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/cli.py", line 149, in sync_metadata
2023-06-06 07:41:05.433972+00:00info = util.get_media_info(vpath)
2023-06-06 07:41:05.433978+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/util.py", line 50, in get_media_info
2023-06-06 07:41:05.433985+00:00logger.warn('Could not extract video info', ex)
2023-06-06 07:41:05.434001+00:00Message: 'Could not extract video info'
2023-06-06 07:41:05.434012+00:00Arguments: (CalledProcessError(1, ['ffprobe', '-v', 'quiet', '-print_format', 'json', '-show_entries', 'stream', '/processed/video_links/d76bbe5d3276a41d08fff725f5f7e66f.mp4']),)
2023-06-06 07:41:05.435199+00:00Traceback (most recent call last):
2023-06-06 07:41:05.435234+00:00File "/usr/local/bin/fireshare", line 8, in <module>
2023-06-06 07:41:05.435577+00:00sys.exit(cli())
2023-06-06 07:41:05.435612+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
2023-06-06 07:41:05.435834+00:00return self.main(*args, **kwargs)
2023-06-06 07:41:05.435855+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
2023-06-06 07:41:05.436120+00:00rv = self.invoke(ctx)
2023-06-06 07:41:05.436143+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
2023-06-06 07:41:05.436533+00:00return _process_result(sub_ctx.command.invoke(sub_ctx))
2023-06-06 07:41:05.436557+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
2023-06-06 07:41:05.436917+00:00return ctx.invoke(self.callback, **ctx.params)
2023-06-06 07:41:05.436941+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2023-06-06 07:41:05.437118+00:00return __callback(*args, **kwargs)
2023-06-06 07:41:05.437143+00:00File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
2023-06-06 07:41:05.437247+00:00return f(get_current_context(), *args, **kwargs)
2023-06-06 07:41:05.437270+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/cli.py", line 265, in bulk_import
2023-06-06 07:41:05.437423+00:00ctx.invoke(sync_metadata)
2023-06-06 07:41:05.437455+00:00File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
2023-06-06 07:41:05.437622+00:00return __callback(*args, **kwargs)
2023-06-06 07:41:05.437642+00:00File "/usr/local/lib/python3.9/site-packages/fireshare/cli.py", line 150, in sync_metadata
2023-06-06 07:41:05.437724+00:00vcodec = [i for i in info if i['codec_type'] == 'video'][0]
2023-06-06 07:41:05.437746+00:00TypeError: 'NoneType' object is not iterable
Desktop (please complete the following information):
- Browser - Chrome (doesn't really matter)
Server:
- TrueNas Scale Running FireShare in a k3s pod
- Files uploaded via an SMB share to my server. Where fireshare can access them via a mount
So this tends to happen if there is a non video file in the folder that ffmpeg tries to load or a corrupt video file.
2023-06-06 07:41:05.434001+00:00Message: 'Could not extract video info'
I would double check your mount point for videos and make sure there isn't any hidden non-video files. If there are, remove them and try a scan again and it should work. If there isn't, you may have a corrupt video file somewhere.
If you look closely at the stacktrace you'll see that the file ffmpeg is having an issue with is this one.
2023-06-06 07:41:05.434012+00:00Arguments: (CalledProcessError(1, ['ffprobe', '-v', 'quiet', '-print_format', 'json', '-show_entries', 'stream', '/processed/video_links/d76bbe5d3276a41d08fff725f5f7e66f.mp4']),)
/processed/video_links/d76bbe5d3276a41d08fff725f5f7e66f.mp4
Thats a symlink that fireshare creates that points to the actual file. Find the file that it is pointing to and inspect it to make sure that it is indeed a video file and isn't corrupt or maybe something else wrong with it.
To figure out where that symlink is pointing you can run this command against it.
stat d76bbe5d3276a41d08fff725f5f7e66f.mp4
I think I figured out the problem.
My scan interval was set to every minute. Fireshare was scanning files that were in the process of being uploaded to the SMB share (over the Network). After setting my scan interval to 120 minutes, the malformed file error seems to have stopped
@J-Stuff while that is a decent fix, you can still run into issues in the event you start uploading a file right before the scan starts.
If you really want to make sure this issue never happens again, I would upload files to a folder that is not watched by fireshare first and then move the files (either manually or with some script) so that fireshare doesn't attempt to scan a partially moved file.