[Bug] Initial mkvpropedit changes the file modification time
arabcoders opened this issue · comments
Please put plugin requests/bugs at: https://github.com/HaveAGitGat/Tdarr_Plugins
Describe the bug
The bug occurs if the file ffprobe doesn't have STATISTICS already generated, it caused by using --add-track-statistics-tags used probably as running mkvpropedit by itself doesn't cause side effects
To Reproduce
1- get a video file without STATISTICS already generated.
2- take snapshot of the stat command, mainly the modify time.
3- scan it into tdarr.
4- re-run stat it will have updated a/c/m time.
Expected behavior
I have plugin that restore original date times, but due to this bug the plugin is picking up the modified time after tdarr already ran mkvpropedit.
So, I would propose, to take copy of the current datetime before running analysis on file, and restore them if they are different than what the file report after running the analysis. i.e.
const original_stat = fs.stat(file, force: true)
const mkv_data = mkvpropedit.run(file)
const exiftool_data = exittool.run(file)
if (fs.stat(file, force: true) > original_stat){
touch.sync(file, {
time: original_stat,
force: true
})
}The code is just to communicate the idea. This mainly concerns mkv container files, other file type doesn't have the same problem as mkvpropedit won't run/work on them.
Also, i think there should be an option on initial scan to respect the existing tags.BPS if it's already exists and do not run
mkvpropedit --add-track-statistics-tags unless the file actually went through process stage. This should dramatically speed up initial scan process, as calculating bitrate for thousands of files is time consuming and resources intensive, right now the load my server load average: 17.33, 18.90, 18.32 due to this re-scanning.
Some logs
2024-06-29T13:58:14.547Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Fetching plugin data from server
2024-06-29T13:58:14.547Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Scanning original library file
2024-06-29T13:58:14.548Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:{
2024-06-29T13:58:14.548Z "exifToolScan": true,
2024-06-29T13:58:14.548Z "mediaInfoScan": true,
2024-06-29T13:58:14.548Z "closedCaptionScan": false
2024-06-29T13:58:14.548Z }
2024-06-29T13:58:14.548Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Updating file properties using mkvpropedit. This may take a long time for large files.
2024-06-29T13:58:20.556Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Updated file properties using mkvpropedit
2024-06-29T13:58:20.556Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Requesting Tdarr Server to scan file
2024-06-29T13:58:24.609Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Scan complete
2024-06-29T13:58:24.609Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Original file scanned
2024-06-29T13:58:24.610Z Nqqu3uHpG:Node[tdarr_node_nvidia]:Worker[big-bonobo]:Source file to work on determined:/pool/media/variety_shows/Hinatazaka46 Takkyuu No. 1 Senshuken/Season 2023/230427 - #02.mkv
After scanning into tdarr the following changes happens
$ stat "/pool/media/variety_shows/Hinatazaka46 Takkyuu No. 1 Senshuken/Season 2023/230427 - #02.mkv"
File: /pool/media/variety_shows/Hinatazaka46 Takkyuu No. 1 Senshuken/Season 2023/230427 - #02.mkv
Size: 410478681 Blocks: 801728 IO Block: 4096 regular file
Device: 35h/53d Inode: 4890559679385254827 Links: 1
Access: (0664/-rw-rw-r--) Uid: ( 1000/ coders) Gid: ( 1000/ coders)
Access: 2024-06-29 13:58:23.745889303 +0300
Modify: 2024-06-29 13:58:23.741889302 +0300
Change: 2024-06-29 13:58:23.741889302 +0300
Birth: -{
"streams": [
{
"index": 0,
"codec_name": "ass",
"codec_long_name": "ASS (Advanced SSA) subtitle",
"codec_type": "subtitle",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
},
"tags": {
"language": "eng",
"title": "English",
"BPS": "134",
"DURATION": "00:23:38.700000000",
"NUMBER_OF_FRAMES": "495",
"NUMBER_OF_BYTES": "23812",
"_STATISTICS_WRITING_APP": "mkvpropedit v85.0 ('Shame For You') 64-bit",
"_STATISTICS_WRITING_DATE_UTC": "2024-06-29 10:58:23",
"_STATISTICS_TAGS": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
}
},
{
"index": 1,
"codec_name": "h264",
"codec_long_name": "H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10",
"profile": "High",
"codec_type": "video",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"width": 1280,
"height": 720,
"coded_width": 1280,
"coded_height": 720,
"closed_captions": 0,
"has_b_frames": 2,
"sample_aspect_ratio": "1:1",
"display_aspect_ratio": "16:9",
"pix_fmt": "yuv420p",
"level": 32,
"color_range": "tv",
"color_space": "bt709",
"color_transfer": "bt709",
"color_primaries": "bt709",
"chroma_location": "left",
"field_order": "progressive",
"refs": 1,
"is_avc": "true",
"nal_length_size": "4",
"r_frame_rate": "30000/1001",
"avg_frame_rate": "30000/1001",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"bits_per_raw_sample": "8",
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
},
"tags": {
"language": "eng",
"title": "English",
"BPS": "2173738",
"DURATION": "00:23:44.923366666",
"NUMBER_OF_FRAMES": "42705",
"NUMBER_OF_BYTES": "387176274",
"_STATISTICS_WRITING_APP": "mkvpropedit v85.0 ('Shame For You') 64-bit",
"_STATISTICS_WRITING_DATE_UTC": "2024-06-29 10:58:23",
"_STATISTICS_TAGS": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
}
},
{
"index": 2,
"codec_name": "aac",
"codec_long_name": "AAC (Advanced Audio Coding)",
"profile": "LC",
"codec_type": "audio",
"codec_tag_string": "[0][0][0][0]",
"codec_tag": "0x0000",
"sample_fmt": "fltp",
"sample_rate": "48000",
"channels": 2,
"channel_layout": "stereo",
"bits_per_sample": 0,
"r_frame_rate": "0/0",
"avg_frame_rate": "0/0",
"time_base": "1/1000",
"start_pts": 0,
"start_time": "0.000000",
"disposition": {
"default": 1,
"dub": 0,
"original": 0,
"comment": 0,
"lyrics": 0,
"karaoke": 0,
"forced": 0,
"hearing_impaired": 0,
"visual_impaired": 0,
"clean_effects": 0,
"attached_pic": 0,
"timed_thumbnails": 0
},
"tags": {
"language": "jpn",
"title": "Japanese",
"BPS": "128000",
"DURATION": "00:23:44.981333333",
"NUMBER_OF_FRAMES": "66796",
"NUMBER_OF_BYTES": "22799702",
"_STATISTICS_WRITING_APP": "mkvpropedit v85.0 ('Shame For You') 64-bit",
"_STATISTICS_WRITING_DATE_UTC": "2024-06-29 10:58:23",
"_STATISTICS_TAGS": "BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES"
}
}
],
"format": {
"filename": "/pool/media/variety_shows/Hinatazaka46 Takkyuu No. 1 Senshuken/Season 2023/230427 - #02.mkv",
"nb_streams": 3,
"nb_programs": 0,
"format_name": "matroska,webm",
"format_long_name": "Matroska / WebM",
"start_time": "0.000000",
"duration": "1424.981000",
"size": "410478681",
"bit_rate": "2304472",
"probe_score": 100,
"tags": {
"encoder": "libebml v1.4.2 + libmatroska v1.6.3",
"creation_time": "2023-04-28T11:09:38.000000Z"
}
}
}This behavior is causing really unwanted side effects like, inotifiy triggering, snapraid reporting many changed files due to mtime change etc.
I noticed this problem when i had massive library restructure, which triggered rescan in tdarr causing it change mdate on pretty much all mkv files that aren't already have stats generated for them.
Hi the server does not run mkvpropedit on the file when it scans it. That's only run by the worker using a plugin stack when Run mkvpropedit on files before running plugins is enabled on the Options tab or when you are using Run MKVPropEdit in a flow. mkvpropedit creates a new/modified file, that's how the application works and is expected behaviour, reverting the time may break some other peoples' processes.
If you are not looking to have this run then can disable Run mkvpropedit on files before running plugins on the Options tab or use a flow and have a flow plugin restore the times/dates or whatever else you're after, it's very customisable. Can also use Run CLI to run mkvpropedit with whatever custom arguments you're after:
