pull_ntfs_metadata_off_files running too long
jeffshumphreys opened this issue · comments
It has to run fsutil on every file listed in database. How about, only if scan_for_ntfs flag set? by any directory change (and then any child must be marked??) or any update or new files from scan_file_directories_for_files script?
Same would probably hold true for scanning for alternate data streams, though changes in ADS may not actually trigger file size and date changes, or even hash changes!!! Yikes!
Something to ponder. Probably why people ignore this stuff.
One possibility is to open up Tasks to run in streams under the file processing Run Set. Also, which Tasks across Run Sets can run in parallel? Will schedule maintenance\pull_scheduled_task_events interfere with user processing\load_user_spreadsheet_interface?
How to implement Run Set Streams.
Added "scan_for_ntfs_id" to files table.
There was a bug (!!!!!). So down to 40K files in 10 minutes, not bad.
Now let it sit. No hasty-pastie.
I could alter it to have only new files or updated files get checked, from setting a flag. But not really a problem?