ntfs_forensics tables ntfs_indx_data and ntfs_file_data not returning results
mike-myers-tob opened this issue · comments
observed behavior:
osquery> SELECT * FROM ntfs_part_data;
+--------------------+---------+------------------------------+
| device | address | description |
+--------------------+---------+------------------------------+
| \\.\PhysicalDrive0 | 0 | Safety Table |
| \\.\PhysicalDrive0 | 1 | Unallocated |
| \\.\PhysicalDrive0 | 2 | GPT Header |
| \\.\PhysicalDrive0 | 3 | Partition Table |
| \\.\PhysicalDrive0 | 4 | Basic data partition |
| \\.\PhysicalDrive0 | 5 | EFI system partition |
| \\.\PhysicalDrive0 | 6 | Microsoft reserved partition |
| \\.\PhysicalDrive0 | 7 | Basic data partition |
| \\.\PhysicalDrive0 | 8 | Unallocated |
+--------------------+---------+------------------------------+
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=4;
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=4 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=5 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=7 AND parent_path="\";
unable to open file \
osquery> SELECT * FROM ntfs_indx_data WHERE device="\\.\PhysicalDrive0" AND partition=7 AND parent_path="/";
expected behavior: shown in example
osquery version: the release binary from this repo
Name and version of OS tested on: Windows 10
I have improved constraint handling for that table, and it is now capable of listing all disks and partitions in case there is no device or partition condition in the WHERE clause.
The table now only needs one mandatory parameter (either parent_inode or parent_path) rather than three and will print an error if it is missing (first query was being silently discarded). Sadly, I could only print it to the status log (LOG(WARNING) << ...), but in the future we could expand the custom error messages feature we implemented for writable extension tables to the ::generate() methods as well.
The other queries didn't work because it seems like specifying the root path as a constraint is not yet implemented.
Digging into the original issue, the cause appears to be GUID Partition Tables--Windows 10 Pro uses MBR, which Sleuth Kit handles just fine, but Windows 10 Enterprise uses GPT and that doesn't work as well. Fortunately it seems that SleuthKit can still open the file system just fine if we use the raw image and offset to the file system, rather than attempting to open the partition.
Debugging messages when attempting to open the partitions one at a time on the system:
osquery> select * from ntfs_file_data WHERE path="/ProgramData/osquery"
...> ;
W0803 15:53:50.953084 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.959982 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid sector size 0)))
W0803 15:53:50.965960 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid sector size 0)))
W0803 15:53:50.972040 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.977958 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:50.993963 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (invalid MFT entry size))
W0803 15:53:51.000965 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (magic))
W0803 15:53:51.006969 17900 ntfsfileinfotable.cpp:186] unable to open filesystem: Invalid magic value (Not a NTFS file system (volume size is 0))
Further info:
$> .\mmls.exe -B \\.\PhysicalDrive0
GUID Partition Table (EFI)
Offset Sector: 0
Units are in 512-byte sectors
Slot Start End Length Size Description
000: Meta 0000000000 0000000000 0000000001 0512B Safety Table
001: ------- 0000000000 0000002047 0000002048 1024K Unallocated
002: Meta 0000000001 0000000001 0000000001 0512B GPT Header
003: Meta 0000000002 0000000033 0000000032 0016K Partition Table
004: 000 0000002048 0000616447 0000614400 0300M Basic data partition
005: 001 0000616448 0001640447 0001024000 0500M EFI system partition
006: 002 0001640448 0001902591 0000262144 0128M Microsoft reserved partition
007: 003 0001902592 0500117503 0498214912 0237G Basic data partition
008: ------- 0500117504 0500118191 0000000688 0344K Unallocated
$> .\fsstat.exe -o 0001902592 \\.\PhysicalDrive0
Cannot determine file system type
Manual inspection of the GPT verified that the displayed values match the on-disk contents.
What is needed to close this issue? Is it a limitation of SleuthKit that we should document, and file an issue upstream with them?
@GarretReece Any update on this issue? I'm moving this back to the Up next for the time being