Share ScanNet skip.lst

[Sasha Sax]

I believe that ScanNet has a few frames where the pose matrices are all -inf, and others where the depth is all missing. For both cases, the Ponder pretraining code ends up throwing an error, and I'm guessing that is why you added the skip.lst file for the scannet.py dataloader.

Would it be possible to share this skip list, since it is not included in the repo and I want to make sure that I am training on the same data that you did :)

Thank you!

[Sasha Sax]

I generated the skip list using this file -- hope it helps!

import argparse
import os

import numpy as np
from tqdm import tqdm


def crawl_and_check_pose_files(root_dir, output_file=None):
    scenes = [
        d for d in os.listdir(root_dir) if os.path.isdir(os.path.join(root_dir, d))
    ]
    with open(output_file, "w") if output_file else None as f:
        for scene in tqdm(scenes, desc="Processing scenes"):
            scene_dir = os.path.join(root_dir, scene, "pose")
            if os.path.exists(scene_dir):
                for file in os.listdir(scene_dir):
                    if file.endswith(".txt"):
                        file_path = os.path.join(scene_dir, file)
                        try:
                            matrix = np.loadtxt(file_path)
                            if np.any(np.isinf(matrix)):
                                frame_id = file.replace(".txt", "")
                                output = f"{scene} {frame_id}\n"
                                if f:
                                    f.write(output)
                                else:
                                    print(output)
                        except Exception as e:
                            print(f"Error reading {file_path}: {e}")


def main():
    parser = argparse.ArgumentParser(
        description="Crawl directories for pose files, identify files with all -inf values, and save the results."
    )
    parser.add_argument(
        "path", type=str, help="The root directory to start crawling from."
    )
    parser.add_argument(
        "--output",
        "-o",
        type=str,
        default=None,
        help="File to save the output. Prints to stdout if not specified.",
    )

    args = parser.parse_args()
    crawl_and_check_pose_files(args.path, args.output)


if __name__ == "__main__":
    main()

[Haoyi Zhu]

Hi, thanks for your interest and your awesome code! Sorry for the late reply, I was on vacation last month ：）Yeah we skip some broken data, and this is the skip.lst we use. I think your code can produce similar file with ours:)
skip.lst.txt

[Haoyi Zhu]

I have pushed our skip.lst file to the repo :)