Exclude binary files
fjrdomingues opened this issue · comments
Background story explaining the current situation:
I had a situation where autopilot was looking for .ts files and it found a file that was actually a video (a binary), causing an error.
Solution
Find a way to exclude binaries files in the functions that retrieve files in the repo to get the files content and to create summaries.
To exclude binary files in the functions that retrieve files in the repo, you can use the isbinaryfile
package to check if a file is binary or not. Here's how you can do it:
- Install the
isbinaryfile
package by running:
npm install isbinaryfile
- In the
modules/fsInput.js
file, update thegetFilePaths
function as follows:
const { isBinaryFileSync } = require("isbinaryfile");
function getFilePaths(dir) {
const files = fs.readdirSync(dir);
const projectFiles = [];
for (const file of files) {
const filePath = path.posix.join(dir, file);
const stats = fs.statSync(filePath);
if (stats.isDirectory() && !ignoreList.includes(file)) {
projectFiles.push(...getFilePaths(filePath));
} else if (fileExtensionsToProcess.includes(path.extname(filePath))) {
// Check if the file is not a binary file
if (!isBinaryFileSync(filePath)) {
projectFiles.push(filePath);
}
}
}
return projectFiles;
};
By using the isbinaryfile
package, the getFilePaths
function will now exclude binary files when retrieving files in the repo, thus preventing errors caused by processing binary files.