Globbing doesn't make sense

[Abdullah Atta]

Currently the API allows a .glob(...patterns) method in the Builder alongside the crawl one. All the globbing libraries (fast-glob & glob) that I have used do not allow the user to specify the source directory.

Maybe glob and crawl should be at an equal API level.

[Ian VanSchooten]

I see this is closed, but I'm trying to figure out how to use globbing myself. It seems that adding a pattern to .glob() isn't enough? I get a typescript error about Property 'sync' does not exist on type 'Builder'. Trying to find the right way to do this...

[Ian VanSchooten]

I guess I need to split my glob pattern into multiple pieces? One piece with the path, and the other with the globby parts of the glob pattern?

[Ian VanSchooten]

@thecodrr could you explain how glob and crawl interact? I'm having a spot of trouble for example I have a file like this:

/Users/ianvs/code/storybook/storybook/sandbox/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.ts

And this command is not finding it:

const files = (await new FDir()
          .withFullPaths()
          .withSymlinks()
          .glob('/Users/ianvs/code/storybook/storybook/sandbox/react-vite-default-js/template-stories/**/*.stories.@(js|jsx|ts|tsx|mdx)')
          .crawl('/Users/ianvs/code/storybook/storybook/sandbox/react-vite-default-js/template-stories')
          .withPromise()) as string[];

And this doesn't work either:

const files = (await new FDir()
          .withFullPaths()
          .withSymlinks()
          .glob('**/*.stories.@(js|jsx|ts|tsx|mdx)')
          .crawl('/Users/ianvs/code/storybook/storybook/sandbox/react-vite-default-js/template-stories')
          .withPromise()) as string[];

In fact that second option finds all kinds of files outside of the directory specified by crawl.

I think I must be missing something...

[Abdullah Atta]

@IanVS

glob allows you to specify the pattern that'll be used for finding the files. crawl allows you to specify the root directory from where the crawling will start. The crawler pattern matches all the files it finds and returns only those that match the pattern.

I tested the code samples you gave and it is working without an issue. I created the same directory structure using:

mkdir -p react-vite-default-js/template-stories/addons/docs/docs2/
touch react-vite-default-js/template-stories/addons/docs/docs2/button.stories.{ts,tsx,js,jsx,mdx}
touch react-vite-default-js/button.stories.{ts,tsx,js,jsx,mdx}

The directory structure now looks like this:

react-vite-default-js/
├── button.stories.js
├── button.stories.jsx
├── button.stories.mdx
├── button.stories.ts
├── button.stories.tsx
└── template-stories
    └── addons
        └── docs
            └── docs2
                ├── button.stories.js
                ├── button.stories.jsx
                ├── button.stories.mdx
                ├── button.stories.ts
                └── button.stories.tsx

4 directories, 10 files

Running the first sample:

  const files = await new fdir()
    .withFullPaths()
    .withSymlinks()
    .glob(
      "/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/**/*.stories.@(js|jsx|ts|tsx|mdx)"
    )
    .crawl(
      "/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories"
    )
    .withPromise();

I get:

[
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.js',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.jsx',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.mdx',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.ts',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.tsx'
]

And the second sample:

  const files = await new fdir()
    .withFullPaths()
    .withSymlinks()
    .glob("**/*.stories.@(js|jsx|ts|tsx|mdx)")
    .crawl(
      "/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories"
    )
    .withPromise();

I get:

[
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.js',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.jsx',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.mdx',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.ts',
  '/home/thecodrr/Sources/Repos/fdir/react-vite-default-js/template-stories/addons/docs/docs2/button.stories.tsx'
]

[Ian VanSchooten]

Interesting. The only other wrinkle might be the handling of symlinks, then. Are globs resolved based on actual path, or the symlink path? In my case, react-vite-default-js/template-stories/addons/docs is a symlink to /Users/ianvs/code/storybook/storybook/code/addons/docs/template/stories

[Ian VanSchooten]

FWIW, I'm seeing this behavior in storybookjs/storybook#19297. It's easy to reproduce if you'd like to clone and checkout the branch, then try running yarn task --task build --template react-vite/default-js --start-from install (after you run it once -- it'll take a while -- if you want to run it again, you can use yarn task --task build --template react-vite/default-js --start-from build)

[Abdullah Atta]

The only other wrinkle might be the handling of symlinks

Indeed. Testing again with the following directory structure:

react-vite-default-js/
├── button.stories.js
├── button.stories.jsx
├── button.stories.mdx
├── button.stories.ts
├── button.stories.tsx
└── template-stories
    └── addons
        └── docs -> /home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2
            ├── button.stories.js
            ├── button.stories.jsx
            ├── button.stories.mdx
            ├── button.stories.ts
            └── button.stories.tsx

The first sample returns nothing which is actually correct since pattern is matched against the resolved file path, not the symlink path. Changing the glob pattern to /home/thecodrr/Sources/Repos/fdir/code/addons/docs/**/*.stories.@(js|jsx|ts|tsx|mdx) makes it work again.

The second sample works without an issue though since it matches against all files at any depth irrespective of their base path. The output looks like this:

[
  '/home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2/button.stories.js',
  '/home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2/button.stories.jsx',
  '/home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2/button.stories.mdx',
  '/home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2/button.stories.ts',
  '/home/thecodrr/Sources/Repos/fdir/code/addons/docs/template/stories/docs2/button.stories.tsx'
]

[Abdullah Atta]

Perhaps I can adding something like withRealPaths to use real paths for resolved symlinks but use symlink paths by default. Or we can think of another solution for this.