fs.Glob skips directories while using .zip file system
mtojek opened this issue · comments
While I was working on using the Package Registry's model in the Package Storage Infra, I found a bug in handling fs.Glob. GetDataStreamPaths checks if there are any data stream directories in the .zip archive, but it may happen that there are no intermediate directories and the .zip is 100% correct.
$ unzip -l jobs/indexing/testdata/1password-0.1.1.zip
Archive: jobs/indexing/testdata/1password-0.1.1.zip
Length Date Time Name
--------- ---------- ----- ----
351 03-04-2022 14:43 1password-0.1.1/changelog.yml
1344 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/agent/stream/httpjson.yml.hbs
3560 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/elasticsearch/ingest_pipeline/default.yml
592 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/fields/base-fields.yml
3605 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/fields/ecs.yml
1085 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/fields/fields.yml
1619 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/manifest.yml
2242 03-04-2022 14:43 1password-0.1.1/data_stream/item_usages/sample_event.json
1348 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/agent/stream/httpjson.yml.hbs
4081 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/elasticsearch/ingest_pipeline/default.yml
596 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/fields/base-fields.yml
4865 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/fields/ecs.yml
1282 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/fields/fields.yml
1628 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/manifest.yml
2291 03-04-2022 14:43 1password-0.1.1/data_stream/signin_attempts/sample_event.json
8657 03-04-2022 14:43 1password-0.1.1/docs/README.md
450008 03-04-2022 14:43 1password-0.1.1/img/1password-itemusages-screenshot.png
4441 03-04-2022 14:43 1password-0.1.1/img/1password-logo-light-bg.svg
427378 03-04-2022 14:43 1password-0.1.1/img/1password-signinattempts-screenshot.png
3814 03-04-2022 14:43 1password-0.1.1/kibana/dashboard/1password-item-usages-full-dashboard.json
3872 03-04-2022 14:43 1password-0.1.1/kibana/dashboard/1password-signin-attempts-full-dashboard.json
2214 03-04-2022 14:43 1password-0.1.1/kibana/map/1password-item-usages-source-IPs-map.json
2227 03-04-2022 14:43 1password-0.1.1/kibana/map/1password-signin-attempts-source-IPs-map.json
1065 03-04-2022 14:43 1password-0.1.1/kibana/search/1password-all-events.json
1083 03-04-2022 14:43 1password-0.1.1/kibana/search/1password-item-usages.json
1081 03-04-2022 14:43 1password-0.1.1/kibana/search/1password-signin-attempts.json
1412 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-item-usages-hot-items.json
1715 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-item-usages-hot-users.json
1523 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-item-usages-hot-vaults.json
2333 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-item-usages-over-time.json
2859 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-signin-attempts-categories-over-time.json
2352 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-signin-attempts-count-over-time.json
1795 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-signin-attempts-failed-gauge.json
1735 03-04-2022 14:43 1password-0.1.1/kibana/visualization/1password-signin-attempts-hot-users.json
2376 03-04-2022 14:43 1password-0.1.1/manifest.yml
--------- -------For the 1password-0.1.1.zip file, the logic assumes that there are no data streams, but you can unpack .zip file and all dirs will be put in right places.
@jsoriano I can try to adjust the logic, but would like to confirm if this is a bug.