Download shell get more invalid urls
AmberCheng opened this issue · comments
Hi,
I am downloading the train datasets thses days. As for its a big data, I divided all urls into 34 parts. So every part may contains 20w images. Then I used your shell to download every part. But a strange thing happened, the number of invalid urls add the number of images is more than 20w. I checked it in one part, the invalid urls contain some image is downloaded successfully. I wonder have you met this situation?
@AmberCheng I guess, when the url is valid, it also saves an image showing "not available".
@wubaoyuan I have just check it. The image is actual a image, not "not avaliable",but it is a broken one. I wonder why you don't package them, for downloading them has much more trouble......
@AmberCheng Personally I would like to share the images. But, there is copyright risk for our company.
@AmberCheng Please follow the suggestion that, downloading all images of ImageNet, then using the list we provide to extract the images used in our ML-Images. The URLs from Open Images are valid.