Issue while running repo_sync
finerkill opened this issue · comments
Hi, we've been using reposado for a while now, but when I went to sync today it threw out the following error...:
repo_sync run started
Downloading 1026658 bytes from http://swscan.apple.com/content/catalogs/index.sucatalog...
277 products found in http://swscan.apple.com/content/catalogs/index.sucatalog
Building index.sucatalog...
Downloading 579820 bytes from http://swscan.apple.com/content/catalogs/index-1.sucatalog...
185 products found in http://swscan.apple.com/content/catalogs/index-1.sucatalog
Building index-1.sucatalog...
Downloading 1026658 bytes from http://swscan.apple.com/content/catalogs/others/index-leopard.merged-1.sucatalog...
277 products found in http://swscan.apple.com/content/catalogs/others/index-leopard.merged-1.sucatalog
Building index-leopard.merged-1.sucatalog...
Downloading 1978538 bytes from http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog...
354 products found in http://swscan.apple.com/content/catalogs/others/index-leopard-snowleopard.merged-1.sucatalog
Building index-leopard-snowleopard.merged-1.sucatalog...
Downloading 2628773 bytes from http://swscan.apple.com/content/catalogs/others/index-lion-snowleopard-leopard.merged-1.sucatalog...
594 products found in http://swscan.apple.com/content/catalogs/others/index-lion-snowleopard-leopard.merged-1.sucatalog
Building index-lion-snowleopard-leopard.merged-1.sucatalog...
Downloading 2060902 bytes from http://swscan.apple.com/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
436 products found in http://swscan.apple.com/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Building index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
Downloading 1755656 bytes from https://swscan.apple.com/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
453 products found in https://swscan.apple.com/content/catalogs/others/index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Building index-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
Downloading 5658151 bytes from https://swscan.apple.com/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
692 products found in https://swscan.apple.com/content/catalogs/others/index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Building index-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
Downloading 5610105 bytes from https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
559 products found in https://swscan.apple.com/content/catalogs/others/index-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog
Traceback (most recent call last):
File "./repo_sync", line 845, in
main()
File "./repo_sync", line 841, in main
download_packages=download_packages)
File "./repo_sync", line 743, in sync
copy_only_if_missing=fast_scan)
File "./repo_sync", line 550, in replicateURLtoFilesystem
getURL(full_url, local_file_path)
File "./repo_sync", line 465, in getURL
onlyifnewer=True, etag=saved_etag)
File "./repo_sync", line 349, in curl
header['http_result_description'] = line.split(None, 2)[2]
IndexError: list index out of range
Am I missing something here or have I done something wrong?
Cheers.
I can't say that I've seen that before, but it looks like it could just be a transient error where curl is not returning the expected data.
If this continues, I'd add a line before line 349 in repo_sync:
print header['http_result_description']
As an attempt to collect more data for debugging.
some of the endpoint have switched to http/2 and the output doesn't include a description.
it chokes on trying to get that in there.
adding an option to curl to force http/1.1 resolves the issue for me - working on setting up a PR for this shortly...
#72 to resolve this.
Ah. That makes sense.
Thanks for the prompt response!
Confirmed that has fixed it