final error: ValueError: 'pv' is not in list
dunkgray opened this issue · comments
I'm connecting to this db;
db_hostname: agdcstaging-db.nci.org.au
db_port: 6432
db_database: ard_interop
datacube-alchemist run-many -E ard_interop --limit 1 fc_config.yaml [product: ls5_ard]
(py)[vdi-n4:aws_fc_testing] datacube-alchemist run-many -E ard_interop --limit 1 fc_config.yaml
2019-08-14 11:48.16 started dask dask_client=<Client: scheduler='tcp://127.0.0.1:42524' processes=8 cores=8>
2019-08-14 11:48.16 processing task stream
2019-08-14 11:48.19 data loaded task=AlchemistTask(dataset=Dataset <id=279f62b2-7d0a-4fe8-80ba-b0e235dcc0b3 type=ls5_ard location=file:///g/data1b/if87/ARD_interoperability/ga-packaged_collection/1986-08-17/LT50950741986229ASA00/ARD-METADATA.yaml>, settings=AlchemistSettings(specification=Specification(product='ls5_ard', measurements=['nbart_green', 'nbart_red', 'nbart_nir', 'nbart_swir_1', 'nbart_swir_2'], transform='fc.virtualproduct.FractionalCover', measurement_renames={'nbart_green': 'green', 'nbart_red': 'red', 'nbart_nir': 'nir', 'nbart_swir_1': 'swir1', 'nbart_swir_2': 'swir2'}, transform_args={'regression_coefficients': {'blue': [0.00041, 0.9747], 'green': [0.00289, 0.99779], 'red': [0.00274, 1.00446], 'nir': [4e-05, 0.98906], 'swir1': [0.00256, 0.99467], 'swir2': [-0.00327, 1.02551]}}), output=OutputSettings(location='/g/data/u46/users/dsg547/data/c3-testing/', dtype=dtype('uint8'), nodata=255, preview_image=['pv', 'npv', 'bs'], metadata={'product_family': 'fractional_cover', 'producer': 'ga.gov.au', 'dataset_version': '2.0.0'}, properties={'dea:dataset_maturity': 'interim'}), processing=ProcessingSettings(dask_chunks={'x': 1000, 'y': 1000}, dask_client={})))
distributed.worker - WARNING - Compute Failed
Function: lump_proc
args: ((AlchemistTask(dataset=Dataset <id=279f62b2-7d0a-4fe8-80ba-b0e235dcc0b3 type=ls5_ard location=file:///g/data1b/if87/ARD_interoperability/ga-packaged_collection/1986-08-17/LT50950741986229ASA00/ARD-METADATA.yaml>, settings=AlchemistSettings(specification=Specification(product='ls5_ard', measurements=['nbart_green', 'nbart_red', 'nbart_nir', 'nbart_swir_1', 'nbart_swir_2'], transform='fc.virtualproduct.FractionalCover', measurement_renames={'nbart_green': 'green', 'nbart_red': 'red', 'nbart_nir': 'nir', 'nbart_swir_1': 'swir1', 'nbart_swir_2': 'swir2'}, transform_args={'regression_coefficients': {'blue': [0.00041, 0.9747], 'green': [0.00289, 0.99779], 'red': [0.00274, 1.00446], 'nir': [4e-05, 0.98906], 'swir1': [0.00256, 0.99467], 'swir2': [-0.00327, 1.02551]}}), output=OutputSettings(location='/g/data/u46/users/dsg547/data/c3-testing/', dtype=dtype('uint8'), nodata=255, preview_image=['pv', 'npv', 'bs'], metadata={'product_family': 'fractional_cover', 'producer': 'ga.gov.au', 'dataset_
kwargs: {}
Exception: ValueError("'pv' is not in list",)
Traceback (most recent call last):
File "/g/data/u46/users/dsg547/py/current/bin/datacube-alchemist", line 11, in <module>
load_entry_point('datacube-alchemist', 'console_scripts', 'datacube-alchemist')()
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/cli.py", line 38, in run_many
execute_with_dask(client, tasks)
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/worker.py", line 102, in execute_with_dask
for result in completed:
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 80, in dask_compute_stream
yield from yy.result()
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/distributed/client.py", line 195, in result
six.reraise(*result)
File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/six.py", line 692, in reraise
raise value.with_traceback(tb)
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 49, in lump_proc
return [func(d) for d in dd]
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 49, in <listcomp>
return [func(d) for d in dd]
File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/worker.py", line 124, in execute_task
output_data = transform.compute(data)
File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/virtualproduct.py", line 61, in compute
fc.append(fractional_cover(data.sel(**s), measurements, self.regression_coefficients))
File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/fractional_cover.py", line 104, in fractional_cover
dataset = Datacube.create_storage({}, nbar_tile.geobox, measurements, data_func)
File "/g/data1a/u46/users/dsg547/sandpit/datacube-core/datacube/api/core.py", line 453, in create_storage
data = data_func(measurement)
File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/fractional_cover.py", line 96, in data_func
i = band_names.index(src_var)
ValueError: 'pv' is not in list
I think this has been fixed.