opendatacube / datacube-alchemist

Dataset to Dataset Transformations

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

final error: ValueError: 'pv' is not in list

dunkgray opened this issue · comments

I'm connecting to this db;
db_hostname: agdcstaging-db.nci.org.au
db_port: 6432
db_database: ard_interop

datacube-alchemist run-many -E ard_interop --limit 1 fc_config.yaml [product: ls5_ard]

(py)[vdi-n4:aws_fc_testing] datacube-alchemist run-many -E ard_interop --limit 1 fc_config.yaml
2019-08-14 11:48.16 started dask                   dask_client=<Client: scheduler='tcp://127.0.0.1:42524' processes=8 cores=8>
2019-08-14 11:48.16 processing task stream
2019-08-14 11:48.19 data loaded                    task=AlchemistTask(dataset=Dataset <id=279f62b2-7d0a-4fe8-80ba-b0e235dcc0b3 type=ls5_ard location=file:///g/data1b/if87/ARD_interoperability/ga-packaged_collection/1986-08-17/LT50950741986229ASA00/ARD-METADATA.yaml>, settings=AlchemistSettings(specification=Specification(product='ls5_ard', measurements=['nbart_green', 'nbart_red', 'nbart_nir', 'nbart_swir_1', 'nbart_swir_2'], transform='fc.virtualproduct.FractionalCover', measurement_renames={'nbart_green': 'green', 'nbart_red': 'red', 'nbart_nir': 'nir', 'nbart_swir_1': 'swir1', 'nbart_swir_2': 'swir2'}, transform_args={'regression_coefficients': {'blue': [0.00041, 0.9747], 'green': [0.00289, 0.99779], 'red': [0.00274, 1.00446], 'nir': [4e-05, 0.98906], 'swir1': [0.00256, 0.99467], 'swir2': [-0.00327, 1.02551]}}), output=OutputSettings(location='/g/data/u46/users/dsg547/data/c3-testing/', dtype=dtype('uint8'), nodata=255, preview_image=['pv', 'npv', 'bs'], metadata={'product_family': 'fractional_cover', 'producer': 'ga.gov.au', 'dataset_version': '2.0.0'}, properties={'dea:dataset_maturity': 'interim'}), processing=ProcessingSettings(dask_chunks={'x': 1000, 'y': 1000}, dask_client={})))

distributed.worker - WARNING -  Compute Failed
Function:  lump_proc
args:      ((AlchemistTask(dataset=Dataset <id=279f62b2-7d0a-4fe8-80ba-b0e235dcc0b3 type=ls5_ard location=file:///g/data1b/if87/ARD_interoperability/ga-packaged_collection/1986-08-17/LT50950741986229ASA00/ARD-METADATA.yaml>, settings=AlchemistSettings(specification=Specification(product='ls5_ard', measurements=['nbart_green', 'nbart_red', 'nbart_nir', 'nbart_swir_1', 'nbart_swir_2'], transform='fc.virtualproduct.FractionalCover', measurement_renames={'nbart_green': 'green', 'nbart_red': 'red', 'nbart_nir': 'nir', 'nbart_swir_1': 'swir1', 'nbart_swir_2': 'swir2'}, transform_args={'regression_coefficients': {'blue': [0.00041, 0.9747], 'green': [0.00289, 0.99779], 'red': [0.00274, 1.00446], 'nir': [4e-05, 0.98906], 'swir1': [0.00256, 0.99467], 'swir2': [-0.00327, 1.02551]}}), output=OutputSettings(location='/g/data/u46/users/dsg547/data/c3-testing/', dtype=dtype('uint8'), nodata=255, preview_image=['pv', 'npv', 'bs'], metadata={'product_family': 'fractional_cover', 'producer': 'ga.gov.au', 'dataset_
kwargs:    {}
Exception: ValueError("'pv' is not in list",)

Traceback (most recent call last):
  File "/g/data/u46/users/dsg547/py/current/bin/datacube-alchemist", line 11, in <module>
    load_entry_point('datacube-alchemist', 'console_scripts', 'datacube-alchemist')()
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 764, in __call__
    return self.main(*args, **kwargs)
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 717, in main
    rv = self.invoke(ctx)
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 1137, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 956, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/click/core.py", line 555, in invoke
    return callback(*args, **kwargs)
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/cli.py", line 38, in run_many
    execute_with_dask(client, tasks)
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/worker.py", line 102, in execute_with_dask
    for result in completed:
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 80, in dask_compute_stream
    yield from yy.result()
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/distributed/client.py", line 195, in result
    six.reraise(*result)
  File "/g/data/v10/public/modules/dea-env/20181015/lib/python3.6/site-packages/six.py", line 692, in reraise
    raise value.with_traceback(tb)
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 49, in lump_proc
    return [func(d) for d in dd]
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/_dask.py", line 49, in <listcomp>
    return [func(d) for d in dd]
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-alchemist/datacube_alchemist/worker.py", line 124, in execute_task
    output_data = transform.compute(data)
  File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/virtualproduct.py", line 61, in compute
    fc.append(fractional_cover(data.sel(**s), measurements, self.regression_coefficients))
  File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/fractional_cover.py", line 104, in fractional_cover
    dataset = Datacube.create_storage({}, nbar_tile.geobox, measurements, data_func)
  File "/g/data1a/u46/users/dsg547/sandpit/datacube-core/datacube/api/core.py", line 453, in create_storage
    data = data_func(measurement)
  File "/g/data1a/u46/users/dsg547/sandpit/fc/fc/fractional_cover.py", line 96, in data_func
    i = band_names.index(src_var)
ValueError: 'pv' is not in list

I think this has been fixed.