xdmod ticket #33183 The script /usr/lib64/xdmod/batch_export_manager.php doesn't run to completion

[Robert Baron]

Robert Bartlett Baron, reported about 2 months ago
The data export section of xdmod is not currently working as it requires the script /usr/lib64/xdmod/batch_export_manager.php to run. Currently it is giving the following errors:

sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php
2023-06-20 20:57:00 [notice] batch_export_manager start (process_start_time: 2023-06-20 20:57:00)
2023-06-20 20:57:01 [error] Unknown "batchExport" option 'true' (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Data/BatchDataset.php(94): DataWarehouse\Data\RawStatisticsConfiguration->getBatchExportFieldDefinitions('Cloud')
#1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(309): DataWarehouse\Data\BatchDataset->__construct(Object(DataWarehouse\Query\Cloud\JobDataset), Object(XDUser), Object(CCR\Logger))
#2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))
#3 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)
#4 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()
#5 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()
#6 {main})
2023-06-20 20:57:01 [error] Failed to export data: Failed to create batch export query (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))
#1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)
#2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()
#3 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()
#4 {main})
postdrop: warning: unable to look up public/pickup: No such file or directory
postdrop: warning: unable to look up public/pickup: No such file or directory
2023-06-20 20:57:01 [error] Unknown "batchExport" option 'true' (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Data/BatchDataset.php(94): DataWarehouse\Data\RawStatisticsConfiguration->getBatchExportFieldDefinitions('Cloud')
#1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(309): DataWarehouse\Data\BatchDataset->__construct(Object(DataWarehouse\Query\Cloud\JobDataset), Object(XDUser), Object(CCR\Logger))
#2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))
#3 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)
#4 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()
#5 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()
#6 {main})
2023-06-20 20:57:01 [error] Failed to export data: Failed to create batch export query (module: data-warehouse-export, stacktrace: #0 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(156): DataWarehouse\Export\BatchProcessor->getDataSet(Array, Object(XDUser))
#1 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(118): DataWarehouse\Export\BatchProcessor->processSubmittedRequest(Array)
#2 /usr/share/xdmod/classes/DataWarehouse/Export/BatchProcessor.php(103): DataWarehouse\Export\BatchProcessor->processSubmittedRequests()
#3 /usr/lib64/xdmod/batch_export_manager.php(89): DataWarehouse\Export\BatchProcessor->processRequests()
#4 {main})
postdrop: warning: unable to look up public/pickup: No such file or directory
postdrop: warning: unable to look up public/pickup: No such file or directory
postdrop: warning: unable to look up public/pickup: No such file or directory
postdrop: warning: unable to look up public/pickup: No such file or directory
2023-06-20 20:57:09 [notice] batch_export_manager end (process_end_time: 2023-06-20 20:57:09)

Any suggestions?
Robert Bartlett Baron , said about 2 months ago
I got the script to run to completion, it actually wants a "public/pickup" directory from the directory that is is running in. For example:

sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php
2023-06-27 21:24:11 [notice] batch_export_manager start (process_start_time: 2023-06-27 21:24:11)
postdrop: warning: unable to look up public/pickup: No such file or directory
2023-06-27 21:24:14 [notice] batch_export_manager end (process_end_time: 2023-06-27 21:24:14)
sh-4.2$ pwd
/
sh-4.2$ cd /root/xdmod_data
sh-4.2$ /usr/lib64/xdmod/batch_export_manager.php
2023-06-27 21:24:38 [notice] batch_export_manager start (process_start_time: 2023-06-27 21:24:38)
2023-06-27 21:24:38 [notice] batch_export_manager end (process_end_time: 2023-06-27 21:24:38)

The '/' directory does not have a "public/pickup" directory from it, where as /root/xdmod_data does.

Running this does produce the requested export files.

I think we can close this for now.
Ryan Rathsam , said about 2 months ago
Ticket: https://help.xdmod.org/support/tickets/33183

Hi Robert,

Sounds good!

Thanks!

Ryan