xdmod ticket #32824 Where would we find the memory used on the HPC side
rob-baron opened this issue · comments
#32824 Where would we find the memory used on the HPC side
Robert Bartlett Baron, reported 3 months ago
We have modeled our OpenShift clusters to import data into xdmod similar to a slurm job. We have included memory data for these jobs.
Unfortunately, we cannot find a similar metric in the job section as is found in the cloud section labeled "average memory reserved weighted by wall hours"
Do we need to do anything to enable this in the job section?
Any advise?
PNG
Screenshot 2...
(40.1 KB)
PNG
Screenshot 2...
(27 KB)
Robert Bartlett Baron , said 3 months ago
We have determined that we need to use supremm to get this data.
So far I have set up mongodb, confirmed that I am able to add accounts to it.
mongosh mongo:27017 -u root -p
Enter password: ****
Warning: Could not access file: EACCES: permission denied, mkdir '/.mongodb'
Current Mongosh Log ID: 645d90182070f2e0d4281be6
Connecting to: mongodb://@mongo:27017/?directConnection=true&appName=mongosh+1.8.2
Using MongoDB: 6.0.5
Using Mongosh: 1.8.2
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
Error: Could not open history file.
REPL session history will not be persisted.
The server generated these startup warnings when booting
2023-05-08T13:00:11.182+00:00: Using the XFS filesystem is strongly recommended with the WiredTiger storage engine. See http://dochub.mongodb.org/core/prodnotes-filesystem
2023-05-08T13:00:12.270+00:00: /sys/kernel/mm/transparent_hugepage/enabled is 'always'. We suggest setting it to 'never'
2023-05-08T13:00:12.271+00:00: vm.max_map_count is too low
Enable MongoDB's free cloud-based monitoring service, which will then receive and display
metrics about your deployment (disk utilization, CPU, operation statistics, etc).
The monitoring data will be available on a MongoDB website with a unique URL accessible to you
and anyone you share the URL with. MongoDB may use this information to make product
improvements and to suggest MongoDB products and deployment options to you.
To enable free monitoring, run the following command: db.enableFreeMonitoring()
To permanently disable this reminder, run the following command: db.disableFreeMonitoring()
test> use admin
switched to db admin
admin> db.getUsers()
{
users: [
{
_id: 'admin.root',
userId: new UUID("c7457715-9916-4b42-95a8-f98aebe0cdfb"),
user: 'root',
db: 'admin',
roles: [ { role: 'root', db: 'admin' } ],
mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
},
{
_id: 'admin.xdmod',
userId: new UUID("56cbf8c1-e73f-4a47-a97a-e7959dfa791b"),
user: 'xdmod',
db: 'admin',
roles: [ { role: 'readWrite', db: 'supremm' } ],
mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
},
{
_id: 'admin.xdmod-ro',
userId: new UUID("5781bfc7-5d72-4397-bf64-3824673143e5"),
user: 'xdmod-ro',
db: 'admin',
roles: [ { role: 'read', db: 'supremm' } ],
mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
}
],
ok: 1
}
sh-4.2$ mongosh mongo:27017 -u xdmod -p
Enter password: ****
Warning: Could not access file: EACCES: permission denied, mkdir '/.mongodb'
Current Mongosh Log ID: 645d9062c880d17c66fcf0e6
Connecting to: mongodb://@mongo:27017/?directConnection=true&appName=mongosh+1.8.2
Using MongoDB: 6.0.5
Using Mongosh: 1.8.2
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
Error: Could not open history file.
REPL session history will not be persisted.
sh-4.2$ mongosh mongo:27017 -u xdmod-ro -p
Enter password: ****
Warning: Could not access file: EACCES: permission denied, mkdir '/.mongodb'
Current Mongosh Log ID: 645d9089378da7f40e19653c
Connecting to: mongodb://@mongo:27017/?directConnection=true&appName=mongosh+1.8.2
Using MongoDB: 6.0.5
Using Mongosh: 1.8.2
For mongosh info see: https://docs.mongodb.com/mongodb-shell/
To help improve our products, anonymous usage data is collected and sent to MongoDB periodically (https://www.mongodb.com/legal/privacy-policy).
You can opt-out by running the disableTelemetry() command.
Error: Could not open history file.
REPL session history will not be persisted.
I have added supremm using the rpm, and here is what it added:
sh-4.2$ find / | grep -i supremm
/tmp/supremm
/usr/bin/aggregate_supremm.sh
/usr/lib64/xdmod/aggregate_supremm.php
/usr/lib64/xdmod/supremm_sharedjobs.php
/usr/lib64/xdmod/xdmod-supremm-admin
/usr/lib64/xdmod/xdmod-supremm-jobinfo
/usr/share/doc/xdmod-supremm-10.0.0
/usr/share/xdmod/html/about/supremm.html
/usr/share/xdmod/html/about/images/Supremm_drop_off.png
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/checksum_data
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/checksum_type
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/command_line
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/from_repo
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/from_repo_revision
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/from_repo_timestamp
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/installed_by
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/origin_url
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/reason
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/releasever
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/ts_install_langs
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/tsflag_nodocs
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/var_contentdir
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/var_infra
/var/lib/yum/yumdb/x/bd1782d4f03ec96385f5d5424a13880ace0f7576-xdmod-supremm-10.0.0-1.0.el7-noarch/var_uuid
I also added:
sh-4.2$ cat /etc/xdmod/portal_settings.d/supremm.ini
[supremm-general]
; The version number is updated during the upgrade process.
version = "10.0.0"
; The path of the ETL schema file relative to [share]/etl/js/config/supremm.
schema_file = "etl.schema.js"
[jobsummarydb]
db_engine = "MongoDB"
uri = "mongodb://xdmod:xsZ0LpZstneBpijLy7@mongodb:27017/supremm?authSource=admin"
db = "supremm"
sh-4.2$ vi /etc/xdmod/portal_settings.d/supremm.ini
sh-4.2$ cat /etc/xdmod/portal_settings.d/supremm.ini
[supremm-general]
; The version number is updated during the upgrade process.
version = "10.0.0"
; The path of the ETL schema file relative to [share]/etl/js/config/supremm.
schema_file = "etl.schema.js"
[jobsummarydb]
db_engine = "MongoDB"
uri = "mongodb://xdmod:pass@mongo:27017/supremm?authSource=admin"
db = "supremm"
However, when I run xdmod-setup, I don't see the supremm option:
========================================================================
Open XDMoD Setup
- General Settings
- Database Settings
- Organization
- Resources
- Create Admin User
- Hierarchy
- Data Warehouse Batch Export
- Automatically Check for Updates
q) Quit
Select an option (1, 2, 3, 4, 5, 6, 7, 8, q):
Any suggestions?
Rob Baron
Joseph White , said 3 months ago
Cc: msd@bu.edu, joachimw@bu.edu
Ticket: https://help.xdmod.org/support/tickets/32824
Hi Robert,
What is the path to the xdmod-setup command as reported by "which xdmod-setup"? If you are using the rPM supplied version it should be in /usr/bin/xdmod-setup and inside this file it should have a line:
require_once '/usr/share/xdmod/configuration/linker.php';
If the RPM install was successful then there should be several files installed that add the suprem setup menu item:
The command
"rpm -ql xdmod-supremm | grep Setup"
should produce the following:
/usr/share/xdmod/classes/OpenXdmod/Setup
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmDbSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmEditResourceSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmResourcesSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmSetup.php
Is this the same on your system?
Thanks,
Joe
Robert Bartlett Baron , said 3 months ago
I can only find one xdmod-setup, which is located at /usr/bin/xdmod-setup. It does contain the line "require_once ... ".
And yes, it is the same, as in:
sh-4.2$ rpm -ql xdmod-supremm | grep Setup
/usr/share/xdmod/classes/OpenXdmod/Setup
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmDbSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmEditResourceSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmResourcesSetup.php
/usr/share/xdmod/classes/OpenXdmod/Setup/SupremmSetup.php
Rob.
Joseph White , said 3 months ago
Cc: msd@bu.edu, joachimw@bu.edu
Ticket: https://help.xdmod.org/support/tickets/32824
Hi Robert,
very odd. Please can you check that the /etc/xdmod/setup.d/supremm.json file is readable by root and has the following contents:
ls -l /etc/xdmod/setup.d/supremm.json
-rw-r--r-- 1 root root 144 Mar 10 2022 /etc/xdmod/setup.d/supremm.json
[root@f95093566a43 SUPREMM]# cat /etc/xdmod/setup.d/supremm.json
{
"+menu": [
{
"position": 200,
"label": "SUPReMM",
"handler": "SupremmSetup"
}
]
}
Joe
Robert Bartlett Baron , said 3 months ago
Joe,
That file doesn't exist. When I added it, I got the following error message:
sh-4.2$ xdmod-setup
You are currently using Open XDMoD 10.0.0, but a newer version
(10.0.2) is available.
Do you want to continue (yes, no)? [no] yes
PHP Fatal error: Class 'OpenXdmod\Setup\SupremmSetup' not found in /usr/share/xdmod/classes/OpenXdmod/Setup/MainMenu.php on line 47
Rob.
Joseph White , said 3 months ago
Cc: msd@bu.edu, joachimw@bu.edu
Ticket: https://help.xdmod.org/support/tickets/32824
Hi Robert,
That file is delivered by the xdmod-supremm RPM. If that file does not exist that indicates that there is a problem with your installation - what is the output of the
rpm -q --verify xdmod-supremm
command?
Do you know what could have happened to the filesystem to cause an RPM delivered file to be removed?
Joe
Robert Bartlett Baron , said 3 months ago
Joe,
That actually makes sense. Because of the relatively simple modifications that we have made to sources in previous tickets, I have placed the source on a persistent volume that gets overlayed on to the file system over the directory the files were installed in.
here is the output of the verification:
rpm -q --verify xdmod-supremm
missing /etc/xdmod/assets.d/supremm.json
missing c /etc/xdmod/datawarehouse.d/jobefficiency.json
.M...UG.. /etc/xdmod/datawarehouse.d/ref
missing /etc/xdmod/datawarehouse.d/ref/job-efficiency-group-bys.json
missing /etc/xdmod/datawarehouse.d/ref/job-efficiency-statistics.json
missing /etc/xdmod/datawarehouse.d/ref/supremm-group-bys.json
missing /etc/xdmod/datawarehouse.d/ref/supremm-statistics.json
missing c /etc/xdmod/datawarehouse.d/supremm.json
SM5..UGT. c /etc/xdmod/efficiency_analytics.json
.M...UG.. /etc/xdmod/etl
.M...UG.. /etc/xdmod/etl/etl.d
missing /etc/xdmod/etl/etl.d/jobefficiency_realm.json
missing /etc/xdmod/etl/etl.d/supremm-migration-9_5_0-10_0_0.json
missing /etc/xdmod/etl/etl.d/supremm_realm.json
.M...UG.. /etc/xdmod/etl/etl_action_defs.d
missing /etc/xdmod/etl/etl_action_defs.d/jobefficiency
missing /etc/xdmod/etl/etl_action_defs.d/jobefficiency/job_category.json
missing /etc/xdmod/etl/etl_action_defs.d/jobefficiency/jobefficiency_by.json
missing /etc/xdmod/etl/etl_action_defs.d/jobefficiency/jobefficiency_by_day.json
missing /etc/xdmod/etl/etl_action_defs.d/jobefficiency/jobefficiency_by_day_joblist.json
missing /etc/xdmod/etl/etl_action_defs.d/supremm
missing /etc/xdmod/etl/etl_action_defs.d/supremm/job_scripts.json
missing /etc/xdmod/etl/etl_action_defs.d/supremm/supremmfact_by.json
missing /etc/xdmod/etl/etl_action_defs.d/supremm/supremmfact_by_day.json
missing /etc/xdmod/etl/etl_action_defs.d/supremm/supremmfact_by_day_joblist.json
.M...UG.. /etc/xdmod/etl/etl_data.d
missing /etc/xdmod/etl/etl_data.d/jobefficiency
missing /etc/xdmod/etl/etl_data.d/jobefficiency/job_category.json
.M...UG.. /etc/xdmod/etl/etl_macros.d
missing /etc/xdmod/etl/etl_macros.d/jobefficiency
missing /etc/xdmod/etl/etl_macros.d/jobefficiency/job_categorization.sql
.M...UG.. /etc/xdmod/etl/etl_sql.d
missing /etc/xdmod/etl/etl_sql.d/supremm
missing /etc/xdmod/etl/etl_sql.d/supremm/migration-9_5_0-10_0_0.sql
missing /etc/xdmod/etl/etl_sql.d/supremm/postprocess.sql
.M...UG.. /etc/xdmod/etl/etl_tables.d
missing /etc/xdmod/etl/etl_tables.d/jobefficiency
missing /etc/xdmod/etl/etl_tables.d/jobefficiency/job_category.json
missing /etc/xdmod/etl/etl_tables.d/jobefficiency/jobefficiency_by.json
missing /etc/xdmod/etl/etl_tables.d/jobefficiency/jobefficiency_by_day.json
missing /etc/xdmod/etl/etl_tables.d/jobefficiency/jobefficiency_by_day_joblist.json
missing /etc/xdmod/etl/etl_tables.d/supremm
missing /etc/xdmod/etl/etl_tables.d/supremm/job.json
missing /etc/xdmod/etl/etl_tables.d/supremm/job_errors.json
missing /etc/xdmod/etl/etl_tables.d/supremm/job_scripts.json
missing /etc/xdmod/etl/etl_tables.d/supremm/jobhost.json
missing /etc/xdmod/etl/etl_tables.d/supremm/supremmfact_by.json
missing /etc/xdmod/etl/etl_tables.d/supremm/supremmfact_by_day.json
missing /etc/xdmod/etl/etl_tables.d/supremm/supremmfact_by_day_joblist.json
missing c /etc/xdmod/internal_dashboard.d/supremm.json
missing /etc/xdmod/modules.d/supremm.json
SM5..UGT. c /etc/xdmod/portal_settings.d/supremm.ini
missing c /etc/xdmod/rawstatistics.d/10_supremm.json
missing c /etc/xdmod/resource_types.d/supremm.json
missing c /etc/xdmod/rest.d/efficiency.json
missing c /etc/xdmod/rest.d/supremm_dataflow.json
missing c /etc/xdmod/roles.d/jobefficiency.json
missing c /etc/xdmod/roles.d/supremm.json
missing c /etc/xdmod/roles.d/supremm:dashboard.json
SM5..UGT. c /etc/xdmod/setup.d/supremm.json
missing c /etc/xdmod/supremmconfig.json
.M....... /usr/bin/aggregate_supremm.sh
.M....G.. /usr/share/xdmod
.M...UG.. /usr/share/xdmod/classes
.M...UG.. /usr/share/xdmod/classes/DB
missing /usr/share/xdmod/classes/DB/EtlJournalHelper.php
.M...UG.. /usr/share/xdmod/classes/DataWarehouse
.M...UG.. /usr/share/xdmod/classes/DataWarehouse/Query
missing /usr/share/xdmod/classes/DataWarehouse/Query/JobEfficiency
missing /usr/share/xdmod/classes/DataWarehouse/Query/JobEfficiency/RawData.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/Aggregate.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/GroupBy.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/GroupBys
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/JobDataset.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/JobMetadata.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/JobTimeseries.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/RawData.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/Statistic.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/Statistics
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/SupremmDbInterface.php
missing /usr/share/xdmod/classes/DataWarehouse/Query/SUPREMM/Timeseries.php
.M...UG.. /usr/share/xdmod/classes/OpenXdmod
.M...UG.. /usr/share/xdmod/classes/OpenXdmod/Setup
missing /usr/share/xdmod/classes/OpenXdmod/Setup/SupremmDbSetup.php
missing /usr/share/xdmod/classes/OpenXdmod/Setup/SupremmEditResourceSetup.php
missing /usr/share/xdmod/classes/OpenXdmod/Setup/SupremmResourcesSetup.php
missing /usr/share/xdmod/classes/OpenXdmod/Setup/SupremmSetup.php
.M...UG.. /usr/share/xdmod/classes/Rest
.M...UG.. /usr/share/xdmod/classes/Rest/Controllers
missing /usr/share/xdmod/classes/Rest/Controllers/EfficiencyControllerProvider.php
missing /usr/share/xdmod/classes/Rest/Controllers/SupremmDataflowControllerProvider.php
.M...UG.. /usr/share/xdmod/configuration
.M...UG.. /usr/share/xdmod/db
missing /usr/share/xdmod/db/migrations
missing /usr/share/xdmod/db/migrations/8.0.0-8.1.0
missing /usr/share/xdmod/db/migrations/8.0.0-8.1.0/modw_supremm.sql
.M...UG.. /usr/share/xdmod/db/schema
missing /usr/share/xdmod/db/schema/modw_etl.sql
missing /usr/share/xdmod/db/schema/modw_supremm.sql
.M...UG.. /usr/share/xdmod/etl
.M...UG.. /usr/share/xdmod/etl/js
missing /usr/share/xdmod/etl/js/config
missing /usr/share/xdmod/etl/js/config/supremm
missing /usr/share/xdmod/etl/js/config/supremm/README.txt
missing /usr/share/xdmod/etl/js/config/supremm/app_ident.js
missing /usr/share/xdmod/etl/js/config/supremm/application.json
missing /usr/share/xdmod/etl/js/config/supremm/applicationtables.js
missing /usr/share/xdmod/etl/js/config/supremm/archivedb.sh
missing /usr/share/xdmod/etl/js/config/supremm/dataset_maps
missing /usr/share/xdmod/etl/js/config/supremm/dataset_maps/pcp.js
missing c /usr/share/xdmod/etl/js/config/supremm/etl.profile.js
missing /usr/share/xdmod/etl/js/config/supremm/etl.schema.js
missing /usr/share/xdmod/etl/js/config/supremm/map_helpers.js
missing /usr/share/xdmod/etl/js/config/supremm/output_db
missing /usr/share/xdmod/etl/js/config/supremm/output_db/groupbys.json
missing /usr/share/xdmod/etl/js/config/supremm/output_db/modw_supremm.sql
missing /usr/share/xdmod/etl/js/config/supremm/output_db/supremm_reset.sql
missing /usr/share/xdmod/etl/js/config/supremm/output_db/update_appid.sql
missing /usr/share/xdmod/etl/js/config/supremm/tests
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected/4655275-1445928790.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected/4843309-1546457856.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected/5659085-1469572318.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected/719452-1470861674.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/expected/8291026-1518721536.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input/4655275-1445928790.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input/4843309-1546457856.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input/5659085-1469572318.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input/719452-1470861674.json
missing /usr/share/xdmod/etl/js/config/supremm/tests/pcp/input/8291026-1518721536.json
missing /usr/share/xdmod/etl/js/config/supremm/unit_test
missing /usr/share/xdmod/etl/js/config/supremm/unit_test/app_ident_test.js
missing /usr/share/xdmod/etl/js/config/supremm/unit_test/map_helpers_test.js
missing /usr/share/xdmod/etl/js/config/supremm/validators
missing /usr/share/xdmod/etl/js/config/supremm/validators/dataset.schema.json
missing /usr/share/xdmod/etl/js/config/supremm/validators/etl.schema.schema.json
.M...UG.. /usr/share/xdmod/html
.M...UG.. /usr/share/xdmod/html/gui
.M...UG.. /usr/share/xdmod/html/gui/css
missing /usr/share/xdmod/html/gui/css/Efficiency.css
missing /usr/share/xdmod/html/gui/css/UserEfficiencyComponent.css
.M...UG.. /usr/share/xdmod/html/gui/images
missing /usr/share/xdmod/html/gui/images/double_headed_arrow.png
.M...UG.. /usr/share/xdmod/html/gui/images/help
missing /usr/share/xdmod/html/gui/images/help/jobefficiency-multi-component.svg
missing /usr/share/xdmod/html/gui/images/help/jobefficiency-single-component.svg
missing /usr/share/xdmod/html/gui/images/help/userjobefficiency-component.svg
missing /usr/share/xdmod/html/gui/images/right_arrow.png
.M...UG.. /usr/share/xdmod/html/gui/js
.M...UG.. /usr/share/xdmod/html/gui/js/modules
.M...UG.. /usr/share/xdmod/html/gui/js/modules/dashboard
missing /usr/share/xdmod/html/gui/js/modules/dashboard/JobEfficiencyComponent.js
missing /usr/share/xdmod/html/gui/js/modules/dashboard/UserJobEfficiencyComponent.js
missing /usr/share/xdmod/html/gui/js/modules/efficiency
missing /usr/share/xdmod/html/gui/js/modules/efficiency/AnalyticPanel.js
missing /usr/share/xdmod/html/gui/js/modules/efficiency/Efficiency.js
missing /usr/share/xdmod/html/gui/js/modules/efficiency/FilterPanel.js
missing /usr/share/xdmod/html/gui/js/modules/efficiency/ScatterPlotPanel.js
.M...UG.. /usr/share/xdmod/html/gui/lib
missing /usr/share/xdmod/html/gui/lib/jsPlumb
missing /usr/share/xdmod/html/gui/lib/jsPlumb/jquery.jsPlumb-1.6.3-min.js
.M...UG.. /usr/share/xdmod/html/internal_dashboard
missing /usr/share/xdmod/html/internal_dashboard/supremm
missing /usr/share/xdmod/html/internal_dashboard/supremm/css
missing /usr/share/xdmod/html/internal_dashboard/supremm/css/style.css
missing /usr/share/xdmod/html/internal_dashboard/supremm/js
missing /usr/share/xdmod/html/internal_dashboard/supremm/js/stats.js
missing /usr/share/xdmod/html/internal_dashboard/supremm/stats.php
missing /usr/share/xdmod/html/internal_dashboard/supremm/user_check.php
.M...UG.. /usr/share/xdmod/templates
missing /usr/share/xdmod/templates/supremm
missing /usr/share/xdmod/templates/supremm/portal_settings.template
Rob.
Robert Bartlett Baron , said 2 months ago
Just to let you know, I have gotten all of the files installed. Not sure if I have everyone configured correctly.
So far, it seems to take quite a bit more disk space - are there any settings to use to minimize the database size?
Robert Bartlett Baron , said 2 months ago
The supremm section show up in the interface, but the data doesn't appear yet.
I have rerun the ingestor:
$ xdmod-ingestor xdmod-ingestor --last-modified-start-date "2018-01-01 12:30:00"
And then run:
$ aggregate_supremm.sh
aggregate_supremm does give the following error:
2023-06-14T18:36:09.374Z - error: nerc_openshift_prod: Create index undefined returned MongoError: command createIndexes requires authentication
Any suggestions?
Joseph White , said 2 months ago
Cc: msd@bu.edu, joachimw@bu.edu
Ticket: https://help.xdmod.org/support/tickets/32824
Hi Robert,
The error message
"Create index undefined returned MongoError: command createIndexes requires authentication"
is caused because your mongo database permission is not allowing the createIndex command. Is authentication switched on in your Mongo Database and what permissions have you granted to the user account that is configured in XDMoD to connect to mongo?
Joe
Robert Bartlett Baron , said 2 months ago
Joe,
I am not that familiar with mongo. Here is what db.getUsers returns:
supremm> db.getUsers()
{
users: [
{
_id: 'supremm.xdmod',
userId: new UUID("9ae8c753-3d7c-4216-8b24-656baa877152"),
user: 'xdmod',
db: 'supremm',
roles: [ { role: 'readWrite', db: 'supremm' } ],
mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
},
{
_id: 'supremm.xdmod-ro',
userId: new UUID("aadff34d-542e-4cfe-acee-7806617a101e"),
user: 'xdmod-ro',
db: 'supremm',
roles: [ { role: 'read', db: 'supremm' } ],
mechanisms: [ 'SCRAM-SHA-1', 'SCRAM-SHA-256' ]
}
],
ok: 1
}
Is this what you were asking?
Rob.
Robert Bartlett Baron , said about 2 months ago
Joe,
Are these permissions correct for the accounts???
Rob.
Robert Bartlett Baron , said about 2 months ago
Joe,
By setting the uri from "mongodb://mongo:27017/supremm" to what is in the following file:
sh-4.2$ cat /etc/xdmod/portal_settings.d/supremm.ini
[supremm-general]
; The version number is updated during the upgrade process.
version = "10.0.0"
; The path of the ETL schema file relative to [share]/etl/js/config/supremm.
schema_file = "etl.schema.js"
[jobsummarydb]
db_engine = "MongoDB"
uri = "mongodb://xdmod:[password]@mongo:27017/supremm?authSource=admin"
db = "supremm"
The script "aggregate_supremm.sh" ran to completion.
However, I still do not see any data appearing in the supremm section within the xdmod GUI.
Any suggestions as to what to check next?
Robert Bartlett Baron , said about 1 month ago
We can close this ticket as I created a new ticket as this ticket has gotten off topic a bit.