Investigate potential cron job issues with Salesforce and Drupal
duffy-james opened this issue · comments
When adding Salesforce records to Metrolist and our Building Housing map, a cron job runs every five minutes that picks up new and updated records from Salesforce. In looking into the issue, @padora-cob suspected that there may be an issue with the cron job running in Drupal that is preventing the new and updated records from appearing.
There is an option to separate the salesforce cron from the drupal cron and have it run in its own process and with its own schedule. This would effectively sandbox it from other cron process which may over-run or have their own processing issues.
Moving to an independent cron process should be investigated.
This is related to #2434
This is a very common, very low risk change to make. We already have other cron processes configured in this way, for this reason.
The proposed change takes 2 minutes to make, and 1 minute to apply. It is provisioned and fully supported by the author of the Drupal contributed salesforce module.
There is no down time on the server and no possibility of affecting the website or any of its services.
It is also completely reversible, just as quickly.
CI environment:
- Added cron task to Acquia (
/usr/bin/wget -O - -q -t 1 https://d8-ci.boston.gov/salesforce_pull/endpoint/xxxxx &>> /var/log/sites/${AH_SITE_NAME}/logs/$(hostname -s)/salesforce.log) to run every 5 mins - Added cron task to Acquia (
/usr/bin/wget -O - -q -t 1 https://d8-ci.boston.gov/salesforce_push/endpoint/xxxxx &>> /var/log/sites/${AH_SITE_NAME}/logs/$(hostname -s)/salesforce.log) to run every 5 mins - Turned off cron sync feature in Drupal console:
Will monitor and compare this environment to prod next time there is a sync issue in prod.
Can check cron log at /var/log/sites/bostond8ci/logs/staging-15139/salesforce.log on acquia server
@ebshrier the ci environment should be updated now with a potential fix if you want to test that out. The cron job is now running from our Acquia server instead of through Drupal. If this addresses the issue, we can move it through to production.
We're still running updates every 5 minutes. We can technically speed that up (or down) in increments of minutes, but there is a concern about that creating a lot of requests to the website. If this continues to be a pain point, we can look into it again.
Thanks James I'm in there now so I'll let you know how it goes.
@duffy-james Yeah, still not seeing updates at all in test environment either. I think for the time being, it would be great to get updates as close to real time as possible to at least take that variable out of the equation as we sort through this.
@ebshrier quick update: @davidrkupton has a fix on CI that seems to be helping with sync issues for the main project or events. He's now working on the updates (taken from chatter feeds) that seem to have issues handling attachments.
Closing issue for now.