Cloudflare reverse proxy purge not working
davidwebca opened this issue · comments
Bug Report
Hi,
I have configured Blitz first and everything's working fine and now since I added Cloudflare in front with reverse proxy purging, I don't see the content update ever. When updating a page, I was expecting Cloudflare to be purging the URL of the entry updated and all its related entries (trackElements and trackElementQueries are both set to true).
Diagnostics Report
Application Info
- PHP version: 8.2.18
- Craft edition & version: Pro 4.8.3
- Database driver & version: MariaDB 10.6.17
Installed Plugins
- Blitz: 4.17.0
- Field Manager: 3.0.8
- Fix FKs: 2.0.0
- Formie: 2.1.7
- GDPR Cookie Consent: 4.0.4
- Hyper: 1.1.22
- Icon Picker: 2.0.17
- Inline SVG: 2.0.0
- Maps: v4.x-dev
- Navigation: 2.0.27
- Neo: 4.0.7
- Redactor: 3.0.4
- Redirections: 4.1.15
- SendGrid: 2.0.1
- SEOmatic: 4.0.44
- Translations: 3.2.3
- Vite: 4.0.9
Loaded Modules
- rab: rab\RentABoxerModule
- codeeditor: nystudio107\codeeditor\CodeEditor
- sprig-core: putyourlightson\sprig\Sprig
- blitz-hints: putyourlightson\blitzhints\BlitzHints
- verbb-base: verbb\base\Base
- debug: craft\debug\Module
Blitz Plugin Settings
{
"debug": false,
"hintsEnabled": true,
"cachingEnabled": true,
"refreshCacheEnabled": true,
"refreshMode": 3,
"includedUriPatterns": [
{
"siteId": "",
"uriPattern": "*"
}
],
"excludedUriPatterns": [
{
"siteId": "",
"uriPattern": "caedfb0c-010c-4f8a-a371-c8dedf9fcbe2"
}
],
"cacheStorageType": "putyourlightson\\blitz\\drivers\\storage\\FileStorage",
"cacheStorageSettings": {
"folderPath": "@webroot\/cache\/blitz",
"compressCachedValues": ""
},
"cacheStorageTypes": [],
"cacheGeneratorType": "putyourlightson\\blitz\\drivers\\generators\\LocalGenerator",
"cacheGeneratorSettings": {
"concurrency": "5"
},
"cacheGeneratorTypes": [],
"customSiteUris": [],
"cachePurgerType": "putyourlightson\\blitz\\drivers\\purgers\\CloudflarePurger",
"cachePurgerSettings": {
"zoneIds": {
"5f474987-e317-4e5e-8177-1021a44afefc": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
},
"437c9636-97d8-4461-805c-b7aae67cadac": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
},
"bd2d0bfb-89c8-4f7a-bf21-89928ee29374": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
}
},
"authenticationMethod": "apiKey",
"apiToken": "****************************************",
"apiKey": "*************************************",
"email": "web@rent-a-boxer.com"
},
"cachePurgerTypes": [],
"deployerType": "putyourlightson\\blitz\\drivers\\deployers\\DummyDeployer",
"deployerSettings": [],
"deployerTypes": [],
"ssiEnabled": false,
"ssiTagFormat": "<!--#include virtual=\"{uri}\" -->",
"detectSsiEnabled": true,
"esiEnabled": false,
"queryStringCaching": 0,
"includedQueryStringParams": [
{
"siteId": "",
"queryStringParam": ".*"
}
],
"excludedQueryStringParams": [
{
"siteId": "",
"queryStringParam": "gclid"
},
{
"siteId": "",
"queryStringParam": "fbclid"
}
],
"apiKey": "",
"generatePagesWithQueryStringParams": true,
"purgeAssetImagesWhenChanged": true,
"refreshCacheAutomaticallyForGlobals": true,
"refreshCacheWhenElementMovedInStructure": true,
"refreshCacheWhenElementSavedUnchanged": true,
"refreshCacheWhenElementSavedNotLive": false,
"cacheNonHtmlResponses": false,
"trackElements": true,
"trackElementQueries": true,
"excludedTrackedElementQueryParams": [],
"cacheElements": true,
"cacheElementQueries": true,
"cacheDuration": null,
"nonCacheableElementTypes": [],
"sourceIdAttributes": [],
"liveStatuses": [],
"integrations": [
"putyourlightson\\blitz\\drivers\\integrations\\CommerceIntegration",
"putyourlightson\\blitz\\drivers\\integrations\\FeedMeIntegration",
"putyourlightson\\blitz\\drivers\\integrations\\SeomaticIntegration"
],
"defaultCacheControlHeader": "no-cache, no-store, must-revalidate",
"cacheControlHeader": "public, s-maxage=31536000, max-age=0",
"cacheControlHeaderExpired": "public, s-maxage=5, max-age=0",
"sendPoweredByHeader": true,
"outputComments": true,
"refreshCacheJobPriority": 10,
"driverJobBatchSize": 100,
"driverJobPriority": 100,
"queueJobTtr": 300,
"maxRetryAttempts": 10,
"maxUriLength": 255,
"mutexTimeout": 1,
"commands": [],
"injectScriptEvent": "DOMContentLoaded",
"injectScriptPosition": 3
}Recommendations
- ❌ Blitz is configured to refresh cached pages when an element is saved but remains unchanged.
- ❌ One or more globals exist and refreshCacheAutomaticallyForGlobals is enabled.
- ❌ The Blitz Hints utility is reporting 2 eager-loading opportunities.
- ❌ The blitz/cache/refresh-expired console command has not been executed within the past 24 hours.
- ✅ Blitz is configured not to refresh cached pages when an element is saved but not live.
- ✅ Image transforms are configured to be generated before page load.
- ✅ The @web alias is explicitly defined.
- ✅ Queue jobs are configured not to run automatically via web requests.
- ✅ The Async Queue plugin is not installed or enabled.
Site Tracking – EN
- Tracked Pages: 54
- Tracked Includes: 0
- Tracked Query String Params: 0
- Tracked Elements: 412
- Tracked Element Queries: 30
- Tracked Tags: 4
Site Tracking – DE
- Tracked Pages: 54
- Tracked Includes: 0
- Tracked Query String Params: 0
- Tracked Elements: 417
- Tracked Element Queries: 30
- Tracked Tags: 4
Your configuration looks fine. Do you see any error messages when saving the Blitz plugin settings from the Cloudflare purger? And have you followed the instructions for enabling static page caching in Cloudflare at https://putyourlightson.com/plugins/blitz#cloudflare-page-rules?
Yes, I followed everything for static page caching and it all works, I've just re-verified just to be sure, and no I don't see any error. To me, it seems like Blitz's cache is being refreshed correctly when saving an element. If I put Cloudflare on "dev mode" temporarily and save an element, everything works fine but not Cloudflare.
One thing that might be specific to my setup is that my host doesn't have a cron job editor so I have setup a remote call to my cron job (excluded by the cache of course) that runs the Craft queue. I've recreated the "run queue" call with some modifications and included Blitz's refreshExpiredCache method call in there so that they get added to the queue right before running it.
<?php
namespace rab\controllers;
use Craft;
use craft\helpers\App;
use craft\helpers\Json;
use craft\elements\Entry;
use craft\web\Controller;
use craft\queue\QueueInterface;
use yii\web\Response;
use putyourlightson\blitz\Blitz;
class MainController extends Controller
{
protected array|bool|int $allowAnonymous = ['run'];
private QueueInterface $queue;
public function init(): void {
parent::init();
$this->queue = Craft::$app->getQueue();
}
public function actionAll() {
// return $this->getView()->render();
}
public function actionRun() {
// Prep the response
$this->response->content = '1';
// Make sure the queue isn't already running, and there are waiting jobs
if ($this->queue->getHasReservedJobs() || !$this->queue->getHasWaitingJobs()) {
return $this->response;
}
// Cron job refreshes expired cache
Blitz::$plugin->refreshCache->refreshExpiredCache();
// Run the queue
App::maxPowerCaptain();
$this->queue->run();
return $this->response;
}
}Maybe that's the source of the problem? Should I be adding something specific to Cloudflare in there?
I don't know the code base really well and I'm probably wrong but it seems to me that the refreshExpiredCache doesn't call the cachePurger, so maybe I should create a queue job that runs right after and calls something like
Blitz::$plugin->cachePurger->purgeUris($purgeableSiteUris);?
Your code looks fine to me and shouldn’t be causing any issues with the cache purger.
Are you seeing any errors in the Blitz log file at storage/logs/blitz-*.log?
No errors and in the queue logs I also get this.
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117837] Refreshing Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5449216}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117837] Refreshing Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.014s) {"memory":5557688}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117838] Refreshing Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5565608}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117838] Refreshing Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.005s) {"memory":5586192}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117846] Refreshing Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5594112}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117846] Refreshing Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.005s) {"memory":5614696}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117847] Purging Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5634968}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117847] Purging Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.982s) {"memory":5657776}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117848] Generating Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5667240}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117848] Generating Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.007s) {"memory":5528624}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117849] Purging Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5552936}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117849] Purging Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.689s) {"memory":5568600}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117850] Generating Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5577096}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117850] Generating Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.002s) {"memory":5625728}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117851] Purging Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5650096}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117851] Purging Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.807s) {"memory":5665888}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117852] Generating Blitz cache (attempt: 1, pid: 27618) - Started {"memory":5683544}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::afterExec] [117852] Generating Blitz cache (attempt: 1, pid: 27618) - Done (time: 0.002s) {"memory":5732176}
2024-05-04 20:45:33 [queue.INFO] [craft\queue\QueueLogBehavior::beforeExec] [117839] Generating motorcycle sitemap (attempt: 1, pid: 27618) - Started {"memory":5824176}
So purging the cache appears to be working. Do you see the following messages in the Blitz log file and are they what you would expect to see?
[2024-04-30 03:04:14] 10 page(s) purged.
After verification, the cron job doesn't log anything in Blitz logs, even when trying to force the 'monologTargetConfig''s level to Psr\Log\LogLevel::DEBUG. Is the call to Blitz::$plugin->refreshCache->refreshExpiredCache() not enough? 🤔
I've continued to dig in since I have time to spend on this project this afternoon and I've found that adding Blitz::$plugin->cachePurger->purgeAll() right before Blitz::$plugin->refreshCache->refreshExpiredCache() does the trick and purges Cloudflare and Blitz correctly. I'd like to dig in more to actually only purge what is necessary and not "all". 🤔
Since you’re using the refresh mode Clear the cache and regenerate in a queue job, modifying any entry should result in pages that output its fields to be regenerated and purged immediately. Calling refreshExpiredCache() is only relevant in this case for entries with future post or expiry dates.
I noticed some possible issues with your URI patterns.
"includedUriPatterns": [
{
"siteId": "",
"uriPattern": "*"
}
],
"excludedUriPatterns": [
{
"siteId": "",
"uriPattern": "caedfb0c-010c-4f8a-a371-c8dedf9fcbe2"
}
],
Please change "*" to ".*", since this is a regular expression.
Also, is the URI pattern caedfb0c-010c-4f8a-a371-c8dedf9fcbe2 intentional?
"cachePurgerSettings": {
"zoneIds": {
"5f474987-e317-4e5e-8177-1021a44afefc": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
},
"437c9636-97d8-4461-805c-b7aae67cadac": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
},
"bd2d0bfb-89c8-4f7a-bf21-89928ee29374": {
"zoneId": "9ede78a1cc4bad23789b5d9aa4d9b4f9"
}
},
Each of the 3 sites share the same zone ID. Is that because they all share the same subdomain?
Everything else looks fine, apart from the fact that the number of pages purged is not being logged in blitz-*.log.
Any update on this, @davidwebca?
Hi! Sorry for the delay responding. This has caused much of a headache and we had weird interactions between Cloudflare's cache and Google Tag Manager / Google Ads retargeting (it was injecting stuff in forms' action URL to enable a "privacy-friendly" way to track, but Cloudflare sometimes wouldn't purge it depending on the region, even when going on their admin interface).
In the end, we had to remove Cloudflare altogether which is sad because I was also using their GeoIP rules to set the default website depending on the location of the visitor.
So to answer your question: the three websites are 3 different languages on the same domain in Craft, so that's why the zoneId is the same.
In the end though, regarding this current issue, it seems like the reverse purge doesn't work on save and has to be called again later, so my idea of the queue task in a cron job in the previous comments was working. I don't know if it's something wrong with the order that the reverse proxy purge code runs or if it's something with my setup - OPCache maybe? - but that's how I had solved it in the end before pulling the plug on Cloudflare.
This issue was very likely specific to your setup as I haven’t had any such reports. If you ever come back to using Cloudflare for this site and want to dig in further then just let me know.