Panopticon tries to update Joomla but there is no new version
Martin511 opened this issue · comments
Due diligence
- I have searched all open and closed issues for this problem.
- I have searched all open and closed discussions for this problem.
- I have looked at the CHANGELOG to make sure this problem has not already been addressed.
- I agree to the Code of Conduct of this repository
Describe the bug
Panopticon tries to update Joomla but there is no new version.
See Screenshot.
Reproduction instructions
See screenshot.
Expected behaviour
There is no new version so updating is not necessary.
Actual behaviour
Panopticon tries to update Joomla but there is no new version.
Screenshots
Panopticon version
1.1.1
PHP version
8.2.13
Database type
MariaDB
Database version
10.5.23
Browser
Firefox
Browser version
No response
Additional information
No response
It looks like Joomla! believes there is no version to download, even though it reported a new version earlier. That's weird.
What is even weirder is that your site is on Joomla! 4, and the reported version is 5.0.3.
Can you please go to your site's System, Update, Joomla, Options and tell me what is the Update Channel selected?
Wait, there is a disparity in your screenshots.
The email screenshot says site number 15, and your second screenshot is site 26. Can you please send me a screenshot of the correct site so I can actually see what is going on?
Sorry, that was my fault.
Here is the screenshot of site 15.
The site´s Update-Server is "Standard".
I have 27 sites connetcted in Panopticon.
The "Unable to Upgrade" messenges came for 21 sites.
But I can´t figure out what is different in these 6 sites...
It is not impossible. My sites were parallel connected to Yoursites as a "backup" because Panopticon is quite new.
Of course the backups and updates were disabled there. But site checks were running.
So now I disconnected / deleted all sites from Yoursites to make sure that this can´t be an influence in the future.
All 6 sites are on "Standard" Update Server
Yeah, if something else did update the site while Panopticon was trying to update your site it makes sense that you got this weird error. It came up with a situation that should have been impossible to occur.
Regarding your sites, please note that I told you to look at the tab in Panopticon, not Joomla itself.
Regarding your sites, please note that I told you to look at the tab in Panopticon, not Joomla itself.
All are on "Use Global" - and global setting is "Send Email".
That's why they are not updated. Set it to Minor and they will be scheduled for an update next time there is a Joomla version available.
No, I will decide by myself if a Joomla Update is done or not.
I only want to be informed about new Updates.
The "Use Global" - and global setting "Send Email" is set for all my sites, not only for the 6 mentioned.
Then the next time these sites do find an update you will receive an email.
Please note my wording: the next time these sites do find an update.
When I say "these sites do find an update" I mean it. It's not Panopticon finding an update. About that, please read https://github.com/akeeba/panopticon/wiki/Periodic-information-update#the-refreshsiteinfo-task where I've written this:
Please remember that Panopticon does not determine whether there is an update for Joomla!. This is something that Joomla! itself determines and conveys that information to Panopticon.
When I say "the next time" I also mean it literally. Panopticon "remembers" which was the last available update version it took an action for (even if the action was "None" at the time). Therefore, what is done is done for these sites right now. The next time Joomla reports a higher version number is available, Panopticon will see it and send you an email.
Now, whether you receive the email is also a different subject. It is absolutely possible that all 27 sites send an email, but you only receive 21 emails because the name of the other 6 sites triggered the spam filter on either the sending or the receiving host, you have Gmail which delays delivery of email messages by as much as a week when they feel an address has sent you "too many" emails (the definition of "too many" being nebulous, inconstant, and inconsistent – probably because they use some "AI" bullshit), you have legitimately gone over your email provider's maximum sending limit per minute / hour / day, or any number of weird reasons. Sending an email is no longer a straightforward affair like it mostly was in the early 00s.
Thank you for these informations.
Regarding emails I don´t use gmail or something else for this. I use my own hosted domain-email. Additionally these are whitelisted so they won´t be filtererd as spam.
I have no worries here.
Thanks.
Again I get error E-Mails for 17 of 27 sites, saying:
_"Unable to upgrade “(Unknown Site)” to Joomla!® (Unknown)
Panopticon was unable to install the new Joomla! version (Unknown) on your site “(Unknown Site)”. The error received is as follows:
The site 21 (here is the correct site name displayed) is on Joomla 5.1.0. The latest available version is 5.1.0. Therefore, it cannot be updated."_
In Panopticon dashboard there are no marks about new Joomla Versions (what is correct as there IS no new Joomla Version).
In "Core Update" tab I see a bell at 23 of 27 sites with the text "ACTION REQUIRED. The Joomla!@ update task is stuck."
In site details there is a warning: "The core files refresh ran into an error: Timed out"
I have no idea whats going on here.
There is no other service active (like Yoursites).
Why do I get 17 Mails if 23 sites of 27 in total have problems?
Why does the E-Mail is for "unknown sites, but at the bottom of the E-Mail the correct site is named?
Regards
Martin
Are you sure you are using Panopticon 1.1.3? I have it installed on two separate dev servers and a yet different live server. I have two dozen sites which were automatically updated by these installations without as much as a peep.
Moreover, the email claims that the site is already on Joomla! 5.1.0. Did you receive another email saying it got updated? Do you see the update taking place in the Overview, Site Reports page?
If not, something else is updating your sites. Do keep in mind that your host may be doing that. For example, if you install a site with Softaculous it will keep updating your site automatically even if you replace the original site e.g. by restoring the backup of a different site onto it. Softaculous is NOT the only hosting-side solution which could possibly do that.
Are you sure you are using Panopticon 1.1.3?
Yes.
Did you receive another email saying it got updated? Do you see the update taking place in the Overview, Site Reports page?
Both: No.
If not, something else is updating your sites. Do keep in mind that your host may be doing that.
No, my host verified that they do not update sites and that they have no software running which could do that.
I´m really confused. I don´t blame Panopticon for this, I´m sure - as you said - that everything works. I believe that it is a problem in my sites but I don´t know where to start solving this.
Does it make sense to set new login passwords for all these sites?
Any tipps for further investigation?
Something or someone is updating those sites. I mean, they are updated! You didn't update them, your host didn't, a different service didn't... who did?!
Is it possible that you have another installation of Panopticon?
Is it possible you have another employee, business partner, or client going in and updating the sites?
Is it possible that you have another installation of Panopticon?
No.
Is it possible you have another employee, business partner, or client going in and updating the sites?
No.
Regarding your question if I see the update taking place in the Overview, Site Reports page:
Not in Overview and not in Site Reports but in "Core Updates" they are listed. (Ready for updating from Version 5.1.0 to 5.1.0 ...)
Even if "something or someone" would try to update: How would that be possible to update a Joomla Site that is already on the latest Version? Joomla would say that there is no update availabe, or not? For me this seems technical impossible.
The timeline is pretty evident, isn't it?
Panopticon detects there is an update. It schedules it to be installed.
Someone or something updates it outside of Panopticon.
Panopticon tries to install the update... but the site is already updated.
May I ask what you would do in my place now?
I still don´t know where to start investigating.
First of all, I have never once been in a position that I have no idea what is going on on my sites. Ever. Not once ever since I started building sites in 1998.
What you should do is first figure out what is going on with your sites. And this, you can only best -- if not only -- do by nuking everything from high orbit.
Remove the sites from the current Panopticon installation, remove Panopticon's CRON jobs, and finally delete Panopticon's files and database.
D the same thing on any kind of third party software or service you have ever used for site monitoring.
Then, wait for Joomla's next update. Does something magically update your site?
If yes, re-read everything I told you. If you're not doing it, no third party service / software is doing it, your host is not doing it, and no other person is doing it then the only reasonable conclusion is that you have forgotten something. What? I don't know. I can't know. It's not my sites, and not my business to know.
If not, congrats, you had already nuked the culprit. Go reinstall Panopticon.
Thank you very much for taking the time to help me. I really appreciate it.
This all makes absolutely sense and I will exactly do this.
Thank you.
You're welcome! And remember to document what you do as you go. It will save you a lot of trouble in the future ;)
First of all, I have never once been in a position that I have no idea what is going on on my sites. Ever. Not once ever since I started building sites in 1998.
What you should do is first figure out what is going on with your sites. And this, you can only best -- if not only -- do by nuking everything from high orbit.
Remove the sites from the current Panopticon installation, remove Panopticon's CRON jobs, and finally delete Panopticon's files and database.
D the same thing on any kind of third party software or service you have ever used for site monitoring.
Then, wait for Joomla's next update. Does something magically update your site?
If yes, re-read everything I told you. If you're not doing it, no third party service / software is doing it, your host is not doing it, and no other person is doing it then the only reasonable conclusion is that you have forgotten something. What? I don't know. I can't know. It's not my sites, and not my business to know.
If not, congrats, you had already nuked the culprit. Go reinstall Panopticon.
Hello Nicholas,
I did everything you suggested.
The last update from Joomla 5.1.0 to 5.1.1. was completed without any irregularities.
Nothing magically tried to update my sites.
So now I would like to restart to use Panopticon again.
As I didn´t notice any irregularities -exept the one in the post above which only appearde on one site- after disconnecting the sites and uninstalling everything that was linked with Panopticon and all former monitoring services: Is there anything I additionally should check before I install Panopticon again and add these sites again? I really want a clean restart to make sure the issues of this thread do not happen again.
Go ahead and relink your sites to Panopticon.
For what it's worth, I have a Panopticon installation in production which updates our business site, and my blog. My wife has another Panopticon installation in production, hosted with a different hosting company, for her clients' sites. These installations always use the latest published version; I have another installation which uses the in-development version, but I am excluding it from any support discussion as it's code you guys and gals don't typically run (except, maybe, Brian who installs from source). In any case, neither me nor my wife have ever observed anything like what you described on our production installations.
I have two installs - one from source for testing and a live install using the published version
@brianteeman Ah, I only knew about the from source one. Makes sense you have two :)
Here we are again...
How are you triggering the Panopticon tasks? CLI CRON jobs or web CRON? How often? How many do you have?
I have one web Cron running every minute.
I just checked if the "Unable to Update" affected the same sites as in February, as not all sites are affected.
But it is not. It is different.
This is a pity, because it would have been very helpful in the search for errors if it had only affected certain websites.
Sorry to report: I am also getting these mails: Unable to upgrade “(Unknown Site)” to Joomla!™ (Unknown)
When I look at Overview > Core Updates most sites (80 out of 96) show the bell symbol telling me:
"ACTION REQUIRED. The Joomla!® update task is stuck."
Which is simply not true: most of the updates to J5.1.2 went without a glitch about 3 days ago.
A few sites (4) needed a second try the next day. One site was stuck, but this was due to an empty #__tuf_metadata, so after manually running the SQL, it updated to J5.1.2 just fine.
There is no second install, nor is there any other process trying to update the sites, I am on v1.1.3.
All sites use the same global settings: Install Minor versions only - for Joomla and Extension Updates.
Affected sites are across different servers. I see absolutely no difference, in fact there is a series of cloned sites, that are basically identical technically, but one of them is not affected.
Very strange.
Also, I got 104 mails, but have only 96 sites.
@r3dvorak Your issue is completely unrelated. The Joomla! update task can get stuck, e.g. when Joomla! fails to contact its update server. It happens on random network conditions. The issue we are discussing here has to do with web CRON jobs. Please don't hijack the thread with something irrelevant, it makes it impossible for me to work.
@r3dvorak
I am so happy that I am not alone.
May I ask where you host Panopticon?
@Martin511 The two issues are UNRELATED.
@nikosdion But there are similarities. I also have the the bell symbol telling me:
"ACTION REQUIRED. The Joomla!® update task is stuck." at these sites.
@nikosdion Sorry, it looked the same to me. Should I start a new thread then?
@Martin511: Panopticon hosted on one of my root servers @ hetzner
@Martin511 The problem that you have has to do with using web-based CRON jobs. Because they can only do one thing (they do not wait around to fetch tasks, and/or perform more than one task at a time) you end up with a situation where an update is scheduled after it has been installed. That's problem one.
Further compounding your problem, when an update fails over a web CRON job you get an email which does not list that actual site name. That's problem two.
What @r3dvorak has is actually UNRELATED. He's using CLI CRON jobs and his sites are updating fine. However, some sites report that the Joomla! update task is stuck. Yes, this does indeed happen when the site itself cannot connect to Joomla's update server (https://update.joomla.org), or when an internal error happens in Joomla's com_joomlaupdate (e.g. the TUF root metadata is gone, or the #__tuf_metadata table is broken – understanding that required a week-long diversion into how TUF works I just pulled myself out of, you're welcome).
There is a good reason why we require manual intervention in this case. We cannot know if it's a permanent issue (the host did something stupid with networking, the TUF metadata table is gone, the TUF root metadata is missing, the download of the update package takes so long that it times out – an issue already reported years ago to Joomla, etc) or something temporary (the Joomla! update server is temporarily down, the network on the hosting side had a hiccup, the download was too slow because everyone is updating etc).
If we reset the task and retry and it fails again you will get yet another failure email. This would keep happening up to 4 times per hour for every site with a failed update. If you have 100 sites and the updates are taking place you're asleep, you might wake up to hundreds upon hundreds of emails. Who are you going to shout at, then? Your host and Joomla for actually screwing up, or me? Right. That's why a manual intervention is needed and it won't be automated.
In the future (see #721) we will be able to identify and fix TUF issues, so we might indeed allow recovery from these on Joomla! 5.1 and later. All other issues will still require manual intervention. That's why I said that this is something else and implied I can't "fix" it. Sure I can, but the fix is worse than the problem in many cases, so let's not even go there. When an update fails, go to your Panopticon installation, reset the task, schedule an update; or use the core updates page which does the same, basically, but with far less clicks.
Same here - no problem with the update check failing. shit happens. But panopticon is confused about which site etc. Twice it says unkown. Once it says example.com and once it gets the correct site name
So this part was as expected but the rest was not
@brianteeman Yes, the problem is that when the site is already up-to-date the email does not include the site name. This is fixed now (I just pushed the fix).
About the double scheduling, all I can think is that two or more update director tasks run at the same time, but I have not been able to reproduce it either organically or by forcing it.
glad you were able to fix the bug part
@nikosdion thank you for the in-depth explanation.
Hi there,
we have exactly the same problems as Martin511:
- Emails with "Unknown Site"
- Core Files refresh error: "Threw exception" (in 2 out of 25 cases)
- Core Files refresh error: "The core files refresh ran into an error: Timed out" (after a functioning update in 90% of all cases)
- Sometimes there is no email notification when the update has been carried out (in 17 out of 25 cases)
- Message "ACTION REQUIRED. The Joomla!® update task is stuck."
We have currently only connected the first 25 websites for testing, but they were all created from an original J!-Installation.
We noticed that the API code is the same for all installations.
Unfortunately, we currently only have webCron. Is our execution time of 60 seconds too long?
Thanks for your support!
@Picannick You do not have the same issue. Your issue is more like what @r3dvorak had.
Please read my detailed explanation.
To clarify, the timeout comes from Joomla! trying to download its update package. It has nothing to do with your Panopticon web pseudo-CRON not having enough capacity. It times out because Joomla! itself times out. I explain that and why the task is not reset in the link above.
BTW, Joomla's update download being dog slow and timing out was reported about 2 years ago to the project by core contributor Allon Moritz, and I gave them the solution: use CloudFlare (ironically, something the Joomla! project has already set up) instead of the Amazon S3 bucket to deliver update packages. Not only that would solve this problem, it would make updates faster for most people, scale better, and save a shitload of money the project is currently paying to Amazon (even if it mostly comes in the form of free credits). They chose to ignore the solution, and make the Amazon S3 bucket the only update source, thus making updates even slower and less reliable for everybody. I can fix code, I can't fix stupid.
The problem that you have has to do with using web-based CRON jobs. Because they can only do one thing (they do not wait around to fetch tasks, and/or perform more than one task at a time) you end up with a situation where an update is scheduled after it has been installed. That's problem one.
@nikosdion I´m a little confused now.
You told me that the problem is my web-based Cron.
Now you tell @Picannick that it has nothing to do with the web-based Cron because Joomla itself times out.
Do I understand correctly that because Joomla itself times out, the web-based Cron is not ideal (but could be if Joomla was better "configured") so we should better change to a CLI CRON solution? Is that briefly correct so far?
Let me put it this way. You both drive down the same road using the same make and model of car. One gets a flat tire. The other one sees steam coming out of the engine. Is it the same issue?
So no. Just because you both use web CRON doesn't mean you have the same issue when you get totally different issues. You get duplicate updates. He gets stuck updates. Totally different.