lag437289 msg every 30 seconds

Question

lag437289 msg every 30 seconds

ask2018 opened this issue 6 years ago · comments

ask2018 commented 6 years ago

Since I've updated to dlFilter 2.04 I'm getting
"Private: -myickname- lag437289 somenumber" every 30 seconds.
It was not there with old version 1.16 and its very annoying, because it basically spams me more than it filter the spam. Any idea what can cause this and how to get rid of it?

Sophist · Answer 1 · Sun Apr 22 2018 20:46:13 GMT+0800 (China Standard Time)

Is there a user lag437289 on any of the servers you are connected to?

If you are willing to say, which network / server is this happening on?

Which window is that appearing in? If it is a channel, is lag437289 in that channel also? If you are willing can you say which channel it is in?

(From the limited information you have provided it seems that the current version is not filtering a private message being sent to you by another user, whereas the previous version was filtering this by default. I have no idea why this user is sending you these frequent messages. If you can provide the above information I will look into it. Either way, you should be able to set a custom filter to filter it out.)

ask2018 · Answer 2 · Sun Apr 22 2018 21:05:35 GMT+0800 (China Standard Time)

There is no such user on any servers I'm connected to. It's more like some internal notice and it's posted on the current active window - doesn't matter if that's some channel or query window and doesn't matter on what irc server. All the time just the active window.

I'm connected on more mostly private irc servers and I don't think this is somehow related that somebody is sending that to me, because if I disable dlFilter it's gone.
Must be some internal message which is dlFilter causing.
I'm running MIRC 7.52 with nbs-irc - https://github.com/ElectronicWar/nbs-irc and I'm connected through ZNC 1.6.5

But trying to google that and it seems to be something inside nbs-irc code. So probably some incompatibility between nbs-irc and new dlFilter version
Look here https://github.com/ElectronicWar/nbs-irc/blob/master/scripts/main.nbs
lines 848, 890 and 1327
So the question is now if you can somehow make this compatible with nbs-irc again or can suggest me some workaround.

ask2018 · Answer 3 · Sun Apr 22 2018 21:13:32 GMT+0800 (China Standard Time)

So ticking off "Enable automatic lag check" inside nbs-irc settings fix that. Problem solved, but still not sure why this was not happening with the old 1.16 version of dlFilter.

Sophist · Answer 4 · Sun Apr 22 2018 22:00:00 GMT+0800 (China Standard Time)

I am not quite sure what the reason for this happening in v2 but not v1 is - I can think of a few possible explanations:

dlF tries to ensure that it runs before other mIRC scripts - dlF v1 also tried to do that but it is possible that it ran after the nbs-irc code.
dlF v2 does something different from v1 for notices that come from self. This is the most likely explanation - that we do something with it in v2 (i.e. echo it to the active window whereas previous we did nothing and it got processed and hidden by nbs-irc.

So - how to fix it:

a. We could change to run dlF last rather than first - not sure what impact this would have if you are also running sbClient, because dlF does stuff to make sbClient run better. We also try to run first in order to filter out the spam before another script processes it - but perhaps we shouldn't do this either. This needs careful thought before we take a decision to make dlF run last instead of first.

b. For this specific nbs-irc case, we can include a filter for it in dlF - or you can add a custom Private Notice filter for "lag437289 *" - but this is an example of a more general case of other scripts processing stuff, so we should probably try to think more widely and have a generic solution (which might be option a. above).

Sophist · Answer 5 · Sun Apr 22 2018 22:12:50 GMT+0800 (China Standard Time)

@DukeLupus Can you give your thoughts on this pls?

Sophist · Answer 6 · Thu Jul 19 2018 23:26:04 GMT+0800 (China Standard Time)

@DukeLupus - the more I think about this the more I think dlFilter should run last.

The purpose of dlFilter is to filter stuff from the mIRC UI not from other scripts. Other scripts tend to look for specific stuff - DLF runs on practically every message.

Sophist · Answer 7 · Fri Jul 20 2018 05:19:53 GMT+0800 (China Standard Time)

I have created a PR #59 to run DLF as the last script rather than the first and so avoid the symptom reported in this issue.

@ask2018 - If you could download this version from https://raw.githubusercontent.com/DukeLupus/dlFilter/62859f7537f907e4ee17b6d3dabfeff2ff94953f/dlFilter.mrc and see if it resolves your issue that would be appreciated.

ask2018 · Answer 8 · Fri Jul 20 2018 05:56:21 GMT+0800 (China Standard Time)

@Sophist-UK It doesn't resolve the issue and side effect is dlFilter filtering doesn't work at all in my client with nbs-irc.

Sander Säde · Answer 9 · Fri Jul 20 2018 15:57:14 GMT+0800 (China Standard Time)

I remember there were major issues when dlF was loaded to "wherever". I think that caused Invision (or something else) to spit our all messages to channels.

Sophist · Answer 10 · Fri Jul 20 2018 16:00:42 GMT+0800 (China Standard Time)

Well - that is why I did it as a PR and not a commit to the master branch.

If a previous script has halted and event, AFAICT mIRC is supposed to still call the matchig events in later scripts unless they use the "&" prefix, and you can see if they were halted using the $halted identifier.

If $halted is true, then mIRC is not going to display the text on screen - the PR is supposed to respect that the display has been stopped (rather than decide to echo it itself) and still process the message (which could be an advert), but also log it to the Filter window so it is not lost completely.

If dlF is not working at all with nbs-irc, the most obvious guess is that the events are not being called as they should. But it is equally possible that there is something in DLF which is failing to handle them.

Sophist · Answer 11 · Fri Jul 20 2018 16:10:44 GMT+0800 (China Standard Time)

@ask2018 Can you type /DLF.Watch and post a few examples of the debugging output here.

ask2018 · Answer 12 · Fri Jul 20 2018 19:17:26 GMT+0800 (China Standard Time)

@Sophist-UK Sure. Log is untouched, just nick, irc, channel etc is censored:
It is with "Enable automatic lag check" enabled inside nbs-irc.

(13:06:36)+000 -> irc.server PRIVMSG #channel :testing
(13:07:09)+000 -> irc.server PRIVMSG #channel :123 test
(13:07:16)+000 -> irc.server NOTICE nickname :lag437289 6039079
(13:07:16)+000 <- @time=2018-07-20T11:07:21.554Z :nickname!at@ip NOTICE nickname :lag437289 6039079
(13:07:16)+032 571 ON NOTICE called DLF.Priv.Notice: lag437289 6039079
(13:07:16)+047 571 ON NOTICE called DLF.@find.Response
(13:07:16)+063 571 ON NOTICE called DLF.@find.IsResponse
(13:07:16)+078 571 ON NOTICE called DLF.Custom.Filter
(13:07:16)+110 571 ON NOTICE called DLF.Priv.RegularUser
(13:07:16)+125 571 Not regular user: Me
(13:07:16)+141 571 Not a regular user
(13:07:16)+156 571 DLF.Win.Echo notice Private nickname lag437289 6039079
(13:07:16)+188 571 Echoed: To status window because active window is custom / listbox

And version 2.05 filtering works fine, just there is that annoying every 30s msg if I dont disable "Enable automatic lag check" inside nbs-irc.
Only the version with "run DLF as the last script" you posted above, doesn't filter anything with nbs-irc script installed and running.

Sophist · Answer 13 · Fri Jul 20 2018 19:58:55 GMT+0800 (China Standard Time)

Unfortunately this is not a helpful example, because the messages are from yourself. But it does show that at least IRC notices are being processed by DLF.

Any chance of some real-life examples in a real life channel? (Please feel free to obfuscate the actual channel name / or specific text of anything that you don't wish to disclose.)

Sophist · Answer 14 · Fri Jul 20 2018 20:06:42 GMT+0800 (China Standard Time)

P.S. I have made a tweak to the Watch functionality so please use the updated version from https://raw.githubusercontent.com/DukeLupus/dlFilter/04bbc51023ad58eb93de7f5de6199ee1d0af3369/dlFilter.mrc

ask2018 · Answer 15 · Fri Jul 20 2018 20:12:54 GMT+0800 (China Standard Time)

Yes, but please specify what exactly you need from the log, because from real life channels there is really lot of output in that window and really lot of text etc I need to obfuscate (I'm not filtering ads, but content). Also specify if you need it with the "Enable automatic lag check" in nbs-irc enabled (even more text) or disabled and specify if you need it from the broken version with "run DLF as the last script" or from the working version 2.05

Sophist · Answer 16 · Fri Jul 20 2018 20:15:59 GMT+0800 (China Standard Time)

I need it from the version last linked to.

I don't need many examples, but it needs to be a channel text message from other users.

And an example of "automatic lag check" would also be useful.

ask2018 · Answer 17 · Fri Jul 20 2018 21:02:12 GMT+0800 (China Standard Time)

ok so here is it
First of all there is this inside regular filtering window for all channels and thats the main problem probably:

(14:36:41) [#channelname] * Already halted:

this is the lag msg problem:

�14(�14�7:�37�7:�48�14)�+000 -> irc.server NOTICE mynick :lag437289 11471689
�14(�14�7:�37�7:�49�14)�+000 <- @time=2018-07-20T12:37:53.899Z :mynick!at@ip NOTICE mynick :lag437289 11471689
�14(�14�7:�37�7:�49�14)�+016 1001 ON NOTICE called DLF.Priv.Notice: lag437289 11471689
�14(�14�7:�37�7:�49�14)�+031 1001 ON NOTICE called DLF.@find.Response
�14(�14�7:�37�7:�49�14)�+031 1001 ON NOTICE called DLF.@find.IsResponse
�14(�14�7:�37�7:�49�14)�+047 1001 ON NOTICE called DLF.Custom.Filter
�14(�14�7:�37�7:�49�14)�+047 1001 ON NOTICE called DLF.Priv.RegularUser
�14(�14�7:�37�7:�49�14)�+062 1001 Not regular user: Me
�14(�14�7:�37�7:�49�14)�+062 1001 Not a regular user
�14(�14�7:�37�7:�49�14)�+062 1001 DLF.Win.Echo notice Private mynick lag437289 11471689
�14(�14�7:�37�7:�49�14)�+078 1001 Echoed: To status window because active window is custom / listbox

and this is some text from other user from channel where filtering is enabled:
but with this version of dlFilter the text filtering is not working and it shows in both - main channel windows and also filtering window

�14(�14�7:�48�7:�38�14)�+000 <- @time=2018-07-20T12:48:43.436Z :otheruser@ip PRIVMSG #channel1 : {b}{k}12{194}{183}{k}{b}
�14(�14�7:�48�7:�39�14)�+000 <- @time=2018-07-20T12:48:44.442Z :otheruser2@ip PRIVMSG #channel2 :{k}14[{k}10{b}{b}URL{k}14{b}{b}] this.text.should.be.filteredk}10{b}{b} another.text.should.be.filtered{o} {b}{k}12{194}{183}{k}{b}
�14(�14�7:�48�7:�39�14)�+047 843 ON TEXT called DLF.Chan.IsChanEvent
�14(�14�7:�48�7:�39�14)�+047 843 ON TEXT called DLF.Stats.Count: #channel2 Total
�14(�14�7:�48�7:�39�14)�+062 843 ON TEXT called DLF.Chan.Text
�14(�14�7:�48�7:�39�14)�+062 843 ON TEXT called DLF.Custom.Filter
�14(�14�7:�48�7:�39�14)�+078 843 Matched in custfilt.chantext by "custom.filter.pattern"
�14(�14�7:�48�7:�39�14)�+078 843 Matched in custom.chantext: custom.filter.pattern
�14(�14�7:�48�7:�39�14)�+078 843 ON TEXT called DLF.Win.Log: Filter text #channel2 otherusernick
�14(�14�7:�48�7:�39�14)�+093 843 ON TEXT called DLF.Stats.Count: #channel2 Filter
�14(�14�7:�48�7:�39�14)�+109 843 Filtered: To @dlf.Filter.network text #channel2 otherusernick

Sophist · Answer 18 · Sat Jul 21 2018 03:32:41 GMT+0800 (China Standard Time)

I decided we should send halted messages to the filter window on the basis that another script has effectively filtered it already. In my mind that is the purpose of the filter window - to be a record of what has been filtered. If you don't like this, please explain why and I can reconsider.
I had forgotten that we echo the text ourselves in some cases and then halt - I will add code to handle that situation which will stop it being displayed in channel.

Sophist · Answer 19 · Sat Jul 21 2018 05:11:17 GMT+0800 (China Standard Time)

Please download and use the revised version for testing from https://raw.githubusercontent.com/DukeLupus/dlFilter/27b22226d826a39d73ce57860a5cd29832a7db3a/dlFilter.mrc

Sophist · Answer 20 · Sat Jul 21 2018 05:12:57 GMT+0800 (China Standard Time)

If it still doesn't work please give examples of both channel / status window output, matching filter window output and matching Watch window output.

ask2018 · Answer 21 · Sat Jul 21 2018 06:01:00 GMT+0800 (China Standard Time)

Those "[#channelname] * Already halted:" msges was not in the filter window in previous version. Problem with them is it opens filter windows on all irc networks I have.
And thats additional unnacesary opened windows, because I don't filter channels on all networks I'm connected. So filter windows are needed only on networks, where filtered channels exists.
Last posted version sends the lag437289 msg to filter windows as " -nickname- Already halted: lag437289 43686270" Still not sure why is that msg there, but if thats filtered, it can be considered as fixed I guess.
But the main problem with the new version is still there. And that's filtering simply doesn't work. Text which should be filtered is sent correctly to filter window, but it remains also in main channel window as no filter exists.

This is from watch window:

�14(�23�7:�40�7:�42�14)�+000 <- @time=2018-07-20T21:40:47.201Z :usernick@ip PRIVMSG #channel :{k}14[{k}{b}{b}text{k}14{b}{b}] text.with.pattern{k}8{b}{b} text{o} {b}{k}12{194}{183}{k}{b}
�14(�23�7:�40�7:�43�14)�+031 4152 ON TEXT called DLF.Chan.IsChanEvent
�14(�23�7:�40�7:�43�14)�+047 4152 ON TEXT called DLF.Stats.Count: #channel Total
�14(�23�7:�40�7:�43�14)�+063 4152 ON TEXT called DLF.Chan.Text
�14(�23�7:�40�7:�43�14)�+078 4152 ON TEXT called DLF.Custom.Filter
�14(�23�7:�40�7:�43�14)�+078 4152 Matched in custfilt.chantext by "pattern"
�14(�23�7:�40�7:�43�14)�+094 4152 Matched in custom.chantext: pattern
�14(�23�7:�40�7:�43�14)�+109 4152 ON TEXT called DLF.Win.Log: Filter text #channel nickname
�14(�23�7:�40�7:�43�14)�+125 4152 ON TEXT called DLF.Stats.Count: #channel Filter
�14(�23�7:�40�7:�43�14)�+141 4152 Filtered: To @dlf.Filter.network text #channel nickname

this is posted msg in main channel window, which should be filtered, but it's not:

(23:40:47) (@Nickname) text.with.pattern ·

this is the same in filter window:

(23:40:43) [#channel] text.with.pattern ·

Notice that time. It's posted in channel window 4 seconds after filter window
Also text which should not be filtered is sent also to filter window with prefix " Already halted:".

Maybe the problem is nbs-irc is already somehow processing the text and if you don't load dlFilter before it, then it doesn't work.

Sophist · Answer 22 · Sat Jul 21 2018 19:15:18 GMT+0800 (China Standard Time)

Sorry that this is taking several iterations to get right - but we are getting closer.

No they weren't previously filtered with "Already halted" because DLF ran first and so other scripts could not have halted them before DLF processed them. DLF either filtered them or explicitly echoed them or left them for other scripts to process. But since DLF echoed and halted some of them, it was thus impossible for later scripts to halt and stop them being displayed. So DLF really does need to run last.
On reflection I think we should only send previously halted messages to the Watch window.
We still need to track down what is happening because I have no idea where the channel message is coming from. Could the 4 second delay actually be a different message? Also can you confirm whether your own nick or the nicks sending the message are in your mIRC notify list?
Acting on non-DLF channels is a bug and I will fix it. I will let you know when the next version is ready for testing as I have other stuff to do for a while.

Sophist · Answer 23 · Sun Jul 22 2018 05:32:17 GMT+0800 (China Standard Time)

Please retest using https://raw.githubusercontent.com/DukeLupus/dlFilter/7b8ad85ca0e689ffadcc3c9e7dfe771811de31eb/dlFilter.mrc

ask2018 · Answer 24 · Sun Jul 22 2018 19:16:31 GMT+0800 (China Standard Time)

"[#channelname] * Already halted:" are not displayed anymore in filter window. So additional unnacesary opened windows problem is fixed.
lag437289 msges are not displayed anymore in filter window or anywhere else. That's also better now, so also fixed.
Only problem that remains is filtered msges are still displayed in both windows - main channel window and filter window.

My mIRC notify list is empty and notify is not even enabled in the settings. The 4 second delay was the same msg and there is some delay for all msges in few seconds. It seems to be in +-5 seconds range mostly.

Sophist · Answer 25 · Sun Jul 22 2018 19:46:23 GMT+0800 (China Standard Time)

Yes - now sent only to the Watch window if open.

I cannot understand why there would be a lag of several seconds though unless you are running on a fairly low powered machine and DLF and other scripts are making mIRC run at 100% of a CPU core (I don't think mIRC is multi-threaded). Please provide another example with all Watch messages from the original message through until after the time that the message appears in the channel window.

ask2018 · Answer 26 · Sun Jul 22 2018 20:51:26 GMT+0800 (China Standard Time)

Also no idea why there is the delay, but it's not cpu issue. The cpu load is around 5-10% included all other processes. Here is the log:

filter window

(14:34:07) [#channel] <@UserNick> text text.with.filtered.pattern text ·

main channel window

(14:34:09) (@UserNick) text text.with.filtered.pattern text ·

watch window

�14(�14�7:�34�7:�06�14)�+000 <- @time=2018-07-22T12:34:08.611Z :usernick!~usernick@ip PRIVMSG #channel :{k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{194}{183}{k}{b}
�14(�14�7:�34�7:�06�14)�+046 683 ON TEXT called DLF.Chan.IsChanEvent: usernick/usernick:
�14(�14�7:�34�7:�06�14)�+062 683 Stats: Total 6, Filtered 5: #channel Total
�14(�14�7:�34�7:�06�14)�+078 683 Is DLF channel event: usernick in #channel
�14(�14�7:�34�7:�06�14)�+093 683 ON TEXT called DLF.Chan.Text: : {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b}
�14(�14�7:�34�7:�06�14)�+124 683 Filtered: Already halted by previous script: {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b}
�14(�14�7:�34�7:�06�14)�+156 683 ON TEXT called DLF.Custom.Filter: : chantext {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b}
�14(�14�7:�34�7:�06�14)�+187 683 Matched in custfilt.chantext by "pattern"
�14(�14�7:�34�7:�06�14)�+202 683 Matched in custom.chantext: pattern
�14(�14�7:�34�7:�06�14)�+218 683 ON TEXT called DLF.Win.Log: Filter text #channel usernick: {k}14[{k}{b}{b}text{k}14{b}{b}] text.with.filtered.pattern{k}8{b}{b} text{o} {b}{k}12{183}{k}{b}
�14(�14�7:�34�7:�06�14)�+249 683 Stats: Total 6, Filtered 6: #channel Filter
�14(�14�7:�34�7:�06�14)�+265 683 Filtered: To @dlf.Filter.network text #channel usernick

Sophist · Answer 27 · Mon Jul 23 2018 04:40:10 GMT+0800 (China Standard Time)

I cannot understand where this lag is coming from - unless one of the other scripts is halting the message and using a timer to delay showing the message.

ask2018 · Answer 28 · Mon Jul 23 2018 05:18:38 GMT+0800 (China Standard Time)

No idea and only scripts I have here is the already mentioned nbs-irc and then fish 10 from here:
https://syndicode.org/fish_10/

Sophist · Answer 29 · Mon Jul 23 2018 05:46:53 GMT+0800 (China Standard Time)

A quick read through of the nbs-irc script shows that it intercepts and halts every channel text message in every channel (see https://github.com/ElectronicWar/nbs-irc/blob/b64f76a02ad59d8b961faa0ef54f3cbf5b198236/scripts/main.nbs#L4225) and then haltdef's it - in other words every line is halted - and then echos it out later - in other words, I would rate nbs-irc as being very poorly written because it is indiscriminate in what it does.

Sophist · Answer 30 · Mon Jul 23 2018 20:55:58 GMT+0800 (China Standard Time)

The bottom line is that DLF and nbs-irc both execute on every mIRC event, and two scripts which both halt the message and then echo it instead of letting mIRC display it are effectively incompatible.

DLF only halts messages that it is displaying in the non-default window. nbs-irc halts everything. DLF deliberately runs last (as per this PR) because it does this, nbs-irc does not consider the consequences and does not position itself cf. other scripts.

In summary, DLF and nbs-irc (as it stands today) are fundamentally technically incompatible. If nbs.irc stops halting everything, it can be made more compatible.

ask2018 · Answer 31 · Tue Jul 24 2018 15:55:16 GMT+0800 (China Standard Time)

I know nbs-irc is not ideal, but I've found it as closest replacement for NoNameScript which was no more compatible with latest mirc and there is no other alternative as far as I know. Those few seconds lags are not much problem for me and filtering is working with DLF version 2.05.
So my question is now can I just change the loading order back to first and it will work again for me as version 2.05 or the other changes you did are not compatible with load as first as before? And then I guess we can close this issue. Thank you for support.

Sophist · Answer 32 · Tue Jul 24 2018 16:04:06 GMT+0800 (China Standard Time)

If you download and switch back to the version from the master branch, it will load first.

However I am still pondering whether to merge the PR that changes it to load last - because I think that may be the most appropriate approach overall.

My advice to you would be to get nbs-irc fixed - perhaps submit the necessary changes yourself.

(I will raise an issue on their Github repo though.)

Sophist · Answer 33 · Tue Jul 24 2018 16:39:56 GMT+0800 (China Standard Time)

Issue raised on nbs-irc: ElectronicWar/nbs-irc#17 though unsure that this is actively maintained and someone will address it.

My current plan will be to merge the PR except for the bit that moves the script to run last. So for the moment it will continue to run first, but will contain all the other code necessary to handle the situation where another script halts messages first.

I may also add a global option to allow the user to decide whether the script should run first or last, with running first being the default.

Sophist · Answer 34 · Wed Aug 08 2018 21:39:27 GMT+0800 (China Standard Time)

I have now merged #59 and included an option to allow the user to decide whether to run it first or last. I have a couple of quiet days which I can use to code dlF stuff (for other issues) and then I intend to bump the version to 2.07 so updates will be offered to users.

Sophist · Answer 35 · Wed Aug 08 2018 21:40:54 GMT+0800 (China Standard Time)

I am now closing this issue as AFAIK there is no more to be done to fix it - fixes being needed in https://github.com/ElectronicWar/nbs-irc/ to make that script more compatible with other scripts like dlF which process and halt lots of messages.