devanshbatham / CertEagle

Weaponizing Live CT logs for automated monitoring of assets

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Error connecting to CertStream

joeldeleep opened this issue · comments

I am getting this error as shown below, is this intended behaviour ?

Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...
Error connecting to CertStream - Connection is already closed. - Sleeping for a few seconds and trying again...

In between the error lines if a match is found it is pushed to slack . When I checked with the help of certstream package , I got the below output.

certstream 
[INFO:certstream] 2020-08-11 10:54:44,338 - Connection established to CertStream! Listening for events...
[2020-08-11T10:54:44.328468] ct.googleapis.com/logs/argon2021/ - *.admin-eu2.cas.ms
[2020-08-11T10:54:44.327114] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.326011] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.324387] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.322587] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.320814] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.318968] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.317324] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.315464] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com
[2020-08-11T10:54:44.313734] ct.googleapis.com/logs/argon2021/ - sni.cloudflaressl.com

Hello,

The information you provided seems incomplete, provide me the details of the following :

  • Which VPS are you using
  • Is you internet connection is stable
  • Which OS you are using

I tested this on Azure VPS and it's running from past 2 months continuesly, working for me !

I am into this anyways, will let you know if I will find the root cause

Thanks

Hai there ,

  1. VPS - Digital Ocean
    2.Connection is stable , I am running other tools and scripts without issues
    3.Ubuntu 20.04

As I mentioned int the issue , the certsream command is working without any problems . I think maybe filtering the domains are causing the error , since there is no match and when the match is there it is pushed to slack .

same problem

Hello

After digging into this I came to know that the error message is emitted by websocket-client package

  • If you are using Python2x , then you will get this error (use python3.6.9)
  • I downgraded the websocket-client version in requirements.txt from 0.57.0 to 0.48.0 , so uninstall the previously installed version of websocket-client and install version 0.48.0

Note : Yes you will get these errors in between sometimes as the websocket connection will get closed and connected again , that is expected behavior , But if you are only getting these errors and no output then let me know !

I am looking for a permanent fix, I will push the changes as soon I will figure out something

rgds
Devansh

Kindly note the below error for certstream

ERROR: certstream 1.11 has requirement websocket-client>=0.56.0, but you'll have
 websocket-client 0.48.0 which is incompatible.

Hello,

Turns the problem is more tricky then speculated, I am looking for a fix , and will update if any success !

Thanks

Change requirements.txt to show the same as I have.
certstream==1.10
websocket-client==0.48.0

It will work after this. Went from not working to working with just this change.

Hello,

I pushed the changes! Hopefully, it should work fine now

Let me know if the error persists.

rgds
Devansh

I think its ok now , I will update if any issue is there .