Cloudflare blocked

[Kevin Pham]

When I try something like:

https://ladder.mydomain.com/https://medium.com/mapresearch/does-the-fourth-industrial-revolution-have-a-new-path-818723d06030

I get:

Sorry, you have been blocked
You are unable to access medium.com

Why have I been blocked?
This website is using a security service to protect itself from online attacks. The action you just performed triggered the security solution. There are several actions that could trigger this block including submitting a certain word or phrase, a SQL command or malformed data.

What can I do to resolve this?
You can email the site owner to let them know you were blocked. Please include what you were doing when this page came up and the Cloudflare Ray ID found at the bottom of this page.

From Cloudflare.

---

I'm using the following docker-compose config:

version: '3'
services:
  ladder:
    image: ghcr.io/kubero-dev/ladder:latest
    container_name: ladder
    build: .
    #restart: always
    #command: sh -c ./ladder
    environment:
      - PORT=8080
      #- PREFORK=true
      #- X_FORWARDED_FOR=66.249.66.1
      #- USER_AGENT=Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
      #- USERPASS=foo:bar
      #- LOG_URLS=true
      #- GODEBUG=netdns=go
    ports:
      - "8080:8080"
    deploy:
      resources:
        limits:
          cpus: "0.50"
          memory: 512M
        reservations:
          cpus: "0.25"
          memory: 128Mroot@ladder01:/opt# docker-compose up -d

Do you know of a workaround for Cloudflare?

[Gianni C.]

Did you try to run the binary?

I can't reproduce that on my instance.

[Abdelrahman Essawy]

im facing the same issue,

[José Ferreira]

Also seeing this issue when trying to check anything from Medium

[Kevin Pham]

I think this is related to the default user agent. There must be a cloudflare WAF rule that detects fake Googlebot UA's. When I configure a different user agent, I don't get the Cloudflare WAF page.

USER_AGENT="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/118.0.0.0 Safai/537.36" go run main.go

However, I get a 404 with Medium now. Should this work out of the box, or does a rule for medium need to be added in ruleset.yaml?

[jontstaz]

FYI @deoxykev - using that exact same USER_AGENT, i'm now getting error code: 1010 whenever trying to visit any Medium article/page. Other sites it seems to work fine for though. Only Medium so far that's throwing that 1010 error code.

[Kevin Pham]

Looks like the changes in the ruleset.yaml worked. Thank you!

[jontstaz]

@deoxykev - Hey I've updated the ruleset with the latest but now I'm getting a 404 error on all medium URLs. I can see the article appear for a split second and then it turns to 404.

And then if i click any of the featured articles below the 404 error, I get a 500 error: