How to integrate with python requests?

[Scylla2020]

Hi I'm just wondering if this can be integrated with python requests library? What will the steps look like? I'm interested in running tests mimicking a web browser on a site that uses cloudfare. I want to use a library like requests or urllib, not browser emulation. My requests arent going through because of TLS fingerprinting since the requests library cant do this.

Normally I would request a site like this:

import requests

headers = {
    'authority': 'vplates.com.au',
    'sec-ch-ua': '" Not A;Brand";v="99", "Chromium";v="99", "Google Chrome";v="99"',
    'sec-ch-ua-mobile': '?0',
    'sec-ch-ua-platform': '"Windows"',
    'upgrade-insecure-requests': '1',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.74 Safari/537.36',
    'accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9',
    'sec-fetch-site': 'none',
    'sec-fetch-mode': 'navigate',
    'sec-fetch-user': '?1',
    'sec-fetch-dest': 'document',
    'accept-language': 'en-US,en;q=0.9',
}

response = requests.get('https://example.com/', headers=headers)

But the site site uses cloudflare so I get no response. Anyway to use ja3 to get around this?

[Scylla2020]

Thanks. I had a look at the python github but couldnt get it working as I cant see any clear examples. So using the Go code what would the above request look like? Will be a good reason for me to finally learn a bit of go but I couldnt find any examples there either

[Scylla2020]

Awesome thanks, will give it a shot

[Scylla2020]

I tried on a different site but cloudfare error still showed up. Any ideas?

package main

import (
	"net/http"
	"os"
	"github.com/89z/format/crypto"
)

const ja3 = "771,49195-49196-52393-49199-49200-52392-49161-49162-49171-" +
	"49172-156-157-47-53,65281-0-23-35-13-5-16-11-10,29-23-24,0"

func main() {
	req, err := http.NewRequest("GET", "https://vplates.com.au", nil)
	if err != nil {
		panic(err)
	}
	req.Header = http.Header{
		"sec-fetch-dest": {"document"},
		"user-agent":     {"Mozilla/5.0 (Windows NT 10.0; Win64; x64)"},
		"authority":      {"vplates.com.au"},
		"accept":         {"text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"},
	}
	hello, err := crypto.ParseJA3(ja3)
	if err != nil {
		panic(err)
	}
	res, err := crypto.Transport(hello).RoundTrip(req)
	if err != nil {
		panic(err)
	}
	defer res.Body.Close()
	os.Stdout.ReadFrom(res.Body)
}

[Scylla2020]

True okay

[Kieso]

Hi @Scylla2020, how did you solve?

[Scylla2020]

@Kies8 i didnt solve it. It worked for some sites but not all