Can we support UOS with puppeteer?

Question

Can we support UOS with puppeteer?

huan opened this issue 3 years ago · comments

According to this Pull Request to ItChat created by @luvletter2333, it seems we can improve our puppeteer version as well.

UOS

I believe the most important is the headers.

UOS_PATCH_CLIENT_VERSION = '2.0.0'
UOS_PATCH_EXTSPAM = 'Gp8ICJkIEpkICggwMDAwMDAwMRAGGoAI1GiJSIpeO1RZTq9QBKsRbPJdi84ropi16EYI10WB6g74sGmRwSNXjPQnYUKYotKkvLGpshucCaeWZMOylnc6o2AgDX9grhQQx7fm2DJRTyuNhUlwmEoWhjoG3F0ySAWUsEbH3bJMsEBwoB//0qmFJob74ffdaslqL+IrSy7LJ76/G5TkvNC+J0VQkpH1u3iJJs0uUYyLDzdBIQ6Ogd8LDQ3VKnJLm4g/uDLe+G7zzzkOPzCjXL+70naaQ9medzqmh+/SmaQ6uFWLDQLcRln++wBwoEibNpG4uOJvqXy+ql50DjlNchSuqLmeadFoo9/mDT0q3G7o/80P15ostktjb7h9bfNc+nZVSnUEJXbCjTeqS5UYuxn+HTS5nZsPVxJA2O5GdKCYK4x8lTTKShRstqPfbQpplfllx2fwXcSljuYi3YipPyS3GCAqf5A7aYYwJ7AvGqUiR2SsVQ9Nbp8MGHET1GxhifC692APj6SJxZD3i1drSYZPMMsS9rKAJTGz2FEupohtpf2tgXm6c16nDk/cw+C7K7me5j5PLHv55DFCS84b06AytZPdkFZLj7FHOkcFGJXitHkX5cgww7vuf6F3p0yM/W73SoXTx6GX4G6Hg2rYx3O/9VU2Uq8lvURB4qIbD9XQpzmyiFMaytMnqxcZJcoXCtfkTJ6pI7a92JpRUvdSitg967VUDUAQnCXCM/m0snRkR9LtoXAO1FUGpwlp1EfIdCZFPKNnXMeqev0j9W9ZrkEs9ZWcUEexSj5z+dKYQBhIICviYUQHVqBTZSNy22PlUIeDeIs11j7q4t8rD8LPvzAKWVqXE+5lS1JPZkjg4y5hfX1Dod3t96clFfwsvDP6xBSe1NBcoKbkyGxYK0UvPGtKQEE0Se2zAymYDv41klYE9s+rxp8e94/H8XhrL9oGm8KWb2RmYnAE7ry9gd6e8ZuBRIsISlJAE/e8y8xFmP031S6Lnaet6YXPsFpuFsdQs535IjcFd75hh6DNMBYhSfjv456cvhsb99+fRw/KVZLC3yzNSCbLSyo9d9BI45Plma6V8akURQA/qsaAzU0VyTIqZJkPDTzhuCl92vD2AD/QOhx6iwRSVPAxcRFZcWjgc2wCKh+uCYkTVbNQpB9B90YlNmI3fWTuUOUjwOzQRxJZj11NsimjOJ50qQwTTFj6qQvQ1a/I+MkTx5UO+yNHl718JWcR3AXGmv/aa9rD1eNP8ioTGlOZwPgmr2sor2iBpKTOrB83QgZXP+xRYkb4zVC+LoAXEoIa1+zArywlgREer7DLePukkU6wHTkuSaF+ge5Of1bXuU4i938WJHj0t3D8uQxkJvoFi/EYN/7u2P1zGRLV4dHVUsZMGCCtnO6BBigFMAA='

headers = {
  'client-version' : UOS_PATCH_CLIENT_VERSION,
  'extspam' : UOS_PATCH_EXTSPAM,
  'referer' : 'https://wx.qq.com/?&lang=zh_CN&target=t'
}

To be investigated.

Huan Li · Answer 1 · Sun Mar 21 2021 22:50:14 GMT+0800 (China Standard Time)

It seems not to work on my account (can not log in via web) with the following modifications:

https://github.com/wechaty/wechaty-puppet-wechat/blob/04fae37abc714b21b19ad66eca827971d3a63a35/src/bridge.ts#L228-L242

@luvletter2333 Please feel free to join us if you are interested. :-)

Luv Letter · Answer 2 · Sun Mar 21 2021 23:00:31 GMT+0800 (China Standard Time)

I am willing to help but I have no idea about typescript and its HTTP library...

Luv Letter · Answer 3 · Sun Mar 21 2021 23:01:35 GMT+0800 (China Standard Time)

This patch is still working, I checked it just before

Luv Letter · Answer 4 · Sun Mar 21 2021 23:24:51 GMT+0800 (China Standard Time)

This patch should be applied to the third step of login.

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

When requesting cgi-bin/mmwebwx-bin/webwxnewloginpage, this patch is needed. Besides, wxsid and wxuin should be obtained from Cookie, not XML response.

Huan Li · Answer 5 · Sun Mar 21 2021 23:48:11 GMT+0800 (China Standard Time)

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

Luv Letter · Answer 6 · Fri Mar 26 2021 14:57:07 GMT+0800 (China Standard Time)

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

I'm sorry to reply so late.
You would need to switch to Firefox for inspection since Chrome Devtools may lose Response Body.

You could follow this repo to try adding custom Headers. Then you may login Wechat Web like UOS.
The pre-generated token is published in issues by me.

Luv Letter · Answer 7 · Fri Mar 26 2021 15:00:54 GMT+0800 (China Standard Time)

About the analysis of the protocol of Wechat Web, you could check this blog https://blog.csdn.net/zhangmiaoping23/article/details/78733231

Huan Li · Answer 8 · Fri Mar 26 2021 15:07:21 GMT+0800 (China Standard Time)

Thank you so much for the detailed explanation!

I'll try Firefox to reproduce it later, and I believe it will not be hard to enable it in Puppeteer because we can study from the Firefox ModHeader plugin.

xia · Answer 9 · Fri Apr 09 2021 18:01:13 GMT+0800 (China Standard Time)

不知道是不是 redirect_uri 的关系？使用 988dd56 依然不能绕开登录限制

Huan Li · Answer 10 · Fri Apr 09 2021 18:09:05 GMT+0800 (China Standard Time)

@artxia If you want to help, I'd like to suggest that you follow this repo https://github.com/BlueSky-07/wechat-token and make everything work with the browser plugin as the repo mentioned.

After you can reproduce the login successfully with the browser plugin, then I believe we will be easily to copy the setting from the browser plugin to the puppeteer configurations.

xia · Answer 11 · Fri Apr 09 2021 18:13:40 GMT+0800 (China Standard Time)

@artxia If you want to help, I'd like to suggest that you follow this repo https://github.com/BlueSky-07/wechat-token and make everything work with the browser plugin as the repo mentioned.

After you can reproduce the login successfully with the browser plugin, then I believe we will be easily to copy the setting from the browser plugin to the puppeteer configurations.

https://github.com/adamyi/wechrome
This browser plugin can be used directly.

Huan Li · Answer 12 · Fri Apr 09 2021 18:16:29 GMT+0800 (China Standard Time)

Awesome, it looks great!

Would you like to send a PR to configure our puppeteer by following the setting from this plugin?

This improvement will be an epic milestone for our Web Protocol since 2017!

xia · Answer 13 · Fri Apr 09 2021 18:21:55 GMT+0800 (China Standard Time)

I am very sorry, my code programming skills are not enough for PR.

Also, the above browser plugin may require adding an address "https://wx2.qq.com/*" to "wechatUrls".

Leo_chen · Answer 14 · Fri Apr 09 2021 19:21:04 GMT+0800 (China Standard Time)

This is an exciting news

Huan Li · Answer 15 · Mon Apr 12 2021 02:41:04 GMT+0800 (China Standard Time)

According to puppeteer/puppeteer#659 and Stackoverflow: No, google-chrome in headless mode doesn't support extensions yet., we can not use https://github.com/adamyi/wechrome directly with the puppeteer.

Need to move the logic to native puppeteer implementation.

Leo_chen · Answer 16 · Tue Apr 13 2021 12:42:33 GMT+0800 (China Standard Time)

Hi @luvletter2333 !

Thank you very much for your reply, and it's great to know that the patch is still working!

When you get a 200 from https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login, the response must contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage.

Cloud you explain how to get a response contain window.redirect_uri="https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage" when visit https://login.weixin.qq.com/cgi-bin/mmwebwx-bin/login?

I tried in the chrome browser and it seems that the webwxnewloginpage does not exist.

Could you please share a minimum code to reproduce it in Python code, or CURL if possible? Appreciate it!

Thank you so much for the detailed explanation!

I'll try Firefox to reproduce it later, and I believe it will not be hard to enable it in Puppeteer because we can study from the Firefox ModHeader plugin.

我本地测试了一个最简化的代码，发现https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage是扫码点击确定后的跳转地址，所以不扫码确定看不到这个path。以下是我使用puppeteer写的最小demo，本地测试已经可以成功登陆网页端，希望能有帮助

const pptr = require('puppeteer');

async function bootstrap() {
    const browser = await pptr.launch({
        headless: false,
        slowMo: 250,
    });
    const page = await browser.newPage();
    // 开启请求拦截
    await page.setRequestInterception(true);
    page.on('request', async req => {
        let url = new URL(req.url());
        if (url.pathname == "/cgi-bin/mmwebwx-bin/webwxnewloginpage") {
            console.log(req.headers())
            const override = {
                headers: {
                    ...req.headers(),
                    extspam: "Gp8ICJkIEpkICggwMDAwMDAwMRAGGoAI1GiJSIpeO1RZTq9QBKsRbPJdi84ropi16EYI10WB6g74sGmRwSNXjPQnYUKYotKkvLGpshucCaeWZMOylnc6o2AgDX9grhQQx7fm2DJRTyuNhUlwmEoWhjoG3F0ySAWUsEbH3bJMsEBwoB//0qmFJob74ffdaslqL+IrSy7LJ76/G5TkvNC+J0VQkpH1u3iJJs0uUYyLDzdBIQ6Ogd8LDQ3VKnJLm4g/uDLe+G7zzzkOPzCjXL+70naaQ9medzqmh+/SmaQ6uFWLDQLcRln++wBwoEibNpG4uOJvqXy+ql50DjlNchSuqLmeadFoo9/mDT0q3G7o/80P15ostktjb7h9bfNc+nZVSnUEJXbCjTeqS5UYuxn+HTS5nZsPVxJA2O5GdKCYK4x8lTTKShRstqPfbQpplfllx2fwXcSljuYi3YipPyS3GCAqf5A7aYYwJ7AvGqUiR2SsVQ9Nbp8MGHET1GxhifC692APj6SJxZD3i1drSYZPMMsS9rKAJTGz2FEupohtpf2tgXm6c16nDk/cw+C7K7me5j5PLHv55DFCS84b06AytZPdkFZLj7FHOkcFGJXitHkX5cgww7vuf6F3p0yM/W73SoXTx6GX4G6Hg2rYx3O/9VU2Uq8lvURB4qIbD9XQpzmyiFMaytMnqxcZJcoXCtfkTJ6pI7a92JpRUvdSitg967VUDUAQnCXCM/m0snRkR9LtoXAO1FUGpwlp1EfIdCZFPKNnXMeqev0j9W9ZrkEs9ZWcUEexSj5z+dKYQBhIICviYUQHVqBTZSNy22PlUIeDeIs11j7q4t8rD8LPvzAKWVqXE+5lS1JPZkjg4y5hfX1Dod3t96clFfwsvDP6xBSe1NBcoKbkyGxYK0UvPGtKQEE0Se2zAymYDv41klYE9s+rxp8e94/H8XhrL9oGm8KWb2RmYnAE7ry9gd6e8ZuBRIsISlJAE/e8y8xFmP031S6Lnaet6YXPsFpuFsdQs535IjcFd75hh6DNMBYhSfjv456cvhsb99+fRw/KVZLC3yzNSCbLSyo9d9BI45Plma6V8akURQA/qsaAzU0VyTIqZJkPDTzhuCl92vD2AD/QOhx6iwRSVPAxcRFZcWjgc2wCKh+uCYkTVbNQpB9B90YlNmI3fWTuUOUjwOzQRxJZj11NsimjOJ50qQwTTFj6qQvQ1a/I+MkTx5UO+yNHl718JWcR3AXGmv/aa9rD1eNP8ioTGlOZwPgmr2sor2iBpKTOrB83QgZXP+xRYkb4zVC+LoAXEoIa1+zArywlgREer7DLePukkU6wHTkuSaF+ge5Of1bXuU4i938WJHj0t3D8uQxkJvoFi/EYN/7u2P1zGRLV4dHVUsZMGCCtnO6BBigFMAA=",
                    'client-version': "2.0.0",
                }
            }
            await req.continue(override);
            return
        }
        await req.continue();

    });

    page.on('response', async res => {
        if (res.url().indexOf('/header') >= 0) {
            console.log(res.status());
            console.log(await res.text());
        }
    });

    page.on('requestfinished', req => {
        // console.log(`请求完成: ${req.url()}`);
    });

    page.on('requestfailed', req => {
        console.log(`请求失败: ${req.url()}`);
    });
    await page.goto('https://wx.qq.com/?target=t');
}

bootstrap();

Luv Letter · Answer 17 · Tue Apr 13 2021 12:46:00 GMT+0800 (China Standard Time)

我本地测试了一个最简化的代码，发现https://wx.qq.com/cgi-bin/mmwebwx-bin/webwxnewloginpage是扫码点击确定后的跳转地址，所以不扫码确定看不到这个path。以下是我使用puppeteer写的最小demo，本地测试已经可以成功登陆网页端，希望能有帮助

是这样的（

Huan Li · Answer 18 · Tue Apr 13 2021 13:03:50 GMT+0800 (China Standard Time)

@gengchen528 Thank you very much for your feedback and minimum reproducible code, it will be a great help and I'll definitely try it today and hope we can integrate it into our puppet soon!

Huan Li · Answer 19 · Tue Apr 13 2021 16:01:09 GMT+0800 (China Standard Time)

Thanks to @gengchen528 for adding UOS support for our Wechaty Puppet WeChat!

The PR has been tested by myself and it works like a charm, I have merged it as wechaty-puppet-wechat@0.27.1 and we can expect for using it soon!

This is a HUGE improvement for our Wechaty ecosystem because this fix lets most of the users can get started with Wechaty more easily. I'd like to invite @gengchen528 to write a blog post for announcing this great improvement, it would be great if we have it soon and I'm looking forward to reading it!

Leo_chen · Answer 20 · Tue Apr 13 2021 16:17:10 GMT+0800 (China Standard Time)

Thanks to @gengchen528 for adding UOS support for our Wechaty Puppet WeChat!

The PR has been tested by myself and it works like a charm, I have merged it as wechaty-puppet-wechat@0.27.1 and we can expect for using it soon!

This is a HUGE improvement for our Wechaty ecosystem because this fix lets most of the users can get started with Wechaty more easily. I'd like to invite @gengchen528 to write a blog post for announcing this great improvement, it would be great if we have it soon and I'm looking forward to reading it!

非常荣幸能够写这篇博客，感谢 @artxia 提供的chrom插件给了我思路。希望这种方式能够一直可用

Huan Li · Answer 21 · Tue Apr 13 2021 21:59:06 GMT+0800 (China Standard Time)

Close this issue because it has been implemented perfectly.

Thanks to @luvletter2333 for letting me know UOS by your Python PR, @artxia for linking to the chrome extension to make everything clear, @gengchen528 for creating the PR for our puppet system!

suntong · Answer 22 · Sat May 08 2021 22:35:34 GMT+0800 (China Standard Time)

Thanks you guys to have made such tremendous breakthrough. Truly a milestone in WX bot history. @gengchen528 your blog post is spreading like a wildfire in WX bot community, 😄😄

One question,

You could follow this repo to try adding custom Headers. Then you may login Wechat Web like UOS.
The pre-generated token is published in issues by me.

Currently, the whole world is using a fixed extspam value from @gengchen528's blog post, however, wechat-token says that the extspam value should be associated with each person's profile. So

@luvletter2333 / @gengchen528, do you think it'd be a problem, if the whole world is using the same fixed extspam value? thx

Huan Li · Answer 23 · Sat May 08 2021 22:50:09 GMT+0800 (China Standard Time)

@luvletter2333 / @gengchen528, do you think it'd be a problem, if the whole world is using the same fixed extspam value? thx

Thanks for pointing this out! I'd also have this question and I believe it should not use the same extspam value for every users.

Leo_chen · Answer 24 · Sat May 08 2021 23:00:24 GMT+0800 (China Standard Time)

目前看来同一个extspam不会有太大问题，但是后期没法保证。如果如果把wechat-token集成进puppet或者让用户自行生成的话，会不会增加使用门槛，而且我看使用wechat-token是需要安装docker环境的

我在想是否能够由我们提前生成一批extspam，在启动的时候随机使用其中一个呢这只是我目前个人的建议

suntong · Answer 25 · Sun May 09 2021 00:36:10 GMT+0800 (China Standard Time)

是否能够由我们提前生成一批extspam，在启动的时候随机使用其中一个呢

I think that's the best solution so far, I was thinking that too.
Otherwise, it can easily be identified by WX, and becomes the biggest giveaway, yelling to WX, "block me, I'm such an easy target and so simple to block"

zpaimon · Answer 26 · Sun May 09 2021 23:13:15 GMT+0800 (China Standard Time)

I think we can build a token server.When user start puppet,the puppet call token api and get an extspam token.

Scomarlf · Answer 27 · Mon Dec 12 2022 21:51:20 GMT+0800 (China Standard Time)

我可以正常登录 UOS 微信，但是无法通过 wechaty 登陆。

扫码后手机微信提示“当前账号无法使用该服务”。

austincityuwen · Answer 28 · Fri Mar 24 2023 17:23:58 GMT+0800 (China Standard Time)

It's too late for me to read this issue at this moment, my question is how to generate personal extspam that associated with each person's profile? In case that it is detected by wechat and may block it someday？