抓取自己的所有信息失败
leon0707 opened this issue · comments
Describe the bug
按照readme 安装 运行 抓取自己的所有信息失败
To Reproduce
Steps to reproduce the behavior:
python manage.py fetch -e email@tom.com -p xxxx -s -g -a -bExpected behavior
抓取自己的所有信息
Error Output:
- File: [renrenBackup/crawl/utils.py"]
- Console Output:
check login, and get homepage for cookie
need login
prepare login encryt info
prepare post login request
login success with email@tom.com as 1234456644
check login, and get homepage for cookie
login valid
login valid
Traceback (most recent call last):
File "manage.py", line 170, in <module>
cli()
File "/Users/lfeng/Dev/projects/renrenBackup/venv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/Users/lfeng/Dev/projects/renrenBackup/venv/lib/python3.8/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/Users/lfeng/Dev/projects/renrenBackup/venv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/lfeng/Dev/projects/renrenBackup/venv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/lfeng/Dev/projects/renrenBackup/venv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "manage.py", line 57, in fetch
fetched = fetch_user(
File "/Users/lfeng/Dev/projects/renrenBackup/fetch.py", line 88, in fetch_user
get_user(uid)
File "/Users/lfeng/Dev/projects/renrenBackup/crawl/utils.py", line 108, in get_user
name = re.findall(
IndexError: list index out of rangeAdditional context
mac m1 os 14
python 3.8.5
有点可惜 好多事记不清还想回忆下的
<!DOCTYPE html>
<html lang="zh-CN">
<head>
<title>xxx的个人主页</title>
<meta name="keywords" content="Xiaonei,Renren,校内,大学,同学,同事,白领,个人主页,博客,相册,群组,社区,交友,聊天,音乐,视频,校园,人人,人人网">
<meta name="description" content="人人是倡导实名制的社交网络平台，用户可以在平台上互相交流、分享信息和用户自创的内容，是一个和谐共处的网络社区。以内容为 >
<meta charset="utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width,initial-scale=1,minimum-scale=1,maximum-scale=1,user-scalable=no" />
<meta name="applicable-device" content="mobile" />
<link rel="icon" href="/static/favicon.ico" />
<script>
// 移动端访问renren.com 跳转u.renren.com
// 并且排除用户协议 http://renren.com/protocol.html
// 隐私协议: http://renren.com/privagreement.html
// 反馈详情: http://renren.com/feedBackDetail/
var isMobile = function () {
return navigator.userAgent.match(
/(phone|pad|pod|iPhone|iPod|ios|iPad|Android|Mobile|BlackBerry|IEMobile|MQQBrowser|JUC|Fennec|wOSBrowser|BrowserNG|WebOS|Symbian|Windows Phone)/i,
);
}
if (isMobile() && window.location.href.indexOf('protocol.html') === -1 && window.location.href.indexOf('privagreement.html')===-1 && window.location.href.indexOf{
//window.location.href = '//u.renren.com';
}
window.onresize = function () {
if (isMobile() && window.location.href.indexOf('protocol.html') === -1 && window.location.href.indexOf('privagreement.html')===-1 && window.location.href.index{
//window.location.href = '//u.renren.com';
}
}
</script>
<link rel="preload" href="/static/css/chunk-vendors.18276495d80a4c66e9828acddda19adba9620d73d7ef0a767982f8f14bacc5ba62d04d9838d2a631acd105da45877ae528207598e0958d845>
<body data-site="renren-web">
<noscript>
<strong>We're sorry but test doesn't work properly without JavaScript enabled. Please
enable it to continue.</strong>
</noscript>
<div id="app" data-server-rendered="true" class="rr-app"><div class="rr-login" style="display:none;"><div class="rr-login_container"><div class="
rr-login_box"><div class="rr-login_head"><div><img src="/static/img/login_logo.ee85f45e.png" alt=""></div><div class="rr-login_slogan">记录·精彩·校园</di >
</body>
<script>
(function () {
// 没有被嵌入到iframe中
if (top !== self) {
var script = document.createElement('script');
script.src = 'https://xyoss.g.com.cn/xy-production/static/strategy.js';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(script, s);
}
})();
</script>
<!-- BI统计代码 -->
<script src="//bdtj.infinities.com.cn/bi-sdk.1.2.1.js"></script>
<script>
/*gray-style-content-start*/
// (function () {
// const currentDate = new Date().getTime()
// const startDate = new Date(2020, 3, 3, 23, 50, 0).getTime()
// const endDate = new Date(2020, 3, 5, 0, 10, 0).getTime()
// if (currentDate >= startDate && endDate >= currentDate) {
// var body = document.body;
// body.style['filter'] = 'progid:DXImageTransform.Microsoft.BasicImage(grayscale=1)';
// if (!body.style['filter'] || body.style['filter'] === 'none') {
// body.style['-webkit-filter'] = 'grayscale(100%)';
// body.style['-moz-filter'] = 'grayscale(100%)';
// body.style['-ms-filter'] = 'grayscale(100%)';
// body.style['-o-filter'] = 'grayscale(100%)';
// body.style['filter'] = 'grayscale(100%)';
// }
// }
// }());
/*gray-style-content-end*/
(function () {
window.BI_SDK.report('webrenrenwang', 'log');
})();
</script>
<!-- 百度统计代码 -->
<script type="text/javascript">
var isChannel = function (url) {
return window.location.href.indexOf(url) > 0
}
var _hmt = _hmt || [];
(function () {
var hm = document.createElement('script');
hm.src = 'https://hm.baidu.com/hm.js?ad6b0fd84f08dc70750c5ee6ba650172';
var s = document.getElementsByTagName('script')[0];
s.parentNode.insertBefore(hm, s);
if (isChannel('/list/iphone')) {
hm = document.createElement('script')
hm.src = 'https://hm.baidu.com/hm.js?e9510b11f4be96624e87e58aa9021619'
s = document.getElementsByTagName('script')[0]
s.parentNode.insertBefore(hm, s)
}
if (isChannel('/list/android')) {
hm = document.createElement('script')
hm.src = 'https://hm.baidu.com/hm.js?805dc8a07eff052a35614357c2853559'
s = document.getElementsByTagName('script')[0]
s.parentNode.insertBefore(hm, s)
}
if (isChannel('/list/tech')) {
hm = document.createElement('script')
hm.src = 'https://hm.baidu.com/hm.js?ec876d8b20beb8c5254a8b816dc29e85'
s = document.getElementsByTagName('script')[0]
s.parentNode.insertBefore(hm, s)
}
})();
</script>
</html>只剩自己的名字在title里 网页版彻底废了
是的，如你所说，人人网页版彻底废了，那这个工具也没法工作了 :(