bowenpay / wechat-spider

微信公众号爬虫

Home Page:http://wechatspider.0fenbei.com/

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

wechatspider_processor.stderr.log 日志打印的错误

vipwpcom opened this issue · comments

commented

启动程序后,添加微信公众号,有些公众号没有爬取到,我看了下日志,失败的情况都一致,其中有如下片段:
INFO 2016-10-27 16:40:50,715 processor 32482 140638797100800 {"publish_time": "2016-10-27 16:40:50.702217", "title": "分享一个特大写的尴尬!", "url": "http://mp.weixin.qq.com/s?timestamp=1477557622&src=3&ver=1&signature=qpCRLCGlM315db1DuiJml0rC3iNHPFuoBkzU9ixxfUyDLxOrV-2-yFbiczQUgLHDtqGPPyaTj2nKF*e3xvxepzroE2qcuzf6M3GS8Ue7W92aEGPnv6B1YzFV2AP52FuyeMVvoQUGT0tmtSURc7poZMc87sJOowfGJHX5cHJDXMI=", "origin_title": "分享一个特大写的尴尬![省略号]............................", "words": 14, "wechat_id": 2, "read_num": ["100000+"]}

Traceback (most recent call last):
File "/var/wdcp1/wechat-spider/bin/processor.py", line 57, in
processor.run()
File "/var/wdcp1/wechat-spider/bin/processor.py", line 52, in run
self.process(data)
File "/var/wdcp1/wechat-spider/bin/processor.py", line 37, in process
backend.process(data)
File "/var/wdcp1/wechat-spider/wechat/processors.py", line 48, in process
C.objects.update_or_create(uniqueid=params['uniqueid'], defaults=params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(_args, _kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 422, in update_or_create
obj, created = self._create_object_from_params(lookup, params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 439, in _create_object_from_params
obj = self.create(
_params)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 348, in create
obj.save(force_insert=True, using=self.db)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 710, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 738, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 822, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/base.py", line 861, in _do_insert
using=using, raw=raw)
File "/usr/local/lib/python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(_args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/query.py", line 920, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 970, in execute_sql
for sql, params in self.as_sql():
File "/usr/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 928, in as_sql
for obj in self.query.objs
File "/usr/local/lib/python2.7/site-packages/django/db/models/fields/init.py", line 710, in get_db_prep_save
prepared=False)
File "/usr/local/lib/python2.7/site-packages/django/db/models/fields/init.py", line 702, in get_db_prep_value
value = self.get_prep_value(value)
File "/usr/local/lib/python2.7/site-packages/django/db/models/fields/init.py", line 1868, in get_prep_value
return int(value)
TypeError: int() argument must be a string or a number, not 'list'

求解答!

"read_num": ["100000+"]
是我处理10万+文章阅读数事处理错了。 已修改代码,你重新pull一下