Tencent / plato

腾讯高性能分布式图计算框架Plato

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

运行pagerank时报错

sonogoto opened this issue · comments

运行环境:Centos7(Docker ce) 6C12T 32G

数据集:LiveJournal数据集

运行参数

+++ dirname ./scripts/run_pagerank_local.sh
++ realpath ./scripts
+ CUR_DIR=/home/plato-master/scripts
++ realpath /home/plato-master/scripts/..
+ ROOT_DIR=/home/plato-master
+ cd /home/plato-master
+ MAIN=/home/plato-master/bazel-bin/example/pagerank
+ WNUM=4
+ WCORES=4
+ INPUT=/home/plato_test.csv
+ OUTPUT=/tmp/pagerank
+ IS_DIRECTED=false
+ EPS=0.00001
+ DAMPING=0.85
+ ITERATIONS=100
+ PARAMS+=' --threads 4'
+ PARAMS+=' --input /home/plato_test.csv --output /tmp/pagerank --is_directed=false'
+ PARAMS+=' --iterations 100 --eps 0.00001 --damping 0.85'
+ MPIRUN_CMD=/home/plato-master/3rd/mpich/bin/mpiexec.hydra
+ export LD_LIBRARY_PATH=:/home/plato-master/3rd/hadoop2/lib
+ LD_LIBRARY_PATH=:/home/plato-master/3rd/hadoop2/lib
+ mkdir -p /tmp/pagerank
+ /home/plato-master/3rd/mpich/bin/mpiexec.hydra -n 4 /home/plato-master/bazel-bin/example/pagerank --threads 4 --input /home/plato_test.csv --output /tmp/pagerank --is_directed=false --iterations 100 --eps 0.00001 --damping 0.85

错误日志

I1116 08:27:54.050612 23529 base.hpp:179] thread support level provided by MPI:
I1116 08:27:54.050709 23529 base.hpp:182] MPI_THREAD_MULTIPLE
I1116 08:27:54.050719 23529 base.hpp:211] threads: 4
I1116 08:27:54.050724 23529 base.hpp:212] sockets: 1
I1116 08:27:54.050727 23529 base.hpp:213] partitions: 4
I1116 08:27:54.168637 23529 detail.hpp:106] partition-0, file-count: 1
I1116 08:27:54.168712 23529 detail.hpp:106] partition-1, file-count: 0
I1116 08:27:54.168720 23529 detail.hpp:106] partition-2, file-count: 0
I1116 08:27:54.168725 23529 detail.hpp:106] partition-3, file-count: 0
I1116 08:28:01.654152 23529 structure.hpp:496] edges:        34681189
I1116 08:28:01.654191 23529 structure.hpp:497] vertices:     3997962
I1116 08:28:01.654196 23529 structure.hpp:498] max_v_id:     4036537
I1116 08:28:01.654201 23529 structure.hpp:499] is_directed_: 0
I1116 08:28:01.654209 23529 structure.hpp:501] load edges cache cost: 7.604s
I1116 08:28:01.981242 23529 structure.hpp:515] generate degrees cost: 0.327s
I1116 08:28:01.981333 23529 sequence.hpp:51] total_amount: 165313466, alpha: 24
I1116 08:28:01.982026 23529 sequence.hpp:71] partition-0: [0,524288), amount: 41328433
I1116 08:28:01.982990 23529 sequence.hpp:71] partition-1: [524288,1380352), amount: 41328365
I1116 08:28:01.984127 23529 sequence.hpp:71] partition-2: [1380352,2543616), amount: 41328358
I1116 08:28:01.985311 23529 sequence.hpp:71] partition-3: [2543616,3997962), amount: 41314099
I1116 08:28:01.986980 23529 structure.hpp:528] partition cost: 0.005s
F1116 08:28:01.999750 23553 dcsc.hpp:362] Check failed: edge->dst_ < graph_info.vertices_
*** Check failure stack trace: ***
F1116 08:28:01.999800 23529 dcsc.hpp:362] Check failed: edge->dst_ < graph_info.vertices_ F1116 08:28:01.999929 23558 dcsc.hpp:362] Check failed: edge->dst_ < graph_info.vertices_
*** Check failure stack trace: ***
F1116 08:28:01.999800 23529 dcsc.hpp:362] Check failed: edge->dst_ < graph_info.vertices_ F1116 08:28:01.999929 23558 dcsc.hpp:362] Check failed: edge->dst_ < graph_info.vertices_
*** Check failure stack trace: ***
    @           0x465016  google::LogMessage::Fail()
Aborting.
Backtrace:
  0x0000000000460158
  0x000000000045f2d7
  0x000000000045f479
  /lib64/libpthread.so.0+0x000000000000f62e
  /lib64/libc.so.6+0x0000000000036386
  /lib64/libc.so.6+0x0000000000037a76
  0x000000000046d076
  0x0000000000465014
  0x0000000000464f70
  0x0000000000464956
  0x0000000000467cd3
  0x0000000000423548
  0x000000000042363d
  0x000000000042377a
  0x0000000000423878
  0x000000000042377a
  0x000000000040e0d4
  0x00000000004281a1
  0x000000000042ead3
  0x000000000042f7f5
  0x000000000043e525
  0x0000000000408146
  /lib64/libc.so.6+0x0000000000022553
  0x000000000040b115

===================================================================================
=   BAD TERMINATION OF ONE OF YOUR APPLICATION PROCESSES
=   PID 23529 RUNNING AT bd3c4fe151fd
=   EXIT CODE: 134
=   CLEANING UP REMAINING PROCESSES
=   YOU CAN IGNORE THE BELOW CLEANUP MESSAGES
===================================================================================
YOUR APPLICATION TERMINATED WITH THE EXIT STRING: Aborted (signal 6)
This typically refers to a problem with your application.
Please see the FAQ page for debugging suggestions

pagerank算法需要对id从0开始连续编码。

pagerank算法需要对id从0开始连续编码。

解决了,tks。

ps

我对比测试了plato与madlib,一个基于Greenplum的计算框架,plato在连通分量算法上优势明显(>10x),然而,在pagerank算法上优势并不明显,甚至不及madlib。 I'm wondering why~

要对齐算法和收敛条件等参数,有可能算法实现不一样,或者迭代轮数不一样

长时间没有回复,关闭本issue,如有问题,请reopen。