run YSB benchmark with 32 threads
chenzongxiong opened this issue · comments
zxchen commented
Hi,
When I try to run YSB with 32 threads, there is no throughput on console.
./run.sh uk.ac.imperial.lsds.saber.experiments.benchmarks.yahoo.YahooBenchmarkApp 32
From the log, I think it's due to the buffer not released. However, I didn't find out the root cause.
One more thing I need to mention is that, this APP run well with 1 to 8 threads.
George Theodorakis commented
Hi,
Unfortunately, I am away in an internship and I don't have much time to
look at the code right now.
The first thing that you have to keep in mind is that SABER is not
NUMA-aware (we are working on it in another project). So when you cross a
socket the performance will drop dramatically, especially in queries such
as YahooBenchmark where you spend most of your time on transferring of data
between operators (it's not computational intensive).
Still, this doesn't reply to your question. The problem lies in the merge
phase. In the current version, there is a race condition when accessing the
atomics in the ResultCollector/ResultHandler and you are pipelining
operators (as happens here). The proper solution would be to create
individual merge tasks (this is not how it works now) and add them in the
GlobalTaskQueue, to ensure that they are going to be executed. This is
something I am working on and I haven't finished yet.
I am sorry I didn't provide a solution for now, but I can keep you updated
with the progress of our new project if you are interested.
Cheers,
George
Στις Πέμ, 12 Σεπ 2019 στις 12:41 μ.μ., ο/η zxchen <notifications@github.com>
έγραψε:
… Hi,
When I try to run YSB with 32 threads, there is no throughput on console.
./run.sh
uk.ac.imperial.lsds.saber.experiments.benchmarks.yahoo.YahooBenchmarkApp 32
[image: image]
<https://user-images.githubusercontent.com/8354165/64777109-caa30e00-d559-11e9-83cd-908eb805aff9.png>
[image: image]
<https://user-images.githubusercontent.com/8354165/64777389-59178f80-d55a-11e9-9f5b-7517e1b4191c.png>
From the log, I think it's due to the buffer not released. However, I
didn't find out the root cause.
One more thing I need to mention is that, this APP run well with 1 to 8
threads.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#3?email_source=notifications&email_token=ADTBN2HJGKMVC2WJHKEQDPDQJIMEXA5CNFSM4IWCFTT2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HK6NCXA>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/ADTBN2ECNTVKYM5TE3EOPCTQJIMEXANCNFSM4IWCFTTQ>
.