konigsberg / tianhe_network

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

TIANHE-2 interconnect network is composed of compute nodes and routers.
We implement an electrical-phonic interconnect network.

Features:
(1) Clock integer counter is used to implement the global forward table.
(2) Every upper-side router and lower-side router has local cache of shared
credit memory. Copy-on-write is used to update local credit cache. When a local
credit cache is modified, it send an update exchange message to shared memory,
and share memory broadcast the new credit to all local cache.

Limitations:
(1) To simplify forward table viewed by side routers and optical switch, we set
the byte length of flit to zero.
(2) Credit message is not embedded into flit.

Bug fixed:
(1) Sending credit is skipped when input buffer is empty.
(2) When setting the port in credit message, the calculated output port is used
instead of the actually switched port.
(3) The credit array index is between P/2 and P for upper-side routers.
(4) The clock calculation based on division of double values has precision
error, making some flit blocked even if it's right time to forward.
(5) The router latency is in second unit while the link latency is in nano
second unit.

Bug unfixed:
(1) Lower-side routers have leak memory of credit object, although the credit
messages are deleted in router's destructor.
(2) Low-throughput, possibly high-latency of long trip flits.
(3) If credit message is not embedded into flit, it is possible that the optical
switch could not forward a credit message because all of its ports are busy. So
the credit message maybe dropped, causing serious traffic issue.
(4) The window of forwarding packet can be minimized to packet size.
(5) Downward forwarding can possibly be optimized.
(6) Frequency should be reset to 437.5MHz.

Simulation results:
injection rate = 1, warm up time = 150ns, stop time = 400ns, uniform traffic:
total_flit_gen = 4700160
total_flit_sent = 4626244
total_flit_recv = 466137
throughput = 9.917470894607844 %
average latency = 640.0395546667785 ns

injection rate = 0.2, warm up time = 150ns, stop time = 400ns, uniform traffic:
total_flit_gen = 1013760
total_flit_sent = 940032
total_flit_recv = 503317
throughput = 49.64853614267677 %
average latency = 687.1205561259072 ns

injection rate = 0.8, warm up time = 400ns, stop time = 500ns, uniform traffic:
total_flit_gen = 1566720
total_flit_sent = 1492973
total_flit_recv = 3142
throughput = 0.20054636437908496 %
average latency = 629.5294234226799 ns

The theoretical latency under uniform injection is
12/18432*100+20/18432*300+32*11/18432*500+11*32*12/18432*800+36*32*12/18432*1000
= 943ns

injection rate = 1, warm up time = 500ns, stop time = 1000ns, uniform traffic:
total_flit_gen = 4055040
total_flit_sent = 4035307
total_flit_recv = 234670
throughput = 99.51337101483585 %
average latency = 1387.679252804978 ns

under injection rate 0.1:
total_flit_gen = 368640
total_flit_sent = 368640
total_flit_recv = 326970
throughput = 100.0 %
average latency = 1339.789096332311 ns
average hopcount = 8.343623114892498

under injection rate 0.3:
total_flit_gen = 1198080
total_flit_sent = 1234944
total_flit_recv = 371116
throughput = 103.07692307692307 %
average latency = 1339.6317548447546 ns
average hopcount = 8.187755678632362

under injection rate 0.5:
total_flit_gen = 2027520
total_flit_sent = 2027520
total_flit_recv = 292636
throughput = 100.0 %
average latency = 1352.5427959258725 ns
average hopcount = 8.098283643024251

under injection rate 0.7:
total_flit_gen = 2856960
total_flit_sent = 2820096
total_flit_recv = 254887
throughput = 98.70967741935483 %
average latency = 1363.050423725556 ns
average hopcount = 8.053255157296771

under injection rate 0.9:
total_flit_gen = 3594240
total_flit_sent = 3631097
total_flit_recv = 240047
throughput = 101.0254462695869 %
average latency = 1376.7932981786926 ns
average hopcount = 8.03796610751557

no flow control
total_flit_gen = 8110080
total_flit_sent = 8073216
total_flit_recv = 1946258
throughput = 99.54545454545455 %
average latency = 1589.6437687827101 ns


no-flow control injection simulation:
under injection rate 0.2:

total_flit_gen = 1658880
total_flit_sent = 1658880
total_flit_recv = 1190659
throughput = 100.0 %
average latency = 1401.166265698821 ns
average hopcount = 8.284091312997605

under injection rate 0.4:

total_flit_gen = 3225600
total_flit_sent = 3225600
total_flit_recv = 1784738
throughput = 100.0 %
average latency = 1475.7285723846496 ns
average hopcount = 8.111347889316967

under injection rate 0.6:

total_flit_gen = 4792320
total_flit_sent = 4847616
total_flit_recv = 1923971
throughput = 101.15384615384615 %
average latency = 1525.5139614903321 ns
average hopcount = 8.025200665819108

under injection rate 0.8:

total_flit_gen = 6451200
total_flit_sent = 6451200
total_flit_recv = 1944935
throughput = 100.0 %
average latency = 1552.968588931679 ns
average hopcount = 8.009616624181248

under injection rate 1.0:

total_flit_gen = 8110080
total_flit_sent = 8073216
total_flit_recv = 1946258
throughput = 99.54545454545455 %
average latency = 1589.6437687827101 ns
average hopcount = 8.007190946303881

under injection rate 0.1:

total_flit_gen = 829440
total_flit_sent = 829440
total_flit_recv = 684527
throughput = 100.0 %
average latency = 1364.4736888226155 ns
average hopcount = 8.360802030144958

under injection rate 0.3:

total_flit_gen = 2396160
total_flit_sent = 2451456
total_flit_recv = 1561240
throughput = 102.30769230769229 %
average latency = 1441.509795906724 ns
average hopcount = 8.202691577732223

under injection rate 0.5:

total_flit_gen = 4055040
total_flit_sent = 4055040
total_flit_recv = 1886550
throughput = 100.0 %
average latency = 1506.1776294486017 ns
average hopcount = 8.053033788568731

under injection rate 0.7:

total_flit_gen = 5621760
total_flit_sent = 5640192
total_flit_recv = 1939377
throughput = 100.327868852459 %
average latency = 1540.799844218341 ns
average hopcount = 8.01936955153792

under injection rate 0.9:

total_flit_gen = 7280640
total_flit_sent = 7262208
total_flit_recv = 1948628
throughput = 99.74683544303798 %
average latency = 1569.1040303361046 ns
average hopcount = 8.009461186401436

hack flow control
under injection rate 0.2:

total_flit_gen = 829440
total_flit_sent = 829440
total_flit_recv = 776860
throughput = 100.0 %
average latency = 1181.4200832866286 ns
average hopcount = 8.38840394384246

under injection rate 0.4:

total_flit_gen = 1658880
total_flit_sent = 1566720
total_flit_recv = 1317379
throughput = 94.44444444444444 %
average latency = 1229.561365461096 ns
average hopcount = 8.329425034483497

under injection rate 0.6:

total_flit_gen = 2396160
total_flit_sent = 2433024
total_flit_recv = 1498394
throughput = 101.53846153846153 %
average latency = 1268.6684525890678 ns
average hopcount = 8.2930934036226

under injection rate 0.7:

total_flit_gen = 2856960
total_flit_sent = 2820096
total_flit_recv = 1525611
throughput = 98.70967741935483 %
average latency = 1281.41146628717 ns
average hopcount = 8.290250784053946

under injection rate 0.8:

total_flit_gen = 3225600
total_flit_sent = 3225600
total_flit_recv = 1537657
throughput = 100.0 %
average latency = 1293.4294204033242 ns
average hopcount = 8.292702280913904

under injection rate 0.9:

total_flit_gen = 3594240
total_flit_sent = 3631104
total_flit_recv = 1543433
throughput = 101.02564102564102 %
average latency = 1307.1876167505443 ns
average hopcount = 8.293634843333159

under injection rate 1.0:

total_flit_gen = 4055040
total_flit_sent = 4036577
total_flit_recv = 1548272
throughput = 99.5446900647096 %
average latency = 1323.87576751024 ns
average hopcount = 8.293480465198593

About


Languages

Language:C++ 94.7%Language:Makefile 2.5%Language:Python 2.4%Language:C 0.2%Language:Shell 0.2%