As more hashers are created, the speed slows down
youngk1019 opened this issue · comments
Hello, I encountered the same problem as this issue #33 (comment) in the production environment. I will create a large number of Hasher instances in the production environment, but there are not many existing Hasher instances at the same time. There are a large number of scenes of creation and destruction, and it is guaranteed that they will be successfully closed eventually, but my speed will continue to decrease, from 3GiB/s to 100MiB/s
I used this test case in this issue #33 (comment),
func TestSomething(t *testing.T) {
var total int64
var finish int64
md5Server := md5simd.NewServer()
n := 2000
go func() {
t := time.NewTicker(1 * time.Second)
lastTime := time.Now()
for range t.C {
elapsed := time.Since(lastTime)
processed := atomic.SwapInt64(&total, 0)
finished := atomic.SwapInt64(&finish, 0)
lastTime = time.Now()
fmt.Printf("%0.2fGB/s finish:%d\n", float64(processed)/elapsed.Seconds()/float64(1<<30), finished)
}
}()
var wg sync.WaitGroup
wg.Add(n)
for i := 0; i < n; i++ {
go func() {
defer wg.Done()
buf := make([]byte, 1048569)
for i := 0; i < len(buf); i++ {
buf[i] = byte(i * 53)
}
for i := 0; i < 1000; i++ {
time.Sleep(time.Duration(1500*rand.Float32()) * time.Millisecond)
c := md5Server.NewHash()
for j := 0; j < 15; j++ {
c.Write(buf)
atomic.AddInt64(&total, int64(len(buf)))
}
c.Sum(nil)
c.Close()
atomic.AddInt64(&finish, 1)
}
}()
}
wg.Wait()
}
and the final results are as follows:
=== RUN TestSomething
2.62GB/s finish:0
3.05GB/s finish:0
3.49GB/s finish:0
3.48GB/s finish:0
2.88GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
0.00GB/s finish:0
I suspect that it is due to competition, because when I try to replace Close when I can use Reset, the decline speed will be much slower in the production environment, but it will still decline to a very low speed in the end