[BUG]: Multimap count and pair_retrieve fail on 1024M input table

[Rui Lan]

Is this a duplicate?

• I confirmed there appear to be no duplicate issues for this bug (https://github.com/NVIDIA/cuCollections/issues)

Type of Bug

Runtime Error

Describe the bug

cuco::static_multimap::count and cuco::static_multimap::pair_retrieve fail when the input array is large. In my case, the APIs work fine on 512M input but have problems with 1024M input. The count call returns negative results, and the pair_retrieve call fails with an illegal memory access error.

The issue seems to be related to the probe sequence. The default double hashing implementation generates the above issues but changing to linear probing resolves the issues.

The issue is also related to the match rate. The input elements of count and pair_retrieve calls have matches to the inserted keys (match rate = 100%). I also tested the 3% match rate which seems to work fine

How to Reproduce

1. Construct a cuco::static_multimap with 4B int key and 4B int payload (8B key/payload also has the above issue). Use 50% occupancy for the map
2. Insert 128M unique keys
3. Call count or pair_retrieve on 1024M keys and all keys have matches with the inserted keys

Expected behavior

cuco::static_multimap::count should return the correct number of matches.
cuco::static_multimap::pair_retrieve should materialize the results and not fail with the illegal memory access error.

Reproduction link

No response

Operating System

Ubuntu 20.04.4 LTS

nvidia-smi output

Thu Dec 15 14:47:48 2022
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 525.33 Driver Version: 525.33 CUDA Version: 12.0 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+======================+======================|
| 0 NVIDIA A100 80G... On | 00000000:01:00.0 Off | Off |
| N/A 44C P0 66W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 1 NVIDIA A100 80G... On | 00000000:23:00.0 Off | Off |
| N/A 44C P0 66W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 2 NVIDIA A100 80G... On | 00000000:41:00.0 Off | Off |
| N/A 43C P0 67W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 3 NVIDIA A100 80G... On | 00000000:61:00.0 Off | 0 |
| N/A 43C P0 65W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 4 NVIDIA A100 80G... On | 00000000:81:00.0 Off | Off |
| N/A 42C P0 63W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 5 NVIDIA A100 80G... On | 00000000:A1:00.0 Off | Off |
| N/A 44C P0 67W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 6 NVIDIA A100 80G... On | 00000000:C1:00.0 Off | 0 |
| N/A 41C P0 61W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+
| 7 NVIDIA A100 80G... On | 00000000:E1:00.0 Off | 0 |
| N/A 43C P0 66W / 300W | 0MiB / 81920MiB | 0% Default |
| | | Disabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=============================================================================|
| No running processes found |
+-----------------------------------------------------------------------------+

NVCC version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2022 NVIDIA Corporation
Built on Wed_Jun__1_21:24:09_PDT_2022
Cuda compilation tools, release 11.8, V11.8.40
Build cuda_11.8.r11.8/compiler.31415974_0