pcstat syscall mincore failed with bad address if file is larger than 512MB on arm64 4K_PAGES machines

[Dianek]

System: Ubuntu 16.04 LTS on Cavium 48core ARMv8 ThunderX1
Linux thunderx1 4.4.0-36-generic #55-Ubuntu SMP Thu Aug 11 18:05:42 UTC 2016 aarch64 aarch64 aarch64 GNU/Linux

pcstat failed if file is larger than 512MB. Our page size was 4096, however pcstat used 65536. pcstat also calculated the wrong page count on arm64 system.

Test 1:
pcstat /var/lib/cassandra/data/keyspace1/standard1-19ce2110716e11e7843f219b546b9b42/test
+-----------------------------------------------------------------------------------+----------------+------------+-----------+---------+
| Name | Size (bytes) | Pages | Cached | Percent |
|-----------------------------------------------------------------------------------+----------------+------------+-----------+---------|
| /var/lib/cassandra/data/keyspace1/standard1-19ce2110716e11e7843f219b546b9b42/test | 525107053 | 8013 | 8013 | 100.000 |
+-----------------------------------------------------------------------------------+----------------+------------+-----------+---------+

Test 2:
cat mc-69-big-Data.db >> test

pcstat /var/lib/cassandra/data/keyspace1/standard1-19ce2110716e11e7843f219b546b9b42/test
2017/08/08 13:08:14 skipping "/var/lib/cassandra/data/keyspace1/standard1-19ce2110716e11e7843f219b546b9b42/test": syscall SYS_MINCORE failed: bad address
+-------+----------------+------------+-----------+---------+
| Name | Size (bytes) | Pages | Cached | Percent |
|-------+----------------+------------+-----------+---------|
+-------+----------------+------------+-----------+---------+

ls -l /var/lib/cassandra/data/keyspace1/standard1-19ce2110716e11e7843f219b546b9b42
-rw-r--r-- 1 root root 599909718 Aug 8 13:08 test

getconf PAGE_SIZE
4096

cat /proc/meminfo
MemTotal: 32929196 kB
MemFree: 184936 kB
MemAvailable: 19768376 kB
Buffers: 87476 kB
Cached: 19477888 kB
SwapCached: 0 kB
Active: 5676236 kB
Inactive: 15960632 kB
Active(anon): 2092140 kB
Inactive(anon): 8664 kB
Active(file): 3584096 kB
Inactive(file): 15951968 kB
Unevictable: 9921816 kB
Mlocked: 9921816 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 4 kB
Writeback: 0 kB
AnonPages: 11993340 kB
Mapped: 11821268 kB
Shmem: 9284 kB
Slab: 423280 kB
SReclaimable: 340060 kB
SUnreclaim: 83220 kB
KernelStack: 18416 kB
PageTables: 279516 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 16464596 kB
Committed_AS: 15227224 kB
VmallocTotal: 133142937536 kB
VmallocUsed: 0 kB
VmallocChunk: 0 kB
AnonHugePages: 9535488 kB
CmaTotal: 0 kB
CmaFree: 0 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

[Dianek]

golang.org hardcode arm64 pagesize to 65536, which may not be correct

golang.org/x/sys/unix/syscall_linux_arm64.go:func Getpagesize() int { return 65536 }

[Dianek]

golang 1.9 fixed page size problem, however, pcstat cannot be built under golang 1.9 (arm64)

[Amy Tobey]

Hey! Sorry about the delay. I'll take a look at cross-compiling and figuring this out soon. I have a Pine64 board here somewhere too so I can test.