rpi4: changing default CMA reservation with kernels 4.19.x fails to initialize vc4-drm gpu
joserpintuitivo opened this issue · comments
Describe the bug
I have a 4 GB RPI 4 running 32 bit kernel on Nerves Linux. The CMA reservation/allocation seems to be set to 64MB by default. If I try to pass 'cma=M' to the kernel command line, the video driver cannot be assigned from CMA.
Steps to reproduce the behaviour
1.) Compile the 4.19.x raspberrypi kernel for 32-bit mode with Nerves
2.) Boot the kernel with dtoverlay=vc4-fkms-v3d and append in the cmdline.txt file 'cma=256M'
3.) Observe the failure to start the vc4-drm gpu driver
Device (s)
Raspberry Pi 4 Mod. B
System
Linux version 4.19.118 (jose@jose-ThinkPad-T430) (gcc version 9.2.0 (crosstool-NG 1.24.0.71-4fa0ba1)) #2 SMP PREEMPT Tue Dec 13 16:25:21 -03 2022
CPU: ARMv7 Processor [410fd083] revision 3 (ARMv7), cr=30c5383d
Logs
cmd("dmesg | grep -i cma")
[ 0.000000] cma: Failed to reserve 256 MiB
[ 0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 cma=64M cma=256M smsc95xx.macaddr=E4:5F:01:63:91:2A vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000 dwc_otg.lpm_enable=0 console=tty1 console=ttyS0,115200 fbcon=scrollback:1024k root=/dev/mmcblk0p2 rootfstype=squashfs rootwait consoleblank=0
[ 0.000000] Memory: 3171444K/3211264K available (6144K kernel code, 384K rwdata, 1248K rodata, 2048K init, 411K bss, 39820K reserved, 0K cma-reserved, 3080192K highmem)
[ 5.441143] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.452259] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
cmd("dmesg | grep -i cma")
[ 0.000000] cma: Failed to reserve 256 MiB
[ 0.000000] Kernel command line: coherent_pool=1M [8250.nr](http://8250.nr/)_uarts=1 cma=64M bcm2708_fb.fbwidth=0 bcm2708_fb.fbheight=0 bcm2708_fb.fbswap=1 smsc95xx.macaddr=E4:5F:01:63:91:2A vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000 dwc_otg.lpm_enable=0 console=tty1 console=ttyS0,115200 fbcon=scrollback:1024k root=/dev/mmcblk0p2 rootfstype=squashfs rootwait consoleblank=0 cma=256M
[ 0.000000] Memory: 3171444K/3211264K available (6144K kernel code, 384K rwdata, 1248K rodata, 2048K init, 411K bss, 39820K reserved, 0K cma-reserved, 3080192K highmem)
[ 5.320059] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.330652] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
Additional context
No response
Any reason for using 4.19? The legacy LTS Buster release is 5.4
Any reason for using 4.19? The legacy LTS Buster release is 5.4
Yes, the reason is because I am using Gstreamer with Omx, and it only compiles for 32bits and is deprecated for later kernel versions.
Yes, the reason is because I am using Gstreamer with Omx, and it only compiles for 32bits and is deprecated for later kernel versions.
??!
OMX is still supported on 5.15 (and 6.1), but the userspace libraries aren't built with Bullseye (they are with Buster), and aren't supported at all with a 64bit userspace.
Yes, the reason is because I am using Gstreamer with Omx, and it only compiles for 32bits and is deprecated for later kernel versions.
??! OMX is still supported on 5.15 (and 6.1), but the userspace libraries aren't built with Bullseye (they are with Buster), and aren't supported at all with a 64bit userspace.
I'm using Nerves, and until this version they used 32 bits. https://github.com/nerves-project/nerves_system_rpi4/tree/v1.13.0
That's why I'm using 4.19 and it works for me, only I have a problem with the size of the CMA, and that's why I need to increase it.
Ok, the solution would be to go from version 4.19 to 5.4?
I think we’d need to know why the CMA parameter has changed. That might not be suitable for a 1GB Pi4
I think we’d need to know why the CMA parameter has changed. That might not be suitable for a 1GB Pi4
it is 4GB Pi4.
Why aren't you using the cma-256 parameter to the vc4-fkms-v3d overlay, as described in the README?
Why aren't you using the
cma-256parameter to thevc4-fkms-v3doverlay, as described in the README?
When I do that, this happens:
cmd("dmesg | grep -i cma")
[ 0.000000] cma: Failed to reserve 256 MiB
[ 0.000000] Kernel command line: coherent_pool=1M 8250.nr_uarts=1 cma=64M cma=256M smsc95xx.macaddr=E4:5F:01:63:91:2A vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000 dwc_otg.lpm_enable=0 console=tty1 console=ttyS0,115200 fbcon=scrollback:1024k root=/dev/mmcblk0p2 rootfstype=squashfs rootwait consoleblank=0
[ 0.000000] Memory: 3171444K/3211264K available (6144K kernel code, 384K rwdata, 1248K rodata, 2048K init, 411K bss, 39820K reserved, 0K cma-reserved, 3080192K highmem)
[ 5.441143] vc_sm_cma: module is from the staging directory, the quality is unknown, you have been warned.
[ 5.452259] bcm2835_vc_sm_cma_probe: Videocore shared memory driver
Later kernels switched to initialising CMA from Device Tree, but 4.19 is still command-line based. You could try using fdtget and fdtset to modify the command line in your current bcm2711-rpi-4-b.dtb, but if that doesn't work it's unlikely you'll get much support for a 4-year-old kernel.
Later kernels switched to initialising CMA from Device Tree, but 4.19 is still command-line based. You could try using fdtget and fdtset to modify the command line in your current bcm2711-rpi-4-b.dtb, but if that doesn't work it's unlikely you'll get much support for a 4-year-old kernel.
Yes, that is the problem I have and I don't know how to edit the devices tree.
Something like this (I advise you to also make a backup copy of the original .dtb file):
$ cp /boot/bcm2711-rpi-4-b.dtb .
$ fdtget -t s bcm2711-rpi-4-b.dtb /chosen bootargs > bootargs
$ nano bootargs
# Change the cma value and save the file
$ fdtput -t s bcm2711-rpi-4-b.dtb /chosen bootargs `cat bootargs`
$ fdtget -t s bcm2711-rpi-4-b.dtb /chosen bootargs
$ sudo cp bcm2711-rpi-4-b.dtb /boot/
Something like this (I advise you to also make a backup copy of the original .dtb file):
$ cp /boot/bcm2711-rpi-4-b.dtb . $ fdtget -t s bcm2711-rpi-4-b.dtb /chosen bootargs > bootargs $ nano bootargs # Change the cma value and save the file $ fdtput -t s bcm2711-rpi-4-b.dtb /chosen bootargs `cat bootargs` $ fdtget -t s bcm2711-rpi-4-b.dtb /chosen bootargs $ sudo cp bcm2711-rpi-4-b.dtb /boot/
Thanks. With the commands I was able to change the value of the CMA memory, but it seems that there is something else, because when I change it to 256M or 320M, its reservation fails.
What is the largest value that works?
And do you have any mem settings in config.txt, e.g. gpu_mem?
And do you have any mem settings in config.txt, e.g. gpu_mem?
# Default Nerves RPi 4 config.txt
#
# It's possible to override this file by using a custom fwup.conf
# configuration to pull in a replacement.
#
# Useful links:
# http://rpf.io/configtxt
# https://www.raspberrypi.org/documentation/configuration/device-tree.md
# https://github.com/raspberrypi/documentation/blob/master/configuration/device-tree.md
# https://github.com/raspberrypi/firmware/blob/master/boot/overlays/README
# Disable the boot rainbow
disable_splash=1
# This, along with the Raspberry Pi "x" firmware is needed for the camera
# to work. The Raspberry Pi "x" firmware is selected via the Buildroot
# configuration. See Target packages->Hardware handling->Firmware.
gpu_mem=192
# Enable I2C, SPI, and audio
dtparam=i2c_arm=on
dtparam=spi=on
dtparam=audio=on
# Enable drivers for the Raspberry Pi 7" Touchscreen
#
# This makes it possible to run Scenic out of the box with the popular
# Raspberry Pi 7" Touchscreen. It appears to be harmless for non-touchscreen
# users, but if not, let us know!
# NOTE: The touchscreen seems to work fine with these commented out on the RPi4
#dtoverlay=rpi-ft5406
#dtoverlay=rpi-backlight
# Comment this in or modify to enable OneWire
# NOTE: check that the overlay that you specify is in the boot partition or
# this won't work.
#dtoverlay=w1-gpio-pullup,gpiopin=4
# The ramoops overlay works with the pstore driver to preserve crash
# information across reboots in DRAM
dtoverlay=ramoops
# Enable the UART (/dev/ttyS0)
enable_uart=1
# Enable DRM VC4 V3D driver on top of the dispmanx display stack
#dtoverlay=vc4-fkms-v3d
#max_framebuffers=2
And do you have any mem settings in config.txt, e.g. gpu_mem?
I'll tell you what the maximum value is.
Do you need that large gpu_mem value? What happens if you remove it?
Do you need that large gpu_mem value? What happens if you remove it?
Yes, I need the GPU to be able to use the OMX to do the video encoding.
And what do you need the CMA for?
And what do you need the CMA for?
Because as I began to record videos, these errors with the application continually appeared and my rpi restarted.
[ 1163.448679] 2_scheduler invoked oom-killer: gfp_mask=0x6000c0(GFP_KERNEL), nodemask=(null), order=0, oom_score_adj=0
[ 1163.459534] 2_scheduler cpuset=/ mems_allowed=0
[ 1163.464301] CPU: 2 PID: 104 Comm: 2_scheduler Tainted: G C 4.19.118 #2
[ 1163.472332] Hardware name: BCM2835
[ 1163.475831] [<c021042c>] (unwind_backtrace) from [<c020bca0>] (show_stack+0x10/0x14)
[ 1163.483768] [<c020bca0>] (show_stack) from [<c077fc98>] (dump_stack+0x94/0xb4)
[ 1163.491173] [<c077fc98>] (dump_stack) from [<c02ea174>] (dump_header+0x70/0x24c)
[ 1163.498719] [<c02ea174>] (dump_header) from [<c02e92b0>] (oom_kill_process+0xb8/0x350)
[ 1163.506828] [<c02e92b0>] (oom_kill_process) from [<c02e9e20>] (out_of_memory+0x11c/0x378)
[ 1163.515201] [<c02e9e20>] (out_of_memory) from [<c02ef534>] (__alloc_pages_nodemask+0xbe8/0x11c0)
[ 1163.524195] [<c02ef534>] (__alloc_pages_nodemask) from [<c0682760>] (skb_page_frag_refill+0xe4/0xfc)
[ 1163.533507] [<c0682760>] (skb_page_frag_refill) from [<c06841b8>] (sk_page_frag_refill+0x14/0x6c)
[ 1163.542590] [<c06841b8>] (sk_page_frag_refill) from [<c06f022c>] (tcp_sendmsg_locked+0x43c/0xc18)
[ 1163.551670] [<c06f022c>] (tcp_sendmsg_locked) from [<c06f0a30>] (tcp_sendmsg+0x28/0x3c)
[ 1163.559873] [<c06f0a30>] (tcp_sendmsg) from [<c06800c4>] (sock_sendmsg+0x14/0x24)
[ 1163.567510] [<c06800c4>] (sock_sendmsg) from [<c068016c>] (sock_write_iter+0x98/0xf4)
[ 1163.575534] [<c068016c>] (sock_write_iter) from [<c0350250>] (do_iter_readv_writev+0x118/0x194)
[ 1163.584437] [<c0350250>] (do_iter_readv_writev) from [<c0350be4>] (do_iter_write+0x84/0x190)
[ 1163.593075] [<c0350be4>] (do_iter_write) from [<c0350d68>] (vfs_writev+0x78/0xe4)
[ 1163.600708] [<c0350d68>] (vfs_writev) from [<c0350e3c>] (do_writev+0x68/0xf0)
[ 1163.608023] [<c0350e3c>] (do_writev) from [<c0201000>] (ret_fast_syscall+0x0/0x4c)
[ 1163.615741] Exception stack(0xc1851fa8 to 0xc1851ff0)
[ 1163.620903] 1fa0: 00000010 b6bc00e0 00000031 b6bc00e0 00000010 00000000
[ 1163.629274] 1fc0: 00000010 b6bc00e0 b403c920 00000092 0005ad95 b6bc00d0 a9b42a0c 00082903
[ 1163.637610] 1fe0: 00000000 b403b9b0 b403c920 b6da3218
[ 1163.643240] Mem-Info:
[ 1163.645594] active_anon:19493 inactive_anon:2 isolated_anon:0
[ 1163.645594] active_file:1477 inactive_file:9296 isolated_file:0
[ 1163.645594] unevictable:0 dirty:0 writeback:5 unstable:0
[ 1163.645594] slab_reclaimable:1130 slab_unreclaimable:3224
[ 1163.645594] mapped:2121 shmem:3 pagetables:200 bounce:0
[ 1163.645594] free:755040 free_pcp:143 free_cma:15788
[ 1163.679473] Node 0 active_anon:79156kB inactive_anon:8kB active_file:2800kB inactive_file:34808kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:8484kB dirty:0kB writeback:20kB shmem:12kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 1163.702629] DMA free:64628kB min:636kB low:792kB high:948kB active_anon:0kB inactive_anon:0kB active_file:112kB inactive_file:20kB unevictable:0kB writepending:20kB present:131072kB managed:93180kB mlocked:0kB kernel_stack:1000kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:63152kB
[ 1163.729648] lowmem_reserve[]: 0 0 3008 3008
[ 1163.734061] HighMem free:2958876kB min:512kB low:19644kB high:38776kB active_anon:79040kB inactive_anon:8kB active_file:2856kB inactive_file:34612kB unevictable:0kB writepending:0kB present:3080192kB managed:3080192kB mlocked:0kB kernel_stack:0kB pagetables:800kB bounce:0kB free_pcp:1556kB local_pcp:0kB free_cma:0kB
[ 1163.762704] lowmem_reserve[]: 0 0 0 0
[ 1163.766577] DMA: 87*4kB (UEHC) 55*8kB (UEHC) 15*16kB (UEHC) 21*32kB (UEHC) 8*64kB (UEC) 5*128kB (UC) 2*256kB (C) 3*512kB (C) 1*1024kB (C) 1*2048kB (C) 14*4096kB (C) = 65316kB
[ 1163.782491] HighMem: 126*4kB (UM) 120*8kB (UM) 151*16kB (UM) 52*32kB (UM) 48*64kB (UM) 22*128kB (UM) 21*256kB (UM) 19*512kB (UM) 12*1024kB (UM) 10*2048kB (UM) 708*4096kB (UM) = 2959272kB
[ 1163.799477] 2754 total pagecache pages
[ 1163.799480] 0 pages in swap cache
[ 1163.799483] Swap cache stats: add 0, delete 0, find 0/0
[ 1163.799485] Free swap = 0kB
[ 1163.799487] Total swap = 0kB
[ 1163.799490] 802816 pages RAM
[ 1163.799492] 770048 pages HighMem/MovableOnly
[ 1163.799494] 9473 pages reserved
[ 1163.799496] 16384 pages cma reserved
[ 1163.799499] Tasks state (memory values in pages):
[ 1163.799502] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 1163.799516] [ 88] 0 88 418 48 28672 0 0 nbtty
[ 1163.854315] [ 95] 0 95 502 19 28672 0 0 rngd
[ 1163.854321] [ 96] 0 96 483 274 32768 0 0 nbtty
[ 1163.870991] [ 97] 0 97 66114 27965 585728 0 0 beam.smp
[ 1163.870996] [ 102] 0 102 433 255 24576 0 0 erl_child_setup
[ 1163.871001] [ 127] 0 127 399 43 20480 0 0 heart
[ 1163.871005] [ 131] 0 131 454 244 32768 0 0 uevent
[ 1163.871009] [ 134] 0 134 398 41 20480 0 0 kmsg_tailer
[ 1163.871012] [ 140] 0 140 455 51 24576 0 0 if_monitor
[ 1163.871016] [ 153] 0 153 400 45 24576 0 0 muontrap
[ 1163.871020] [ 154] 0 154 577 374 28672 0 0 ntpd
[ 1163.871025] [ 176] 0 176 400 45 24576 0 0 muontrap
[ 1163.871029] [ 177] 0 177 489 58 28672 0 0 udhcpc
[ 1163.871036] Out of memory: Kill process 97 (beam.smp) score 34 or sacrifice child
[ 1163.871049] Killed process 102 (erl_child_setup) total-vm:1732kB, anon-rss:56kB, file-rss:964kB, shmem-rss:0kB
[ 1163.871251] oom_reaper: reaped process 102 (erl_child_setup), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[ 1163.892263] heart: Erlang is crashing .. (waiting for crash dump file)
[ 1163.999468] init invoked oom-killer: gfp_mask=0x6040c0(GFP_KERNEL|__GFP_COMP), nodemask=(null), order=2, oom_score_adj=0
[ 1164.017310] init cpuset=/ mems_allowed=0
[ 1164.021332] CPU: 3 PID: 1 Comm: init Tainted: G C 4.19.118 #2
[ 1164.028555] Hardware name: BCM2835
[ 1164.032055] [<c021042c>] (unwind_backtrace) from [<c020bca0>] (show_stack+0x10/0x14)
[ 1164.039989] [<c020bca0>] (show_stack) from [<c077fc98>] (dump_stack+0x94/0xb4)
[ 1164.047394] [<c077fc98>] (dump_stack) from [<c02ea174>] (dump_header+0x70/0x24c)
[ 1164.054938] [<c02ea174>] (dump_header) from [<c02e92b0>] (oom_kill_process+0xb8/0x350)
[ 1164.063044] [<c02e92b0>] (oom_kill_process) from [<c02e9e20>] (out_of_memory+0x11c/0x378)
[ 1164.071415] [<c02e9e20>] (out_of_memory) from [<c02ef534>] (__alloc_pages_nodemask+0xbe8/0x11c0)
[ 1164.080370] [<c02ef534>] (__alloc_pages_nodemask) from [<c0311380>] (kmalloc_order+0x20/0x30)
[ 1164.089094] [<c0311380>] (kmalloc_order) from [<c03113ac>] (kmalloc_order_trace+0x1c/0xf4)
[ 1164.097555] [<c03113ac>] (kmalloc_order_trace) from [<c026b8a4>] (devkmsg_open+0x50/0xf0)
[ 1164.105928] [<c026b8a4>] (devkmsg_open) from [<c03567d8>] (chrdev_open+0xb0/0x19c)
[ 1164.113651] [<c03567d8>] (chrdev_open) from [<c034e4c4>] (do_dentry_open+0x114/0x3bc)
[ 1164.121673] [<c034e4c4>] (do_dentry_open) from [<c035fe9c>] (path_openat+0x2bc/0x109c)
[ 1164.129780] [<c035fe9c>] (path_openat) from [<c0360cf0>] (do_filp_open+0x74/0xe8)
[ 1164.137411] [<c0360cf0>] (do_filp_open) from [<c034fd24>] (do_sys_open+0x17c/0x200)
[ 1164.145254] [<c034fd24>] (do_sys_open) from [<c0201000>] (ret_fast_syscall+0x0/0x4c)
[ 1164.153181] Exception stack(0xc1c33fa8 to 0xc1c33ff0)
[ 1164.158341] 3fa0: 0000001b b6f20d10 ffffff9c 00016118 000a0001 00000000
[ 1164.166710] 3fc0: 0000001b b6f20d10 0000001b 00000142 befddd9c befddd98 00015d24 00015d3c
[ 1164.175078] 3fe0: 0002707c befdd660 000132c8 b6e71248
[ 1164.180301] Mem-Info:
[ 1164.182818] active_anon:75 inactive_anon:2 isolated_anon:0
[ 1164.182818] active_file:396 inactive_file:183 isolated_file:0
[ 1164.182818] unevictable:0 dirty:1 writeback:0 unstable:0
[ 1164.182818] slab_reclaimable:1131 slab_unreclaimable:3144
[ 1164.182818] mapped:372 shmem:3 pagetables:29 bounce:0
[ 1164.182818] free:785910 free_pcp:4 free_cma:15788
[ 1164.215521] Node 0 active_anon:300kB inactive_anon:8kB active_file:1584kB inactive_file:732kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:1488kB dirty:4kB writeback:0kB shmem:12kB writeback_tmp:0kB unstable:0kB all_unreclaimable? yes
[ 1164.238133] DMA free:68076kB min:636kB low:792kB high:948kB active_anon:0kB inactive_anon:0kB active_file:72kB inactive_file:40kB unevictable:0kB writepending:4kB present:131072kB managed:93180kB mlocked:0kB kernel_stack:728kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:63152kB
[ 1164.264834] lowmem_reserve[]: 0 0 3008 3008
[ 1164.269122] HighMem free:3075648kB min:512kB low:19644kB high:38776kB active_anon:300kB inactive_anon:8kB active_file:1664kB inactive_file:1120kB unevictable:0kB writepending:0kB present:3080192kB managed:3080192kB mlocked:0kB kernel_stack:0kB pagetables:116kB bounce:0kB free_pcp:16kB local_pcp:0kB free_cma:0kB
[ 1164.297194] lowmem_reserve[]: 0 0 0 0
[ 1164.300981] DMA: 140*4kB (UEHC) 154*8kB (UEHC) 68*16kB (UEHC) 34*32kB (UEHC) 17*64kB (UEHC) 5*128kB (UC) 2*256kB (C) 3*512kB (C) 1*1024kB (C) 1*2048kB (C) 14*4096kB (C) = 68160kB
[ 1164.317231] HighMem: 188*4kB (UM) 145*8kB (UM) 129*16kB (UM) 121*32kB (UM) 111*64kB (UM) 88*128kB (UM) 72*256kB (UM) 48*512kB (M) 26*1024kB (M) 11*2048kB (M) 722*4096kB (UM) = 3075688kB
[ 1164.334124] 584 total pagecache pages
[ 1164.337873] 0 pages in swap cache
[ 1164.341271] Swap cache stats: add 0, delete 0, find 0/0
[ 1164.346646] Free swap = 0kB
[ 1164.349601] Total swap = 0kB
[ 1164.352562] 802816 pages RAM
[ 1164.355517] 770048 pages HighMem/MovableOnly
[ 1164.359917] 9473 pages reserved
[ 1164.363179] 16384 pages cma reserved
[ 1164.366839] Unreclaimable slab info:
[ 1164.370532] Name Used Total
[ 1164.376062] RAWv6 15KB 15KB
[ 1164.381431] UDPv6 15KB 15KB
[ 1164.386774] TCPv6 31KB 31KB
[ 1164.392151] mqueue_inode_cache 15KB 15KB
[ 1164.397578] UNIX 63KB 63KB
[ 1164.402926] RAW 31KB 31KB
[ 1164.408296] UDP 61KB 61KB
[ 1164.413638] tw_sock_TCP 3KB 3KB
[ 1164.419009] TCP 121KB 121KB
[ 1164.424359] eventpoll_pwq 15KB 15KB
[ 1164.429731] request_queue 62KB 62KB
[ 1164.435073] blkdev_ioc 16KB 16KB
[ 1164.440442] biovec-max 300KB 300KB
[ 1164.445791] biovec-128 126KB 126KB
[ 1164.451162] biovec-64 63KB 63KB
[ 1164.456503] biovec-16 165KB 165KB
[ 1164.461879] skbuff_head_cache 283KB 283KB
[ 1164.467215] configfs_dir_cache 3KB 3KB
[ 1164.472652] file_lock_cache 15KB 15KB
[ 1164.478024] shmem_inode_cache 108KB 108KB
[ 1164.483367] proc_dir_entry 56KB 56KB
[ 1164.488737] pde_opener 15KB 15KB
[ 1164.494088] seq_file 15KB 15KB
[ 1164.499458] sigqueue 15KB 15KB
[ 1164.504800] kernfs_node_cache 1346KB 1346KB
[ 1164.510170] filp 263KB 263KB
[ 1164.515520] names_cache 128KB 128KB
[ 1164.520888] key_jar 16KB 16KB
[ 1164.526233] vm_area_struct 154KB 154KB
[ 1164.531609] mm_struct 32KB 32KB
[ 1164.536945] files_cache 16KB 16KB
[ 1164.542294] signal_cache 109KB 109KB
[ 1164.547664] sighand_cache 220KB 220KB
[ 1164.553006] task_struct 284KB 284KB
[ 1164.558377] cred_jar 36KB 36KB
[ 1164.563725] anon_vma_chain 28KB 28KB
[ 1164.569095] anon_vma 31KB 31KB
[ 1164.574436] pid 16KB 16KB
[ 1164.579804] trace_event_file 35KB 35KB
[ 1164.585152] ftrace_event_field 84KB 84KB
[ 1164.590609] pool_workqueue 16KB 16KB
[ 1164.595952] task_group 83KB 121KB
[ 1164.601326] kmalloc-8192 160KB 160KB
[ 1164.606674] kmalloc-4096 1504KB 1504KB
[ 1164.612017] kmalloc-2048 480KB 480KB
[ 1164.617387] kmalloc-1024 304KB 304KB
[ 1164.622729] kmalloc-512 452KB 488KB
[ 1164.628098] kmalloc-256 76KB 76KB
[ 1164.633447] kmalloc-192 86KB 86KB
[ 1164.638818] kmalloc-128 184KB 184KB
[ 1164.644161] kmalloc-64 4569KB 4584KB
[ 1164.649533] kmem_cache_node 8KB 8KB
[ 1164.654883] kmem_cache 24KB 24KB
[ 1164.660252] Tasks state (memory values in pages):
[ 1164.665064] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[ 1164.673889] [ 95] 0 95 502 19 28672 0 0 rngd
[ 1164.682154] [ 127] 0 127 399 43 20480 0 0 heart
[ 1164.690526] Out of memory: Kill process 127 (heart) score 0 or sacrifice child
[ 1164.697939] Killed process 127 (heart) total-vm:1596kB, anon-rss:28kB, file-rss:144kB, shmem-rss:0kB
[ 1164.707331] erlinit: Erlang VM exited
[ 1164.707435] oom_reaper: reaped process 127 (heart), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
[ 1164.711202] erlinit: Sending SIGTERM to all processes
[ 1164.720354] watchdog: watchdog0: watchdog did not stop!
[ 1165.745917] erlinit: Sending SIGKILL to all processes
[ 1165.881653] erlinit: umount 0 failed: No such file or directory
[ 1166.387850] reboot: Restarting system
20:31:49.738 [warn] kthreadd invoked oom-killer: gfp_mask=0x7080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), nodemask=(null), order=1, oom_score_adj=0
20:31:49.739 [info] kthreadd cpuset=/ mems_allowed=0
20:31:49.739 [warn] CPU: 2 PID: 2 Comm: kthreadd Tainted: G C 4.19.118 #2
20:31:49.739 [warn] Hardware name: BCM2835
20:31:49.739 [warn] [<c021042c>] (unwind_backtrace) from [<c020bca0>] (show_stack+0x10/0x14)
20:31:49.739 [warn] [<c020bca0>] (show_stack) from [<c077fc98>] (dump_stack+0x94/0xb4)
20:31:49.739 [warn] [<c077fc98>] (dump_stack) from [<c02ea174>] (dump_header+0x70/0x24c)
20:31:49.739 [warn] [<c02ea174>] (dump_header) from [<c02e92b0>] (oom_kill_process+0xb8/0x350)
20:31:49.739 [warn] [<c02e92b0>] (oom_kill_process) from [<c02e9e20>] (out_of_memory+0x11c/0x378)
20:31:49.739 [warn] [<c02e9e20>] (out_of_memory) from [<c02ef534>] (__alloc_pages_nodemask+0xbe8/0x11c0)
20:31:49.739 [warn] [<c02ef534>] (__alloc_pages_nodemask) from [<c0218400>] (copy_process.part.0+0x174/0x19c4)
20:31:49.739 [warn] [<c0218400>] (copy_process.part.0) from [<c0219dc8>] (_do_fork+0xb4/0x43c)
20:31:49.739 [warn] [<c0219dc8>] (_do_fork) from [<c021a19c>] (kernel_thread+0x2c/0x34)
20:31:49.740 [warn] [<c021a19c>] (kernel_thread) from [<c02395f0>] (kthreadd+0x190/0x204)
20:31:49.740 [warn] [<c02395f0>] (kthreadd) from [<c02010d8>] (ret_from_fork+0x14/0x3c)
20:31:49.740 [warn] Exception stack(0xc1c95fb0 to 0xc1c95ff8)
20:31:49.740 [warn] 5fa0: 00000000 00000000 00000000 00000000
20:31:49.740 [warn] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
20:31:49.740 [warn] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000
20:31:49.780 [warn] Mem-Info:
20:31:49.780 [warn] active_anon:15954 inactive_anon:2 isolated_anon:0\x0a active_file:2636 inactive_file:10575 isolated_file:0\x0a unevictable:0 dirty:260 writeback:152 unstable:0\x0a slab_reclaimable:1190 slab_unreclaimable:3288\x0a mapped:3819 shmem:3 pagetables:199 bounce:0\x0a free:724785 free_pcp:1038 free_cma:15424
20:31:49.806 [warn] Node 0 active_anon:65888kB inactive_anon:8kB active_file:10544kB inactive_file:42300kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:17496kB dirty:1040kB writeback:608kB shmem:12kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
20:31:49.836 [warn] DMA free:63672kB min:636kB low:792kB high:948kB active_anon:0kB inactive_anon:0kB active_file:636kB inactive_file:596kB unevictable:0kB writepending:1112kB present:131072kB managed:93180kB mlocked:0kB kernel_stack:1184kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:61680kB
20:31:49.844 [warn] lowmem_reserve[]: 0 0 3008 3008
20:31:49.884 [warn] HighMem free:2836476kB min:512kB low:19644kB high:38776kB active_anon:64164kB inactive_anon:8kB active_file:9664kB inactive_file:41376kB unevictable:0kB writepending:0kB present:3080192kB managed:3080192kB mlocked:0kB kernel_stack:0kB pagetables:796kB bounce:0kB free_pcp:2264kB local_pcp:984kB free_cma:0kB
20:31:49.891 [warn] lowmem_reserve[]: 0 0 0 0
20:31:49.907 [warn] DMA: 26*4kB (UEHC) 89*8kB (UEHC) 57*16kB (UEHC) 45*32kB (UHC) 8*64kB (C) 6*128kB (C) 4*256kB (C) 2*512kB (C) 2*1024kB (C) 3*2048kB (C) 12*4096kB (C) = 63840kB
20:31:49.925 [warn] HighMem: 1*4kB (U) 2*8kB (UM) 7*16kB (UM) 1*32kB (U) 33*64kB (UM) 11*128kB (M) 4*256kB (UM) 6*512kB (UM) 8*1024kB (M) 2*2048kB (M) 686*4096kB (UM) = 2829924kB
20:31:49.945 [warn] 4893 total pagecache pages
20:31:49.956 [warn] 0 pages in swap cache
20:31:49.956 [warn] Swap cache stats: add 0, delete 0, find 0/0
20:31:49.963 [warn] Free swap = 0kB
20:31:49.963 [warn] Total swap = 0kB
20:31:49.972 [warn] 802816 pages RAM
20:31:49.972 [warn] 770048 pages HighMem/MovableOnly
20:31:49.983 [warn] 9473 pages reserved
20:31:49.983 [warn] 16384 pages cma reserved
20:31:49.991 [info] Tasks state (memory values in pages):
20:31:50.003 [info] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
20:31:50.012 [info] [ 88] 0 88 418 48 28672 0 0 nbtty
20:31:50.023 [info] [ 96] 0 96 502 19 24576 0 0 rngd
20:31:50.033 [info] [ 97] 0 97 483 276 32768 0 0 nbtty
20:31:50.043 [info] [ 98] 0 98 40315 19253 311296 0 0 beam.smp
20:31:50.076 [info] [ 102] 0 102 433 255 28672 0 0 erl_child_setup
20:31:50.076 [info] [ 125] 0 125 399 43 20480 0 0 heart
20:31:50.094 [info] [ 129] 0 129 454 233 24576 0 0 uevent
20:31:50.094 [info] [ 133] 0 133 398 42 24576 0 0 kmsg_tailer
20:31:50.103 [info] [ 138] 0 138 455 51 32768 0 0 if_monitor
20:31:50.127 [info] [ 151] 0 151 400 45 20480 0 0 muontrap
20:31:50.128 [info] [ 152] 0 152 577 379 28672 0 0 ntpd
20:31:50.128 [info] [ 173] 0 173 400 45 20480 0 0 muontrap
20:31:50.145 [info] [ 174] 0 174 489 58 28672 0 0 udhcpc
20:31:50.145 [info] [ 224] 0 224 586 19 28672 0 0 epmd
20:31:50.163 [info] [ 876] 0 876 400 45 24576 0 0 muontrap
20:31:50.163 [info] [ 877] 0 877 64620 7156 241664 0 0 gst-launch-1.0
20:31:50.190 [error] Out of memory: Kill process 98 (beam.smp) score 24 or sacrifice child
20:31:50.190 [error] Killed process 102 (erl_child_setup) total-vm:1732kB, anon-rss:56kB, file-rss:964kB, shmem-rss:0kB
20:31:50.194 [info] oom_reaper: reaped process 102 (erl_child_setup), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
13:44:24.896 [error] cma: cma_alloc: alloc failed, req-size: 3 pages, ret: -4
……….
21:01:58.306 [warn] kthreadd invoked oom-killer: gfp_mask=0x7080c0(GFP_KERNEL_ACCOUNT|__GFP_ZERO), nodemask=(null), order=1, oom_score_adj=0
21:01:58.307 [info] kthreadd cpuset=/ mems_allowed=0
21:01:58.307 [warn] CPU: 0 PID: 2 Comm: kthreadd Tainted: G C 4.19.118 #2
21:01:58.307 [warn] Hardware name: BCM2835
21:01:58.317 [warn] [<c021042c>] (unwind_backtrace) from [<c020bca0>] (show_stack+0x10/0x14)
21:01:58.318 [warn] [<c020bca0>] (show_stack) from [<c077fc98>] (dump_stack+0x94/0xb4)
21:01:58.318 [warn] [<c077fc98>] (dump_stack) from [<c02ea174>] (dump_header+0x70/0x24c)
21:01:58.318 [warn] [<c02ea174>] (dump_header) from [<c02e92b0>] (oom_kill_process+0xb8/0x350)
21:01:58.318 [warn] [<c02e92b0>] (oom_kill_process) from [<c02e9e20>] (out_of_memory+0x11c/0x378)
21:01:58.318 [warn] [<c02e9e20>] (out_of_memory) from [<c02ef534>] (__alloc_pages_nodemask+0xbe8/0x11c0)
21:01:58.318 [warn] [<c02ef534>] (__alloc_pages_nodemask) from [<c0218400>] (copy_process.part.0+0x174/0x19c4)
21:01:58.318 [warn] [<c0218400>] (copy_process.part.0) from [<c0219dc8>] (_do_fork+0xb4/0x43c)
21:01:58.319 [warn] [<c0219dc8>] (_do_fork) from [<c021a19c>] (kernel_thread+0x2c/0x34)
21:01:58.319 [warn] [<c021a19c>] (kernel_thread) from [<c02395f0>] (kthreadd+0x190/0x204)
21:01:58.319 [warn] [<c02395f0>] (kthreadd) from [<c02010d8>] (ret_from_fork+0x14/0x3c)
21:01:58.319 [warn] Exception stack(0xc1c95fb0 to 0xc1c95ff8)
21:01:58.319 [warn] 5fa0: 00000000 00000000 00000000 00000000
21:01:58.319 [warn] 5fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
21:01:58.319 [warn] 5fe0: 00000000 00000000 00000000 00000000 00000013 00000000
21:01:58.330 [info] MAIN_UPLOAD SUCCESS "/root/videos/cce78a06-2933-46cf-9716-ecedb43379b2/cam0-7461001a-773b-11ed-aa71-e45f0163912a-video-00002.mkv"
21:01:58.350 [warn] Mem-Info:
21:01:58.350 [warn] active_anon:11243 inactive_anon:2 isolated_anon:0\x0a active_file:711 inactive_file:11244 isolated_file:0\x0a unevictable:0 dirty:11 writeback:73 unstable:0\x0a slab_reclaimable:1168 slab_unreclaimable:3312\x0a mapped:2110 shmem:3 pagetables:167 bounce:0\x0a free:762736 free_pcp:63 free_cma:15778
21:01:58.373 [warn] Node 0 active_anon:45252kB inactive_anon:8kB active_file:2852kB inactive_file:44964kB unevictable:0kB isolated(anon):0kB isolated(file):0kB mapped:8440kB dirty:64kB writeback:0kB shmem:12kB writeback_tmp:0kB unstable:0kB all_unreclaimable? no
21:01:58.401 [warn] DMA free:66372kB min:636kB low:792kB high:948kB active_anon:0kB inactive_anon:0kB active_file:120kB inactive_file:100kB unevictable:0kB writepending:24kB present:131072kB managed:93180kB mlocked:0kB kernel_stack:960kB pagetables:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:63112kB
21:01:58.434 [warn] lowmem_reserve[]: 0 0 3008 3008
21:01:58.434 [warn] HighMem free:2983544kB min:512kB low:19644kB high:38776kB active_anon:45264kB inactive_anon:8kB active_file:3172kB inactive_file:45288kB unevictable:0kB writepending:0kB present:3080192kB managed:3080192kB mlocked:0kB kernel_stack:0kB pagetables:672kB bounce:0kB free_pcp:708kB local_pcp:444kB free_cma:0kB
21:01:58.455 [warn] lowmem_reserve[]: 0 0 0 0
21:01:58.455 [warn] DMA: 43*4kB (UHC) 75*8kB (UHC) 71*16kB (UHC) 42*32kB (UHC) 9*64kB (UC) 4*128kB (C) 4*256kB (C) 3*512kB (C) 2*1024kB (C) 2*2048kB (C) 13*4096kB (C) = 66292kB
21:01:58.473 [warn] HighMem: 528*4kB (UM) 747*8kB (UM) 506*16kB (UM) 325*32kB (UM) 190*64kB (UM) 157*128kB (UM) 117*256kB (UM) 61*512kB (UM) 35*1024kB (UM) 17*2048kB (UM) 682*4096kB (UM) = 2984152kB
21:01:58.477 [warn] 2850 total pagecache pages
21:01:58.489 [warn] 0 pages in swap cache
21:01:58.489 [warn] Swap cache stats: add 0, delete 0, find 0/0
21:01:58.489 [warn] Free swap = 0kB
21:01:58.500 [warn] Total swap = 0kB
21:01:58.500 [warn] 802816 pages RAM
21:01:58.500 [warn] 770048 pages HighMem/MovableOnly
21:01:58.511 [warn] 9473 pages reserved
21:01:58.512 [warn] 16384 pages cma reserved
21:01:58.512 [info] Tasks state (memory values in pages):
21:01:58.520 [info] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
21:01:58.529 [info] [ 88] 0 88 418 48 24576 0 0 nbtty
21:01:58.546 [info] [ 95] 0 95 502 19 28672 0 0 rngd
21:01:58.546 [info] [ 96] 0 96 483 307 28672 0 0 nbtty
21:01:58.554 [info] [ 97] 0 97 47247 21489 421888 0 0 beam.smp
21:01:58.564 [info] [ 101] 0 101 433 287 24576 0 0 erl_child_setup
21:01:58.572 [info] [ 126] 0 126 399 43 24576 0 0 heart
21:01:58.582 [info] [ 130] 0 130 454 232 28672 0 0 uevent
21:01:58.582 [info] [ 133] 0 133 398 42 20480 0 0 kmsg_tailer
21:01:58.599 [info] [ 139] 0 139 455 51 28672 0 0 if_monitor
21:01:58.599 [info] [ 152] 0 152 400 45 28672 0 0 muontrap
21:01:58.608 [info] [ 153] 0 153 577 395 32768 0 0 ntpd
21:01:58.625 [info] [ 201] 0 201 400 45 28672 0 0 muontrap
21:01:58.625 [info] [ 202] 0 202 489 58 32768 0 0 udhcpc
21:01:58.633 [info] [ 311] 0 311 586 19 28672 0 0 epmd
21:01:58.649 [error] Out of memory: Kill process 97 (beam.smp) score 27 or sacrifice child
21:01:58.649 [error] Killed process 101 (erl_child_setup) total-vm:1732kB, anon-rss:56kB, file-rss:1092kB, shmem-rss:0kB
21:01:58.670 [info] oom_reaper: reaped process 101 (erl_child_setup), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB
I think it's worth trying to increase CMA as much as you can to see if it makes any difference at all. So far I'm not convinced it will.
I think it's worth trying to increase CMA as much as you can to see if it makes any difference at all. So far I'm not convinced it will.
In the rpi3 with the same Kernel version, I was able to increase the CMA memory without problem and these errors stopped appearing. But really I'm going to use an rpi4, I'm going to have to migrate to kernel 5.4. Thanks for your help.
Yes, I need the GPU to be able to use the OMX to do the video encoding.
The video encoder needs the new and 1 reference frame, and is max 1920x1080. User buffers (as configured via IL's nBufferCount on the port format) are on top of that. It also has a 4MB output FIFO.
Each video encode at 1080p is therefore taking possibly 20MB of gpu_mem.
The camera does require gpu_mem, but the default with startx=1 of 128M is usually sufficient unless asking for multiple 5/8/12MPix 24bpp or 32bpp RGB images.
Just noting from your first post
[ 0.000000] Memory: 3171444K/3211264K available
That makes it look like something has stolen almost 1GB of the RAM (3136MB left, so 960MB taken).
My CM4 with gpu_mem=76 reports
[ 0.000000] Memory: 3599276K/4050944K available (12288K kernel code, 1386K rwdata, 3268K rodata, 2048K init, 591K bss, 123988K reserved, 327680K cma-reserved, 3264512K highmem)
4050944 = 3956MB.