*** stack smashing detected ***: terminated Fatal error 6: Aborted
sahirzm opened this issue · comments
Problem
Hi,
I updated tree-sitter on ArchLinux from 0.22.2-1 -> 0.22.4-1. After the upgrade, emacs has started to crash whenever I load any tsx file with the following backtrace:
*** stack smashing detected ***: terminated
Fatal error 6: Aborted
Backtrace:
emacs(+0x18a436)[0x5de6a212b436]
emacs(+0x24df3)[0x5de6a1fc5df3]
emacs(+0x25d01)[0x5de6a1fc6d01]
emacs(+0x2cf70d)[0x5de6a227070d]
/usr/lib/libc.so.6(+0x3c770)[0x727ab68fd770]
/usr/lib/libc.so.6(+0x8d32c)[0x727ab694e32c]
/usr/lib/libc.so.6(gsignal+0x18)[0x727ab68fd6c8]
/usr/lib/libc.so.6(abort+0xd7)[0x727ab68e54b8]
/usr/lib/libc.so.6(+0x25395)[0x727ab68e6395]
/usr/lib/libc.so.6(+0x11473b)[0x727ab69d573b]
/usr/lib/libc.so.6(+0x115a56)[0x727ab69d6a56]
emacs(+0x299883)[0x5de6a223a883]
/usr/lib/emacs/29.3/native-lisp/29.3-561b282f/treesit-37439c61-97df641d.eln(F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0+0x1f2)[0x727a8eb6e832]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0+0x62)[0x727ab1e38f42]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0+0x4af)[0x727ab1e36bef]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/font-lock-895216f6-1f3b244f.eln(F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0+0x93)[0x727ab1e35873]
emacs(+0x25789e)[0x5de6a21f889e]
emacs(+0x21052e)[0x5de6a21b152e]
emacs(+0x211041)[0x5de6a21b2041]
emacs(+0x20ccac)[0x5de6a21adcac]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0+0xd8)[0x727ab1acac88]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0+0x80a)[0x727ab1acb59a]
emacs(+0x21052e)[0x5de6a21b152e]
/usr/bin/../lib/emacs/29.3/native-lisp/29.3-561b282f/preloaded/jit-lock-8a988e43-a9956d8b.eln(F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0+0x26f)[0x727ab1aca98f]
emacs(+0x21052e)[0x5de6a21b152e]
emacs(+0x2d2b21)[0x5de6a2273b21]
emacs(+0x5706b)[0x5de6a1ff806b]
emacs(+0x5af00)[0x5de6a1ffbf00]
emacs(+0x5c086)[0x5de6a1ffd086]
emacs(+0x5e036)[0x5de6a1fff036]
emacs(+0x5c2b6)[0x5de6a1ffd2b6]
emacs(+0x7ae89)[0x5de6a201be89]
emacs(+0x82d92)[0x5de6a2023d92]
emacs(+0x73863)[0x5de6a2014863]
emacs(+0x20b43c)[0x5de6a21ac43c]
emacs(+0x737b8)[0x5de6a20147b8]
emacs(+0x774b2)[0x5de6a20184b2]
...
zsh: IOT instruction (core dumped) emacs
I downgraded back to 0.22.2-1 and do not face any issues.
Steps to reproduce
- pacman -Syyu <- updates tree-sitter to 0.22.4-1
- Start emacs
- Open any
tsxfile.
Expected behavior
Emacs should not crash and I can open tsx files.
Tree-sitter version (tree-sitter --version)
tree-sitter 0.20.8 (d4c1bf7)
Operating system/version
ArchLinux
there's not much in this backtrace that we can go off of. are you able to build with debug syms and the frame pointer enabled? (-fno-omit-frame-pointer, if using clang you might also need -mno-omit-leaf-frame-pointer)
I suspect we've hit the same thing in Gentoo as https://bugs.gentoo.org/930039 - it looks like 0.22.2 vs 0.22.4 breaks ABI?
EDIT: quoting some details from the Gentoo bug...
libabigail's abidiff output is:
$ abidiff /var/tmp/portage/dev-libs/tree-sitter-0.22.{2,4}/image/usr/lib64/libtree-sitter.so.0.22 --fail-no-debug-info --debug-info-dir1 /var/tmp/portage/dev-libs/tree-sitter-0.22.2/image/usr/lib/debug --debug-info-dir2 /var/tmp/portage/dev-libs/tree-sitter-0.22.4/image/usr/lib/debug
Functions changes summary: 0 Removed, 4 Changed (45 filtered out), 0 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable
4 functions with some indirect sub-type change:
[C] 'function const size_t* ts_parser_cancellation_flag(const TSParser*)' at parser.c:1921:1 has some indirect sub-type changes:
parameter 1 of type 'const TSParser*' has sub-type changes:
in pointed to type 'const TSParser':
in unqualified underlying type 'typedef TSParser' at api.h:45:1:
underlying type 'struct TSParser' at parser.c:90:1 changed:
type size hasn't changed
1 data member insertion:
'bool has_scanner_error', at offset 11616 (in bits) at parser.c:113:1
[C] 'function void ts_query_cursor_delete(TSQueryCursor*)' at query.c:2986:1 has some indirect sub-type changes:
parameter 1 of type 'TSQueryCursor*' has sub-type changes:
in pointed to type 'typedef TSQueryCursor' at api.h:48:1:
underlying type 'struct TSQueryCursor' at query.c:301:1 changed:
type size changed from 1152 to 1216 (in bits)
15 data member changes:
type of 'TSTreeCursor cursor' changed:
underlying type 'struct TSTreeCursor' at api.h:105:1 changed:
type size changed from 192 to 256 (in bits)
1 data member change:
type of 'uint32_t context[2]' changed:
type name changed from 'uint32_t[2]' to 'uint32_t[3]'
array type size changed from 64 to 96
array type subrange 1 changed length from 2 to 3
'struct {QueryState* contents; uint32_t size; uint32_t capacity;} states' offset changed from 256 to 320 (in bits) (by +64 bits)
'struct {QueryState* contents; uint32_t size; uint32_t capacity;} finished_states' offset changed from 384 to 448 (in bits) (by +64 bits)
'CaptureListPool capture_list_pool' offset changed from 512 to 576 (in bits) (by +64 bits)
'uint32_t depth' offset changed from 832 to 896 (in bits) (by +64 bits)
'uint32_t max_start_depth' offset changed from 864 to 928 (in bits) (by +64 bits)
'uint32_t start_byte' offset changed from 896 to 960 (in bits) (by +64 bits)
'uint32_t end_byte' offset changed from 928 to 992 (in bits) (by +64 bits)
'TSPoint start_point' offset changed from 960 to 1024 (in bits) (by +64 bits)
'TSPoint end_point' offset changed from 1024 to 1088 (in bits) (by +64 bits)
'uint32_t next_state_id' offset changed from 1088 to 1152 (in bits) (by +64 bits)
'bool on_visible_node' offset changed from 1120 to 1184 (in bits) (by +64 bits)
'bool ascending' offset changed from 1128 to 1192 (in bits) (by +64 bits)
'bool halted' offset changed from 1136 to 1200 (in bits) (by +64 bits)
'bool did_exceed_match_limit' offset changed from 1144 to 1208 (in bits) (by +64 bits)
[C] 'function TSTreeCursor ts_tree_cursor_copy(const TSTreeCursor*)' at tree_cursor.c:695:1 has some indirect sub-type changes:
[C] 'function TSTreeCursor ts_tree_cursor_new(TSNode)' at tree_cursor.c:153:1 has some indirect sub-type changes:
I've filed a bug upstream with libabigail after discussion on IRC wrt why it didn't flag it as a breaking change (https://sourceware.org/bugzilla/show_bug.cgi?id=31642).
Truncated backtrace (full at https://bugs.gentoo.org/930039#c2) from our pkgcheck tool which uses tree-sitter-bash:
tests/checks/test_codingstyle.py::TestStaticSrcUri::test_no_report[${P}]
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff49a3726 in ts_tree_cursor_init (self=0x555555650008, node=...) at lib/src/tree_cursor.c:167
167 array_push(&self->stack, ((TreeCursorEntry) {
(gdb)
#0 0x00007ffff49a3726 in ts_tree_cursor_init (self=0x555555650008, node=...) at lib/src/tree_cursor.c:167
#1 ts_tree_cursor_reset (_self=0x555555650008, node=...) at lib/src/tree_cursor.c:160
#2 ts_query_cursor_exec (self=0x555555650000, query=0x555555d8f3f0, node=...) at lib/src/query.c:3045
#3 0x00007ffff529dce0 in query_captures (self=0x7ffff2618210, args=(<tree_sitter.Node at remote 0x7ffff2180b70>,), kwargs=0x0) at tree_sitter/binding.c:2112
#4 0x00007ffff7b0c603 in method_vectorcall_VARARGS_KEYWORDS (func=<method_descriptor at remote 0x7ffff260f740>, args=0x7ffff21f8a98, nargsf=<optimized out>, kwnames=<optimized out>) at Objects/descrobject.c:344
#5 0x00007ffff7af816e in _PyObject_VectorcallTstate (tstate=0x555555576f90, callable=<method_descriptor at remote 0x7ffff260f740>, args=0x7ffff21f8a98, nargsf=<optimized out>, kwnames=0x0) at ./Include/cpython/abstract.h:114
[...]
Valgrind output:
tests/checks/test_codingstyle.py::TestStaticSrcUri::test_no_report[${P}] ==602430== Invalid write of size 8
==602430== at 0xDC597D4: ts_query_cursor_set_byte_range (query.c:3064)
==602430== by 0xD609CA2: query_captures (binding.c:2110)
==602430== by 0x49A2602: method_vectorcall_VARARGS_KEYWORDS (descrobject.c:344)
==602430== by 0x498E16D: UnknownInlinedFun (abstract.h:114)
==602430== by 0x498E16D: UnknownInlinedFun (abstract.h:123)
==602430== by 0x498E16D: UnknownInlinedFun (ceval.c:5893)
==602430== by 0x498E16D: _PyEval_EvalFrameDefault (ceval.c:4198)
==602430== by 0x49D4AE9: UnknownInlinedFun (pycore_ceval.h:46)
==602430== by 0x49D4AE9: UnknownInlinedFun (genobject.c:213)
==602430== by 0x49D4AE9: gen_iternext (genobject.c:580)
==602430== by 0x498E39F: _PyEval_EvalFrameDefault (ceval.c:4001)
==602430== by 0x49D4E02: UnknownInlinedFun (pycore_ceval.h:46)
==602430== by 0x49D4E02: gen_send_ex2 (genobject.c:213)
==602430== by 0x4990580: _PyEval_EvalFrameDefault (ceval.c:2586)
==602430== by 0x49D4AE9: UnknownInlinedFun (pycore_ceval.h:46)
==602430== by 0x49D4AE9: UnknownInlinedFun (genobject.c:213)
==602430== by 0x49D4AE9: gen_iternext (genobject.c:580)
==602430== by 0x49DCFC4: list_extend (listobject.c:960)
==602430== by 0x49AC14C: method_vectorcall_O (descrobject.c:460)
==602430== by 0x498E16D: UnknownInlinedFun (abstract.h:114)
==602430== by 0x498E16D: UnknownInlinedFun (abstract.h:123)
==602430== by 0x498E16D: UnknownInlinedFun (ceval.c:5893)
==602430== by 0x498E16D: _PyEval_EvalFrameDefault (ceval.c:4198)
==602430== Address 0xa430078 is 24 bytes after a block of size 944 in arena "client"
==602430==
valgrind: m_mallocfree.c:304 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1008, hi = 18446744069414584320.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.
```
I suspect we've hit the same thing in Gentoo as https://bugs.gentoo.org/930039 - it looks like 0.22.2 vs 0.22.4 breaks ABI?
tree-sitter 0.22.4 also breaks things for me (crashes emacs) vs 0.22.2 works fine.
Been running into this on two different machines; opening any file that tries to use tree-sitter to handle syntax highlighting causes Emacs to crash with the same stack trace posted above.
could anyone attach a debugger to emacs and help give some insight into the crash?
The public TSTreeCursor struct has changed, so I do think that will change the ABI.
Just so I understand - on Arch Linux, Emacs uses tree-sitter via a dynamic library? Is the Arch package based on Git tags in the Tree-sitter repository? So if we push a tag to this repository, emacs users on Arch just start using the new version, based on the fact that it is a patch release?
I think the only way Emacs can use tree-sitter is via dynamic library, I'm not sure it can be statically linked. At least, I can't find any information from some quick searching on how to have Emacs statically link dependencies.
The Arch package I'm using is extra/tree-sitter, and the PKGBUILD file in the package repository seems to indicate it checks out this repo using a specific tag ( currently 0.22.5 for the latest ). I think the package maintainer has to update the PKGBUILD file to point it to a new version. I'm not sure how the package is updated, but it does seem to be kept up-to-date with the latest version in this repository.
That makes sense. Does the Emacs package specify a Semver range of Tree-sitter library that it is compatible with?
Here is the backtrace from gdb on Arch Linux with emacs 29.3-2 and tree-sitter 0.22.5-1:
#0 __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1 0x00007ffff3aab393 in __pthread_kill_internal (signo=6, threadid=<optimized out>) at pthread_kill.c:78
#2 0x00007ffff3a5a6c8 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3 0x00007ffff3a424b8 in __GI_abort () at abort.c:79
#4 0x00007ffff3a43395 in __libc_message_impl (fmt=fmt@entry=0x7ffff3bbb161 "*** %s ***: terminated\n") at ../sysdeps/posix/libc_fatal.c:132
#5 0x00007ffff3b3273b in __GI___fortify_fail (msg=msg@entry=0x7ffff3bbb179 "stack smashing detected") at fortify_fail.c:24
#6 0x00007ffff3b33a56 in __stack_chk_fail () at stack_chk_fail.c:24
#7 0x000055555579e063 in Ftreesit_subtree_stat (node=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/treesit.c:3498
#8 0x00007fffe61d3832 in F747265657369742d666f6e742d6c6f636b2d666f6e746966792d726567696f6e_treesit_font_lock_fontify_region_0 () at /usr/lib/emacs/29.3/native-lisp/29.3-4b59e32f/treesit-37439c61-97df641d.eln
#9 0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8000) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#10 0x00007fffef16cf42 in F666f6e742d6c6f636b2d666f6e746966792d73796e746163746963616c6c792d726567696f6e_font_lock_fontify_syntactically_region_0 ()
at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#11 0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8170) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#12 0x00007fffef16abef in F666f6e742d6c6f636b2d64656661756c742d666f6e746966792d726567696f6e_font_lock_default_fontify_region_0 ()
at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#13 0x000055555570a61e in Ffuncall (nargs=4, args=0x7fffffff8220) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#14 0x00007fffef169873 in F666f6e742d6c6f636b2d666f6e746966792d726567696f6e_font_lock_fontify_region_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/font-lock-895216f6-1f3b244f.eln
#15 0x000055555575b8be in exec_byte_code (fun=0x2cd73, args_template=4294967295, nargs=2, args=0x7fffee400040) at /usr/src/debug/emacs/emacs-29.3-wayland/src/bytecode.c:809
#16 0x000055555570a61e in Ffuncall (nargs=2, args=args@entry=0x7fffffff8480) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#17 0x000055555570ec31 in run_hook_wrapped_funcall (nargs=<optimized out>, args=0x7fffffff8480) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2777
#18 0x000055555570967c in run_hook_with_args (nargs=2, args=0x7fffffff8480, funcall=0x55555570ec10 <run_hook_wrapped_funcall>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2858
#19 0x00007fffeedcec88 in F6a69742d6c6f636b2d2d72756e2d66756e6374696f6e73_jit_lock__run_functions_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#20 0x000055555570a61e in Ffuncall (nargs=3, args=0x7fffffff85a0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#21 0x00007fffeedcf59a in F6a69742d6c6f636b2d666f6e746966792d6e6f77_jit_lock_fontify_now_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#22 0x000055555570a61e in Ffuncall (nargs=3, args=0x7fffffff86b0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#23 0x00007fffeedce98f in F6a69742d6c6f636b2d66756e6374696f6e_jit_lock_function_0 () at /usr/bin/../lib/emacs/29.3/native-lisp/29.3-4b59e32f/preloaded/jit-lock-8a988e43-a9956d8b.eln
#24 0x000055555570a61e in Ffuncall (nargs=nargs@entry=2, args=args@entry=0x7fffffff8770) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:2999
#25 0x00005555557f28d1 in internal_condition_case_n.constprop.0
(bfun=0x55555570a520 <Ffuncall>, nargs=nargs@entry=2, args=args@entry=0x7fffffff8770, hfun=hfun@entry=0x5555555a1790 <safe_eval_handler>, handlers=0x30)
at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1558
#26 0x00005555555a190b in safe__call (inhibit_quit=inhibit_quit@entry=false, nargs=2, func=<optimized out>, ap=ap@entry=0x7fffffff87d0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3059
#27 0x00005555555a7a10 in safe_call (nargs=<optimized out>, func=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3074
#28 0x00005555555abd96 in safe_call1 (arg=0x6, fn=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3085
#29 handle_fontified_prop (it=0x7fffffff89c0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:4601
#30 0x00005555555b1a76 in handle_stop (it=0x7fffffff89c0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:4134
#31 0x00005555555a8976 in start_display (it=0x7fffffff89c0, w=0x55555653c8e8, pos=...) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:3751
#32 0x00005555555caaf9 in try_window (window=window@entry=0x55555653c8ed, pos=..., flags=flags@entry=1) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:20748
#33 0x00005555555d2a02 in redisplay_window (window=<optimized out>, just_this_one_p=<optimized out>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:20140
#34 0x00005555555c34c3 in redisplay_window_0 (window=window@entry=0x55555653c8ed) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17623
#35 0x0000555555707aec in internal_condition_case_1 (bfun=0x5555555c3490 <redisplay_window_0>, arg=0x55555653c8ed, handlers=<optimized out>, hfun=0x5555555c34d0 <redisplay_window_error>)
at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1498
#36 0x00005555555c3418 in redisplay_windows (window=0x55555653c8ed) at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17592
#37 0x00005555555c715c in redisplay_internal () at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:17042
#38 0x0000555555679d5e in redisplay () at /usr/src/debug/emacs/emacs-29.3-wayland/src/xdisp.c:16225
#39 read_char (commandflag=1, map=0x5555585f1d23, prev_event=0x0, used_mouse_menu=0x7fffffffe0db, end_time=0x0) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:2636
#40 0x0000555555682ee1 in read_key_sequence (keybuf=<optimized out>, prompt=0x0, dont_downcase_last=<optimized out>, can_return_switch_frame=true, fix_current_buffer=true, prevent_redisplay=false) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:10084
#41 0x000055555567282f in command_loop_1 () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1384
#42 0x0000555555707a57 in internal_condition_case (bfun=0x5555556725d0 <command_loop_1>, handlers=<optimized out>, hfun=0x555555670bd0 <cmd_error>) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1474
#43 0x000055555566ddae in command_loop_2 (handlers=handlers@entry=0x90) at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1133
#44 0x000055555570799a in internal_catch (tag=<optimized out>, func=0x55555566dd80 <command_loop_2>, arg=0x90) at /usr/src/debug/emacs/emacs-29.3-wayland/src/eval.c:1197
#45 0x00005555556710d9 in command_loop () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:1111
#46 0x000055555580740b in recursive_edit_1.isra.0 () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:720
#47 0x00005555556714b1 in Frecursive_edit () at /usr/src/debug/emacs/emacs-29.3-wayland/src/keyboard.c:803
#48 0x0000555555586d19 in main (argc=<optimized out>, argv=0x7fffffffe748) at /usr/src/debug/emacs/emacs-29.3-wayland/src/emacs.c:2521
The emacs build file doesn't seem to set the dependency package version.
The Tree-sitter C library's ABI has indeed changed (due to the TSQueryCursor struct field addition). I think that the Arch Emacs package should probably pin a specific version of the Tree-sitter package. I really don't want Emacs users to be broken by changes to Tree-sitter. Does anyone know the maintainers of the Arch packages?
In this repo, we try to not make breaking changes unnecessarily, but the library is pre 1.0, in its Semver versioning, and even once it goes 1.0, it's going to be easier to maintain API stability than full-on ABI stability.
Breaking ABI compatibility is not an issue at all, especially pre-1.0.
It would make things easier for consumers if the library's soname was bumped when ABI changed though. Having a different soname allows for the incompatible versions to be installed side by side, and is a signal that consumers need to be rebuilt when it changes.
Good point @MatthewGentoo - Do you know if there is a good tool for automatically detecting ABI changes in a header file? It would be cool to have our version bump script automatically bump the soname when warranted, before creating a new tag.
@maxbrunsfeld The output in #3296 (comment) is from abidiff (from libabigail) which is pretty good. Just in this instance, it didn't seem to catch the problem as an explicit breaking change (it noted diffs. but not that it was breaking). I've reported that upstream - who are great. There is also https://github.com/lvc/abi-compliance-checker.
Breaking ABI compatibility is not an issue at all, especially pre-1.0.
It is an issue when there's no such indication.
It might be a good idea to revert 09d2b23 and reserve it for ABI 15.
It would be cool to have our version bump script automatically bump the soname when warranted
The soname matches the version minus the patch. (0.22)
Downstream emacs issue: https://gitlab.archlinux.org/archlinux/packaging/packages/emacs/-/issues/2
For anyone wanting to downgrade on Arch, I've been able to get Emacs non-crashing again with sudo pacman -U file:///var/cache/pacman/pkg/tree-sitter-0.22.2-1-x86_64.pkg.tar.zst [1].
You can also try rebuilding emacs and see if that fixes it.
That didn't work for me, but hopefully does for others.
If you don't have the old version of tree-sitter locally, you can download it from the Arch Linux archive at https://archive.archlinux.org/packages/t/tree-sitter/tree-sitter-0.22.2-1-x86_64.pkg.tar.zst
Downgrading to this version fixes it for me, for now.
There is also https://github.com/lvc/abi-compliance-checker.
That one catches the breaking change and produces a nice HTML report.
Unfortunately, we can't use the HTML report in CI (except as an artifact).
We might be able to use the XML report with some post-processing though.
abi-dumper /tmp/ts/0.22.2/lib/libtree-sitter.so -vnum 0.22.2 -o 0222.dump \
-dir -public-headers "$PWD/lib/include" -include-paths "$PWD/lib/src;lib/include"
abi-dumper /tmp/ts/0.22.5/lib/libtree-sitter.so -vnum 0.22.5 -o 0225.dump \
-dir -public-headers "$PWD/lib/include" -include-paths "$PWD/lib/src;$PWD/lib/include"
abi-compliance-checker -abi -l tree-sitter -old 0222.dump -new 0225.dump -check-private-abi # -report-format xmlHowever, it does not include line/column numbers in the report like abidiff does.
It sounds like in the future, we should bump the library's minor version when changing any structs in api.h, because with our current strategy, the soname is based on the minor version.
To be precise: the soname is currently only the major version but the file extension also includes the minor so that multiple versions can be installed side-by-side. I would not mind changing the soname (or extension) to an arbitrary number but we will need a workflow that checks for breaking ABI changes so we know when to bump it.
I'm encountering the same issue with Emacs 29.3 after upgrading to Fedora 40. It comes with tree-sitter 0.22.5.
Fortunately, I still have a version of Emacs 30 that I compiled on Fedora 39 previously, and it still works.
@adonig you can downgrade to the previous version of libtree-sitter via sudo dnf install libtree-sitter-0.22.2-1.fc40 and add excludepkgs=libtree-sitter to /etc/dnf/dnf.conf until the issue is resolved. Works for me on Fedora 40 with Emacs 29.3.
Same issue here. Don't know why this critical issue hasn't been fixed in last week.
It will not be fixed on this side since reverting the change would introduce a regression.
Either emacs has to adapt to the breaking change or you have to downgrade tree-sitter.
(Rebuilding emacs with the upgraded library may or may not fix it.)
Same issue here. Don't know why this critical issue hasn't been fixed in last week.
I think the update to 0.22.5 went to stable only yesterday: https://bodhi.fedoraproject.org/updates/FEDORA-2024-e22c159068
I think the update to 0.22.5 went to stable only yesterday
The issue was introduced in 0.22.3
0.22.3 was never in the fedora repos. The last version in fc40 was 0.22.2 from three weeks ago: https://bodhi.fedoraproject.org/updates/FEDORA-2024-4b1211e6ae
It will not be fixed on this side since reverting the change would introduce a regression. Either emacs has to adapt to the breaking change or you have to downgrade tree-sitter. (Rebuilding emacs with the upgraded library may or may not fix it.)
Just so that this is clear to me: The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?
Rebuilding emacs with the upgraded library may or may not fix it
Arch users may be interested to know that rebuilding emacs 29.3-2 with tree-sitter 0.22.5-1 did not fix the issue for me.
It sounds like in the future, we should bump the library's minor version when changing any structs in
api.h, because with our current strategy, the soname is based on the minor version.
My understanding is that if the ABI of, say, libexample.so.0.22 changes, it’s not the 22 that’s expected to be bumped to 23, but rather the 0.22 that’s expected to be bumped to 1.0.
My understanding is that if the ABI of, say,
libexample.so.0.22changes, it’s not the 22 that’s expected to be bumped to 23, but rather the 0.22 that’s expected to be bumped to 1.0.
That's correct, I've written about this in the past for those who want the gory details. @maxbrunsfeld for us, I think we have to commit to not using the semver version in our library names, and to follow libtool-compatible rules for selecting a SONAME instead. Even though our semver version is pre-1.0, I think we need to be diligent about bumping the SONAME version whenever we introduce a breaking change to the ABI. SONAMEs are different, and don't have the same "everything goes before 1.0" rule that semver has. And it is often the case that a libary SONAME is not consistent with the package version number, regardless of whether the project uses semver.
same issue with VoidLinux and tree-sitter-0.22.5_1 and Emacs-29.3
My workaround at the moment is to downgrade tree-sitter
check available packages on disk
$ ls /var/cache/xbps/tree-sitter-0.2*
/var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps /var/cache/xbps/tree-sitter-0.22.5_1.x86_64.xbps
/var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps.sig2 /var/cache/xbps/tree-sitter-0.22.5_1.x86_64.xbps.sig2
register old package from cache
sudo xbps-rindex -a /var/cache/xbps/tree-sitter-0.20.8_1.x86_64.xbps
downgrade
sudo xbps-install -R /var/cache/xbps/ -f tree-sitter-0.20.8_1
opening emacs and I can work for example with rust/golang or any other tree-sitter based buffer,
The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?
Either that, if it works, or Emacs would have to update their code.
Any further comments should be made in distro bug trackers or Emacs itself.
The soname discussion can be moved to #3308 or a new issue.
Just so that this is clear to me: The solution to the immediate problem would be for the relevant Linux distros to issue a new Emacs package?
I really think that in order to provide a reliable end-user experience, and prevent issue like this, Emacs should just statically link a particular version of Tree-sitter.
If some Linux distros mandates that all libraries, no matter how small, must be distributed as dynamic libraries (which is just... staggeringly impractical IMO), then the Emacs package should specify a particular version of the Tree-sitter package to use - not a version range.
I think we have to commit to not using the semver version in our library names, and to follow libtool-compatible rules for selecting a SONAME instead.
I agree with this. Ideally, we should set up tooling to automatically bump the soname on breaking ABI changes. But if any Emacs package maintainer is seeing this issue - we do not yet have this tooling - this library is maintained by a small team, and may not have perfect ABI stability! Consider taking a more conservative approach to dependency versioning, so that end users don't have to deal with situations like this!
This wasn't specific to Emacs, though, it broke other tree-sitter consumers too. Distributions just need to rebuild all consumers of tree-sitter for now on new versions, which is what we're doing going forward in Gentoo.
Agree. The changes broke many tree-sitter consumers. As a basic library, the compatibility should be considered seriously.
It might be worth considering whether to build dynamic libraries at all. Would it make sense to build static libraries only until it hits 1.0 and has a stable ABI?
Go ask the distros.
We still like dynamic libraries :)
Someone distributing their application themselves (e.g. via a GitHub release) can still choose to statically link.
Dynamic libraries are fine, distros just need to ensure that the package that has tree-sitter as a shared lib dep uses the exact same version that the distro provides, but going forward we will try to be more mindful about abi breakage
Rebuilding emacs with the upgraded library may or may not fix it
Arch users may be interested to know that rebuilding emacs 29.3-2 with tree-sitter 0.22.5-1 did not fix the issue for me.
Archlinux: Seems to be fixed with emacs 29.3-3