Always error: line 1:0 mismatched input 'Token' expecting {<EOF>,Token}
DesNevermore opened this issue · comments
ANTLR4 version 4.13.2, C++ target.
I have a Lexer and Parser rules as below.
Lexer:
Parser:
I use the command antlr4 -Dlanguage=Cpp -listener -visitor -o generated/ -package spec SpecLexer.g4 SpecParser.g4 to generate code.
Then, I have a program:
`
antlr4::ANTLRInputStream input("Test: { } ;");
spec::SpecLexer lexer(&input);
antlr4::CommonTokenStream tokens(&lexer);
tokens.fill();
for (auto token : tokens.getTokens()) {
std::cout << token->toString() << std::endl;
}
spec::SpecParser parser(&tokens);
antlr4::tree::ParseTree* tree = parser.root();
std::cout << tree2->toStringTree(&parser) << std::endl;
`
When I try to run it, it always report the error:
I cannot understand, the lexer seems to work fine, but parser cannot always match the first token.
You need to "reset" the lexer after getting the tokens. The lexer is positioned at the end of input after printing out the tokens. This is how it is done in testing the grammar/target in the grammars-v4 repo: https://github.com/antlr/grammars-v4/blob/feb9e9e082842d9cafed1e1558e9d1aad7052b87/_scripts/templates/Cpp/st.Test.cpp#L58-L68
You need to "reset" the lexer after getting the tokens. The lexer is positioned at the end of input after printing out the tokens. This is how it is done in testing the grammar/target in the grammars-v4 repo: https://github.com/antlr/grammars-v4/blob/feb9e9e082842d9cafed1e1558e9d1aad7052b87/_scripts/templates/Cpp/st.Test.cpp#L58-L68
Well, I did what you said as below:
antlr4::ANTLRInputStream input("Test: { } ;");
spec::SpecLexer lexer(&input);
antlr4::CommonTokenStream tokens(&lexer);
tokens.fill();
for (auto token : tokens.getTokens()) {
std::cout << token->toString() << std::endl;
}
lexer.reset();
spec::SpecParser parser(&tokens);
antlr4::tree::ParseTree* tree = parser.root();
std::cout << tree2->toStringTree(&parser) << std::endl;
But it still not works, the same error is triggered.
However, I then put the lexer grammar and parser grammar into a single grammar file, and generate the cpp code. It turns out to work fine now.
I don't know why, I guess this should be a quite serious bug.
I don't know why, I guess this should be a quite serious bug.
It's likely a bug on your part. The split grammar works fine.
$ trgen -t Cpp --force
CSharp SpecParser.g4 success 0.0335088
CSharp SpecLexer.g4 success 0.0058729
Rendering template file from Cpp/st.build.ps1 to ./Generated-Cpp/st.build.ps1
Rendering template file from Cpp/st.build.sh to ./Generated-Cpp/st.build.sh
Rendering template file from Cpp/st.clean.ps1 to ./Generated-Cpp/st.clean.ps1
Rendering template file from Cpp/st.clean.sh to ./Generated-Cpp/st.clean.sh
Rendering template file from Cpp/st.CMakeLists.txt to ./Generated-Cpp/st.CMakeLists.txt
Rendering template file from Cpp/st.ErrorListener.cpp to ./Generated-Cpp/st.ErrorListener.cpp
Rendering template file from Cpp/st.ErrorListener.h to ./Generated-Cpp/st.ErrorListener.h
Rendering template file from Cpp/st.makefile to ./Generated-Cpp/st.makefile
Rendering template file from Cpp/st.perf.sh to ./Generated-Cpp/st.perf.sh
Rendering template file from Cpp/st.readme.md to ./Generated-Cpp/st.readme.md
Rendering template file from Cpp/st.run.ps1 to ./Generated-Cpp/st.run.ps1
Rendering template file from Cpp/st.run.sh to ./Generated-Cpp/st.run.sh
Rendering template file from Cpp/st.Test.cpp to ./Generated-Cpp/st.Test.cpp
Rendering template file from Cpp/st.test.ps1 to ./Generated-Cpp/st.test.ps1
Rendering template file from Cpp/st.test.sh to ./Generated-Cpp/st.test.sh
Rendering template file from Cpp/cmake/st.antlr4-generator.cmake.in to ./Generated-Cpp/cmake/st.antlr4-generator.cmake.in
Rendering template file from Cpp/cmake/st.antlr4-runtime.cmake.in to ./Generated-Cpp/cmake/st.antlr4-runtime.cmake.in
Rendering template file from Cpp/cmake/st.Antlr4Package.md to ./Generated-Cpp/cmake/st.Antlr4Package.md
Rendering template file from Cpp/cmake/st.ExternalAntlr4Cpp.cmake to ./Generated-Cpp/cmake/st.ExternalAntlr4Cpp.cmake
Rendering template file from Cpp/cmake/st.FindANTLR.cmake to ./Generated-Cpp/cmake/st.FindANTLR.cmake
Rendering template file from Cpp/cmake/st.README.md to ./Generated-Cpp/cmake/st.README.md
Copying source file from C:/msys64/home/Kenne/so/78836059/SpecParser.g4 to ./Generated-Cpp/SpecParser.g4
Copying source file from C:/msys64/home/Kenne/so/78836059/SpecLexer.g4 to ./Generated-Cpp/SpecLexer.g4
08/06-08:00:09 ~/so/78836059
$ cd Generated-Cpp/
08/06-08:00:11 ~/so/78836059/Generated-Cpp
$ make
bash build.sh
CMake Warning (dev) in CMakeLists.txt:
No project() command is present. The top-level CMakeLists.txt file must
contain a literal, direct call to the project() command. Add a line of
code such as
project(ProjectName)
near the top of the file, but after cmake_minimum_required().
CMake is pretending there is a "project(Project)" command on the first
line.
This warning is for project developers. Use -Wno-dev to suppress it.
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22635.
-- The C compiler identification is MSVC 19.40.33812.0
-- The CXX compiler identification is MSVC 19.40.33812.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - not found
-- Found Threads: TRUE
CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/ExternalProject.cmake:2254 (message):
Policy CMP0114 is not set: ExternalProject step targets fully adopt their
steps. Run "cmake --help-policy CMP0114" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
ExternalProject target 'antlr4_runtime' would depend on the targets for
step(s) 'build_static' under policy CMP0114, but this is being left out for
compatibility since the policy is not set.
Call Stack (most recent call first):
cmake/ExternalAntlr4Cpp.cmake:129 (ExternalProject_Add_StepTargets)
CMakeLists.txt:18 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/ExternalProject.cmake:2254 (message):
Policy CMP0114 is not set: ExternalProject step targets fully adopt their
steps. Run "cmake --help-policy CMP0114" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
ExternalProject target 'antlr4_runtime' would depend on the targets for
step(s) 'build_shared' under policy CMP0114, but this is being left out for
compatibility since the policy is not set.
Call Stack (most recent call first):
cmake/ExternalAntlr4Cpp.cmake:146 (ExternalProject_Add_StepTargets)
CMakeLists.txt:18 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: C:/msys64/home/Kenne/so/78836059/Generated-Cpp/build
MSBuild version 17.10.4+10fbfbf2e for .NET Framework
Checking Build System
Creating directories for 'antlr4_runtime'
Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt
Performing download step (git clone) for 'antlr4_runtime'
-- Avoiding repeated git clone, stamp file is up to date: 'c:/temp/antlr4_runtime/src/antlr4_runtime-stamp/antlr4_runtime-gitclone
-lastrun.txt'
Performing update step for 'antlr4_runtime'
No patch step for 'antlr4_runtime'
Performing configure step for 'antlr4_runtime'
loading initial cache file c:/temp/antlr4_runtime/tmp/antlr4_runtime-cache-Release.cmake
CMake Warning at CMakeLists.txt:12 (message):
-- Building without demo. To enable demo build use: -DWITH_DEMO=True
Build type not set, falling back to Release mode.
To specify build type use:
-DCMAKE_BUILD_TYPE=<mode> where <mode> is Debug or Release.
-- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22635.
CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/FetchContent.cmake:1267 (message):
The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is
not set. The policy's OLD behavior will be used. When using a URL
download, the timestamps of extracted files should preferably be that of
the time of extraction, otherwise code that depends on the extracted
contents might not be rebuilt if the URL changes. The OLD behavior
preserves the timestamps from the archive instead, but this is usually not
what you want. Update your project to the NEW behavior or specify the
DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this
robustness issue.
Call Stack (most recent call first):
runtime/CMakeLists.txt:63 (FetchContent_Declare)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Configuring done
-- Generating done
-- Build files have been written to: C:/Temp/antlr4_runtime/src/antlr4_runtime/runtime/Cpp
No build step for 'antlr4_runtime'
No install step for 'antlr4_runtime'
Completed 'antlr4_runtime'
Performing build_static step for 'antlr4_runtime'
MSBuild version 17.10.4+10fbfbf2e for .NET Framework
antlr4_static.vcxproj -> C:\Temp\antlr4_runtime\src\antlr4_runtime\runtime\Cpp\runtime\Release\antlr4-runtime-static.lib
Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt
Building SpecLexer with ANTLR 4.13.1
Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt
Building SpecParser with ANTLR 4.13.1
Test.cpp
C:\msys64\home\Kenne\so\78836059\Generated-Cpp\Test.cpp(18,19): warning C4244: 'initializing': conversion from 'uint64_t' to 'long',
possible loss of data [C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Test.vcxproj]
C:\msys64\home\Kenne\so\78836059\Generated-Cpp\Test.cpp(36,19): warning C4244: 'initializing': conversion from 'uint64_t' to 'long',
possible loss of data [C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Test.vcxproj]
ErrorListener.cpp
SpecLexer.cpp
SpecParser.cpp
Generating Code...
Test.vcxproj -> C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Release\Test.exe
Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt
08/06-08:00:35 ~/so/78836059/Generated-Cpp
$ ls
build/ build.sh clean.sh CMakeLists.txt ErrorListener.h perf.sh run.ps1 SpecLexer.g4 Test.cpp test.sh
build.ps1 clean.ps1 cmake/ ErrorListener.cpp makefile readme.md run.sh SpecParser.g4 test.ps1
08/06-08:00:46 ~/so/78836059/Generated-Cpp
$ ./build/
antlr4cpp_generated_src/ CMakeFiles/ Release/ Test.dir/ x64/
08/06-08:00:46 ~/so/78836059/Generated-Cpp
$ ./build/Release/Test.exe -input 'Test: { } ;
> ^C
08/06-08:01:47 ~/so/78836059/Generated-Cpp
$ ./build/Release/Test.exe -input 'Test: { } ;' -tokens
[@-1,0:3='Test',<2>,1:0]
[@-1,4:4=':',<7>,1:4]
[@-1,5:5=' ',<1>,channel=1,1:5]
[@-1,6:6='{',<5>,1:6]
[@-1,7:7=' ',<1>,channel=1,1:7]
[@-1,8:8='}',<6>,1:8]
[@-1,9:9=' ',<1>,channel=1,1:9]
[@-1,10:10=';',<3>,1:10]
[@-1,11:10='<EOF>',<-1>,1:11]
Cpp 0 string success 2.3e-05
Total Time: 0.001567
08/06-08:01:53 ~/so/78836059/Generated-Cpp
I even took your code, added the missing #include's and main(), fixed the reference to the undefined variable test2, and tested it. It works fine.
08/06-08:37:44 ~/so/78836059/foobar
$ cat build.sh
rm -rf antlr4 src *.o *.cpp *.interp *.tokens *.h a.exe
cp test.cpp.sav test.cpp
git clone https://github.com/antlr/antlr4.git
cp -r antlr4/runtime/Cpp/runtime/src .
rm -rf antlr4
antlr4 -Dlanguage=Cpp -package spec SpecLexer.g4 SpecParser.g4
g++ -std='c++17' -DANTLR4CPP_STATIC -pthread -Isrc/ -g `find . -name '*.cpp'`
./a.exe
08/06-08:37:46 ~/so/78836059/foobar
$ bash clean.sh
08/06-08:37:55 ~/so/78836059/foobar
$ ls
build.sh clean.sh SpecLexer.g4 SpecParser.g4 test.cpp.sav
08/06-08:37:56 ~/so/78836059/foobar
$ bash build.sh
Cloning into 'antlr4'...
remote: Enumerating objects: 135097, done.
remote: Counting objects: 100% (335/335), done.
remote: Compressing objects: 100% (201/201), done.
remote: Total 135097 (delta 126), reused 253 (delta 86), pack-reused 134762
Receiving objects: 100% (135097/135097), 68.50 MiB | 23.23 MiB/s, done.
Resolving deltas: 100% (79573/79573), done.
Updating files: 100% (2279/2279), done.
./src/Vocabulary.cpp: In static member function 'static void antlr4::dfa::Vocabulary::__static_initialization_and_destruction_0(int, int)':
./src/Vocabulary.cpp:12:18: warning: 'antlr4::dfa::Vocabulary::EMPTY_VOCABULARY' is deprecated: Use the default constructor of Vocabulary instead. [-Wdeprecated-declarations]
12 | const Vocabulary Vocabulary::EMPTY_VOCABULARY;
| ^~~~~~~~~~
./src/Vocabulary.cpp:12:18: note: declared here
./src/Vocabulary.cpp: In function 'void __tcf_1()':
./src/Vocabulary.cpp:12:18: warning: 'antlr4::dfa::Vocabulary::EMPTY_VOCABULARY' is deprecated: Use the default constructor of Vocabulary instead. [-Wdeprecated-declarations]
./src/Vocabulary.cpp:12:18: note: declared here
[@0,0:3='Test',<2>,1:0]
[@1,4:4=':',<7>,1:4]
[@2,5:5=' ',<1>,channel=1,1:5]
[@3,6:6='{',<5>,1:6]
[@4,7:7=' ',<1>,channel=1,1:7]
[@5,8:8='}',<6>,1:8]
[@6,9:9=' ',<1>,channel=1,1:9]
[@7,10:10=';',<3>,1:10]
[@8,11:10='<EOF>',<-1>,1:11]
(root Test : { } ; <EOF>)
08/06-08:42:00 ~/so/78836059/foobar
$
(Env: Windows 11, MSYS2)
I don't know why, I guess this should be a quite serious bug.
It's likely a bug on your part. The split grammar works fine.
$ trgen -t Cpp --force CSharp SpecParser.g4 success 0.0335088 CSharp SpecLexer.g4 success 0.0058729 Rendering template file from Cpp/st.build.ps1 to ./Generated-Cpp/st.build.ps1 Rendering template file from Cpp/st.build.sh to ./Generated-Cpp/st.build.sh Rendering template file from Cpp/st.clean.ps1 to ./Generated-Cpp/st.clean.ps1 Rendering template file from Cpp/st.clean.sh to ./Generated-Cpp/st.clean.sh Rendering template file from Cpp/st.CMakeLists.txt to ./Generated-Cpp/st.CMakeLists.txt Rendering template file from Cpp/st.ErrorListener.cpp to ./Generated-Cpp/st.ErrorListener.cpp Rendering template file from Cpp/st.ErrorListener.h to ./Generated-Cpp/st.ErrorListener.h Rendering template file from Cpp/st.makefile to ./Generated-Cpp/st.makefile Rendering template file from Cpp/st.perf.sh to ./Generated-Cpp/st.perf.sh Rendering template file from Cpp/st.readme.md to ./Generated-Cpp/st.readme.md Rendering template file from Cpp/st.run.ps1 to ./Generated-Cpp/st.run.ps1 Rendering template file from Cpp/st.run.sh to ./Generated-Cpp/st.run.sh Rendering template file from Cpp/st.Test.cpp to ./Generated-Cpp/st.Test.cpp Rendering template file from Cpp/st.test.ps1 to ./Generated-Cpp/st.test.ps1 Rendering template file from Cpp/st.test.sh to ./Generated-Cpp/st.test.sh Rendering template file from Cpp/cmake/st.antlr4-generator.cmake.in to ./Generated-Cpp/cmake/st.antlr4-generator.cmake.in Rendering template file from Cpp/cmake/st.antlr4-runtime.cmake.in to ./Generated-Cpp/cmake/st.antlr4-runtime.cmake.in Rendering template file from Cpp/cmake/st.Antlr4Package.md to ./Generated-Cpp/cmake/st.Antlr4Package.md Rendering template file from Cpp/cmake/st.ExternalAntlr4Cpp.cmake to ./Generated-Cpp/cmake/st.ExternalAntlr4Cpp.cmake Rendering template file from Cpp/cmake/st.FindANTLR.cmake to ./Generated-Cpp/cmake/st.FindANTLR.cmake Rendering template file from Cpp/cmake/st.README.md to ./Generated-Cpp/cmake/st.README.md Copying source file from C:/msys64/home/Kenne/so/78836059/SpecParser.g4 to ./Generated-Cpp/SpecParser.g4 Copying source file from C:/msys64/home/Kenne/so/78836059/SpecLexer.g4 to ./Generated-Cpp/SpecLexer.g4 08/06-08:00:09 ~/so/78836059 $ cd Generated-Cpp/ 08/06-08:00:11 ~/so/78836059/Generated-Cpp $ make bash build.sh CMake Warning (dev) in CMakeLists.txt: No project() command is present. The top-level CMakeLists.txt file must contain a literal, direct call to the project() command. Add a line of code such as project(ProjectName) near the top of the file, but after cmake_minimum_required(). CMake is pretending there is a "project(Project)" command on the first line. This warning is for project developers. Use -Wno-dev to suppress it. -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22635. -- The C compiler identification is MSVC 19.40.33812.0 -- The CXX compiler identification is MSVC 19.40.33812.0 -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working C compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting C compile features -- Detecting C compile features - done -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Check for working CXX compiler: C:/Program Files/Microsoft Visual Studio/2022/Community/VC/Tools/MSVC/14.40.33807/bin/Hostx64/x64/cl.exe - skipped -- Detecting CXX compile features -- Detecting CXX compile features - done -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - not found -- Found Threads: TRUE CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/ExternalProject.cmake:2254 (message): Policy CMP0114 is not set: ExternalProject step targets fully adopt their steps. Run "cmake --help-policy CMP0114" for policy details. Use the cmake_policy command to set the policy and suppress this warning. ExternalProject target 'antlr4_runtime' would depend on the targets for step(s) 'build_static' under policy CMP0114, but this is being left out for compatibility since the policy is not set. Call Stack (most recent call first): cmake/ExternalAntlr4Cpp.cmake:129 (ExternalProject_Add_StepTargets) CMakeLists.txt:18 (include) This warning is for project developers. Use -Wno-dev to suppress it. CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/ExternalProject.cmake:2254 (message): Policy CMP0114 is not set: ExternalProject step targets fully adopt their steps. Run "cmake --help-policy CMP0114" for policy details. Use the cmake_policy command to set the policy and suppress this warning. ExternalProject target 'antlr4_runtime' would depend on the targets for step(s) 'build_shared' under policy CMP0114, but this is being left out for compatibility since the policy is not set. Call Stack (most recent call first): cmake/ExternalAntlr4Cpp.cmake:146 (ExternalProject_Add_StepTargets) CMakeLists.txt:18 (include) This warning is for project developers. Use -Wno-dev to suppress it. -- Configuring done -- Generating done -- Build files have been written to: C:/msys64/home/Kenne/so/78836059/Generated-Cpp/build MSBuild version 17.10.4+10fbfbf2e for .NET Framework Checking Build System Creating directories for 'antlr4_runtime' Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt Performing download step (git clone) for 'antlr4_runtime' -- Avoiding repeated git clone, stamp file is up to date: 'c:/temp/antlr4_runtime/src/antlr4_runtime-stamp/antlr4_runtime-gitclone -lastrun.txt' Performing update step for 'antlr4_runtime' No patch step for 'antlr4_runtime' Performing configure step for 'antlr4_runtime' loading initial cache file c:/temp/antlr4_runtime/tmp/antlr4_runtime-cache-Release.cmake CMake Warning at CMakeLists.txt:12 (message): -- Building without demo. To enable demo build use: -DWITH_DEMO=True Build type not set, falling back to Release mode. To specify build type use: -DCMAKE_BUILD_TYPE=<mode> where <mode> is Debug or Release. -- Selecting Windows SDK version 10.0.22621.0 to target Windows 10.0.22635. CMake Warning (dev) at C:/msys64/mingw64/share/cmake/Modules/FetchContent.cmake:1267 (message): The DOWNLOAD_EXTRACT_TIMESTAMP option was not given and policy CMP0135 is not set. The policy's OLD behavior will be used. When using a URL download, the timestamps of extracted files should preferably be that of the time of extraction, otherwise code that depends on the extracted contents might not be rebuilt if the URL changes. The OLD behavior preserves the timestamps from the archive instead, but this is usually not what you want. Update your project to the NEW behavior or specify the DOWNLOAD_EXTRACT_TIMESTAMP option with a value of true to avoid this robustness issue. Call Stack (most recent call first): runtime/CMakeLists.txt:63 (FetchContent_Declare) This warning is for project developers. Use -Wno-dev to suppress it. -- Configuring done -- Generating done -- Build files have been written to: C:/Temp/antlr4_runtime/src/antlr4_runtime/runtime/Cpp No build step for 'antlr4_runtime' No install step for 'antlr4_runtime' Completed 'antlr4_runtime' Performing build_static step for 'antlr4_runtime' MSBuild version 17.10.4+10fbfbf2e for .NET Framework antlr4_static.vcxproj -> C:\Temp\antlr4_runtime\src\antlr4_runtime\runtime\Cpp\runtime\Release\antlr4-runtime-static.lib Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt Building SpecLexer with ANTLR 4.13.1 Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt Building SpecParser with ANTLR 4.13.1 Test.cpp C:\msys64\home\Kenne\so\78836059\Generated-Cpp\Test.cpp(18,19): warning C4244: 'initializing': conversion from 'uint64_t' to 'long', possible loss of data [C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Test.vcxproj] C:\msys64\home\Kenne\so\78836059\Generated-Cpp\Test.cpp(36,19): warning C4244: 'initializing': conversion from 'uint64_t' to 'long', possible loss of data [C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Test.vcxproj] ErrorListener.cpp SpecLexer.cpp SpecParser.cpp Generating Code... Test.vcxproj -> C:\msys64\home\Kenne\so\78836059\Generated-Cpp\build\Release\Test.exe Building Custom Rule C:/msys64/home/Kenne/so/78836059/Generated-Cpp/CMakeLists.txt 08/06-08:00:35 ~/so/78836059/Generated-Cpp $ ls build/ build.sh clean.sh CMakeLists.txt ErrorListener.h perf.sh run.ps1 SpecLexer.g4 Test.cpp test.sh build.ps1 clean.ps1 cmake/ ErrorListener.cpp makefile readme.md run.sh SpecParser.g4 test.ps1 08/06-08:00:46 ~/so/78836059/Generated-Cpp $ ./build/ antlr4cpp_generated_src/ CMakeFiles/ Release/ Test.dir/ x64/ 08/06-08:00:46 ~/so/78836059/Generated-Cpp $ ./build/Release/Test.exe -input 'Test: { } ; > ^C 08/06-08:01:47 ~/so/78836059/Generated-Cpp $ ./build/Release/Test.exe -input 'Test: { } ;' -tokens [@-1,0:3='Test',<2>,1:0] [@-1,4:4=':',<7>,1:4] [@-1,5:5=' ',<1>,channel=1,1:5] [@-1,6:6='{',<5>,1:6] [@-1,7:7=' ',<1>,channel=1,1:7] [@-1,8:8='}',<6>,1:8] [@-1,9:9=' ',<1>,channel=1,1:9] [@-1,10:10=';',<3>,1:10] [@-1,11:10='<EOF>',<-1>,1:11] Cpp 0 string success 2.3e-05 Total Time: 0.001567 08/06-08:01:53 ~/so/78836059/Generated-CppI even took your code, added the missing
#include's andmain(), fixed the reference to the undefined variabletest2, and tested it. It works fine.08/06-08:37:44 ~/so/78836059/foobar $ cat build.sh rm -rf antlr4 src *.o *.cpp *.interp *.tokens *.h a.exe cp test.cpp.sav test.cpp git clone https://github.com/antlr/antlr4.git cp -r antlr4/runtime/Cpp/runtime/src . rm -rf antlr4 antlr4 -Dlanguage=Cpp -package spec SpecLexer.g4 SpecParser.g4 g++ -std='c++17' -DANTLR4CPP_STATIC -pthread -Isrc/ -g `find . -name '*.cpp'` ./a.exe 08/06-08:37:46 ~/so/78836059/foobar $ bash clean.sh 08/06-08:37:55 ~/so/78836059/foobar $ ls build.sh clean.sh SpecLexer.g4 SpecParser.g4 test.cpp.sav 08/06-08:37:56 ~/so/78836059/foobar $ bash build.sh Cloning into 'antlr4'... remote: Enumerating objects: 135097, done. remote: Counting objects: 100% (335/335), done. remote: Compressing objects: 100% (201/201), done. remote: Total 135097 (delta 126), reused 253 (delta 86), pack-reused 134762 Receiving objects: 100% (135097/135097), 68.50 MiB | 23.23 MiB/s, done. Resolving deltas: 100% (79573/79573), done. Updating files: 100% (2279/2279), done. ./src/Vocabulary.cpp: In static member function 'static void antlr4::dfa::Vocabulary::__static_initialization_and_destruction_0(int, int)': ./src/Vocabulary.cpp:12:18: warning: 'antlr4::dfa::Vocabulary::EMPTY_VOCABULARY' is deprecated: Use the default constructor of Vocabulary instead. [-Wdeprecated-declarations] 12 | const Vocabulary Vocabulary::EMPTY_VOCABULARY; | ^~~~~~~~~~ ./src/Vocabulary.cpp:12:18: note: declared here ./src/Vocabulary.cpp: In function 'void __tcf_1()': ./src/Vocabulary.cpp:12:18: warning: 'antlr4::dfa::Vocabulary::EMPTY_VOCABULARY' is deprecated: Use the default constructor of Vocabulary instead. [-Wdeprecated-declarations] ./src/Vocabulary.cpp:12:18: note: declared here [@0,0:3='Test',<2>,1:0] [@1,4:4=':',<7>,1:4] [@2,5:5=' ',<1>,channel=1,1:5] [@3,6:6='{',<5>,1:6] [@4,7:7=' ',<1>,channel=1,1:7] [@5,8:8='}',<6>,1:8] [@6,9:9=' ',<1>,channel=1,1:9] [@7,10:10=';',<3>,1:10] [@8,11:10='<EOF>',<-1>,1:11] (root Test : { } ; <EOF>) 08/06-08:42:00 ~/so/78836059/foobar $(Env: Windows 11, MSYS2)
I am using the ANTLR4 cpp runtime as a library.
First I compile the cpp-runtime without demo in the main repository (https://github.com/antlr/antlr4/tree/master/runtime/Cpp) as below:
cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/install_dir -DCMAKE_BUILD_TYPE=Debug -DWITH_DEMO=False
cmake --install
Then I dynamically link the generated libantlr4-runtime.so to another c++ project. And I use the grammar files to generate parser code at this time, then the error is triggered.
I tested the parser functions using the dynamically linked c++ project both on MacOS and Ubuntu 22.04, and the error emerged on both systems.
When I combined the lexer and parser grammar files into a single one, it turned out to be fine both on MacOS and Ubuntu 22.04.
cmake --install is not a legal command, outputting Usage: cmake --install <dir> [options].
Below is the Bash script build.sh that works on Ubuntu 20.04.6 LTS, cmake version 3.16.3, c++ (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0. The script creates the sources, gits the antlr4 repo, builds the dynamic library, links the program, and runs it, all following the commands and steps you say you are using to build your application, including linking to the dynamically-linked library for the Antlr4 runtime. The script builds the program, and it works fine.
I cannot reproduce the problem. You're going to have to fill in the details by providing a complete "build.sh" script, similar to the one I provide above, that reproduces the issue. I suggest you use the -verbose option on cmake, and show the output that the build and test produce as a .txt file.
The split grammar and program code works on Windows 11 and Ubuntu 20.04, static and dynamically linked. Here's the output of the script on Ubuntu:
out.txt