Compilation fails on Ubuntu 22.04 due to linker issues.

Question

Compilation fails on Ubuntu 22.04 due to linker issues.

lmwnshn opened this issue 2 years ago · comments

Hi, thanks for open-sourcing the code! AFAIK, the code is meant to work with newer versions of Linux ("Please run the code with a newer version of Linux and GCC"), but there are some small issues with Ubuntu 22.04.

Summary

A small number of files have multiple definitions of the same global variable. This causes linker errors.
Similarly, yacc wants yydebug to be extern.

Environment

$ lsb_release -d
Description:    Ubuntu 22.04 LTS
$ uname -r
5.15.0-39-generic
$ gcc --version
gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0
$ yacc --version
bison (GNU Bison) 3.8.2

Details

The output of make is copied below.

$ make
...
gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DYYDEBUG  -DLINUX -g -Wall  -o dsdgen s_brand.o s_customer_address.o s_call_center.o s_catalog.o s_catalog_order.o s_catalog_order_lineitem.o s_catalog_page.o s_catalog_promotional_item.o s_catalog_returns.o s_category.o s_class.o s_company.o s_customer.o s_division.o s_inventory.o s_item.o s_manager.o s_manufacturer.o s_market.o s_pline.o s_product.o s_promotion.o s_purchase.o s_reason.o s_store.o s_store_promotional_item.o s_store_returns.o s_subcategory.o s_subclass.o s_warehouse.o s_web_order.o s_web_order_lineitem.o s_web_page.o s_web_promotinal_item.o s_web_returns.o s_web_site.o s_zip_to_gmt.o w_call_center.o w_catalog_page.o w_catalog_returns.o w_catalog_sales.o w_customer_address.o w_customer.o w_customer_demographics.o w_datetbl.o w_household_demographics.o w_income_band.o w_inventory.o w_item.o w_promotion.o w_reason.o w_ship_mode.o w_store.o w_store_returns.o w_store_sales.o w_timetbl.o w_warehouse.o w_web_page.o w_web_returns.o w_web_sales.o w_web_site.o dbgen_version.o joint_distribution.o address.o build_support.o date.o decimal.o dist.o driver.o error_msg.o genrand.o join.o list.o load.o misc.o nulls.o parallel.o permute.o pricing.o print.o r_params.o StringBuffer.o tdef_functions.o tdefs.o text.o scd.o scaling.o release.o sparse.o validate.o -lm
/usr/bin/ld: s_purchase.o:/<snip>/git/dsb/code/tools/s_purchase.c:55: multiple definition of `nItemIndex'; s_catalog_order.o:/<snip>/git/dsb/code/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order.o:/<snip>/git/dsb/code/tools/s_web_order.c:56: multiple definition of `nItemIndex'; s_catalog_order.o:/<snip>/git/dsb/code/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order_lineitem.o:/<snip>/git/dsb/code/tools/s_web_order_lineitem.c:54: multiple definition of `g_s_web_order_lineitem'; s_web_order.o:/<snip>/git/dsb/code/tools/s_web_order.c:54: first defined here
/usr/bin/ld: w_catalog_page.o:/<snip>/git/dsb/code/tools/w_catalog_page.c:52: multiple definition of `g_w_catalog_page'; s_catalog_page.o:/<snip>/git/dsb/code/tools/s_catalog_page.c:51: first defined here
/usr/bin/ld: w_customer.o:/<snip>/git/dsb/code/tools/w_customer.c:59: multiple definition of `pCountyRecord'; w_customer_address.o:/<snip>/git/dsb/code/tools/w_customer_address.c:60: first defined here
/usr/bin/ld: w_customer.o:/<snip>/git/dsb/code/tools/w_customer.c:62: multiple definition of `pStateRecord'; w_customer_address.o:/<snip>/git/dsb/code/tools/w_customer_address.c:61: first defined here
/usr/bin/ld: w_household_demographics.o:/<snip>/git/dsb/code/tools/w_household_demographics.c:62: multiple definition of `pDemoRecord'; w_customer_demographics.o:/<snip>/git/dsb/code/tools/w_customer_demographics.c:54: first defined here
/usr/bin/ld: w_warehouse.o:/<snip>/git/dsb/code/tools/w_warehouse.c:54: multiple definition of `g_w_warehouse'; s_warehouse.o:/<snip>/git/dsb/code/tools/s_warehouse.c:51: first defined here
/usr/bin/ld: w_web_site.o:/<snip>/git/dsb/code/tools/w_web_site.c:60: multiple definition of `g_w_web_site'; s_web_site.o:/<snip>/git/dsb/code/tools/s_web_site.c:51: first defined here
collect2: error: ld returned 1 exit status
make: *** [makefile:233: dsdgen] Error 1

Wan Shen Lim · Answer 1 · Tue Oct 10 2023 22:03:44 GMT+0800 (China Standard Time)

Oct 2023, from @arselzer #5 (comment) ,

I also ran into the same problem when trying to run the makefile.

Using your branch at the latest commit, I was able to compile everything without errors.
In case it still helps, the environment I used was the following:

Ubuntu 22.04.3 LTS
6.2.0-33-generic kernel
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
bison (GNU Bison) 3.8.2
flex 2.6.4

Bailu Ding · Answer 2 · Fri Oct 13 2023 09:03:56 GMT+0800 (China Standard Time)

Oct 2023, from @arselzer #5 (comment) ,

I also ran into the same problem when trying to run the makefile.
Using your branch at the latest commit, I was able to compile everything without errors.
In case it still helps, the environment I used was the following:
Ubuntu 22.04.3 LTS
6.2.0-33-generic kernel
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
bison (GNU Bison) 3.8.2
flex 2.6.4

Hi Wan,

Thanks for the feedback! Just double check: have you been able to successfully build the repo now or do you still have issues in compilation?

Yibo Wang · Answer 3 · Thu Jun 20 2024 22:26:43 GMT+0800 (China Standard Time)

Hi, thanks for open-sourcing the code! AFAIK, the code is meant to work with newer versions of Linux ("Please run the code with a newer version of Linux and GCC"), but there are some small issues with Ubuntu 22.04.

Summary

A small number of files have multiple definitions of the same global variable. This causes linker errors.
Similarly, yacc wants yydebug to be extern.

Environment

$ lsb_release -d
Description:    Ubuntu 22.04 LTS
$ uname -r
5.15.0-39-generic
$ gcc --version
gcc (Ubuntu 11.2.0-19ubuntu1) 11.2.0
$ yacc --version
bison (GNU Bison) 3.8.2

Details

The output of make is copied below.

$ make
...
gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DYYDEBUG  -DLINUX -g -Wall  -o dsdgen s_brand.o s_customer_address.o s_call_center.o s_catalog.o s_catalog_order.o s_catalog_order_lineitem.o s_catalog_page.o s_catalog_promotional_item.o s_catalog_returns.o s_category.o s_class.o s_company.o s_customer.o s_division.o s_inventory.o s_item.o s_manager.o s_manufacturer.o s_market.o s_pline.o s_product.o s_promotion.o s_purchase.o s_reason.o s_store.o s_store_promotional_item.o s_store_returns.o s_subcategory.o s_subclass.o s_warehouse.o s_web_order.o s_web_order_lineitem.o s_web_page.o s_web_promotinal_item.o s_web_returns.o s_web_site.o s_zip_to_gmt.o w_call_center.o w_catalog_page.o w_catalog_returns.o w_catalog_sales.o w_customer_address.o w_customer.o w_customer_demographics.o w_datetbl.o w_household_demographics.o w_income_band.o w_inventory.o w_item.o w_promotion.o w_reason.o w_ship_mode.o w_store.o w_store_returns.o w_store_sales.o w_timetbl.o w_warehouse.o w_web_page.o w_web_returns.o w_web_sales.o w_web_site.o dbgen_version.o joint_distribution.o address.o build_support.o date.o decimal.o dist.o driver.o error_msg.o genrand.o join.o list.o load.o misc.o nulls.o parallel.o permute.o pricing.o print.o r_params.o StringBuffer.o tdef_functions.o tdefs.o text.o scd.o scaling.o release.o sparse.o validate.o -lm
/usr/bin/ld: s_purchase.o:/<snip>/git/dsb/code/tools/s_purchase.c:55: multiple definition of `nItemIndex'; s_catalog_order.o:/<snip>/git/dsb/code/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order.o:/<snip>/git/dsb/code/tools/s_web_order.c:56: multiple definition of `nItemIndex'; s_catalog_order.o:/<snip>/git/dsb/code/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order_lineitem.o:/<snip>/git/dsb/code/tools/s_web_order_lineitem.c:54: multiple definition of `g_s_web_order_lineitem'; s_web_order.o:/<snip>/git/dsb/code/tools/s_web_order.c:54: first defined here
/usr/bin/ld: w_catalog_page.o:/<snip>/git/dsb/code/tools/w_catalog_page.c:52: multiple definition of `g_w_catalog_page'; s_catalog_page.o:/<snip>/git/dsb/code/tools/s_catalog_page.c:51: first defined here
/usr/bin/ld: w_customer.o:/<snip>/git/dsb/code/tools/w_customer.c:59: multiple definition of `pCountyRecord'; w_customer_address.o:/<snip>/git/dsb/code/tools/w_customer_address.c:60: first defined here
/usr/bin/ld: w_customer.o:/<snip>/git/dsb/code/tools/w_customer.c:62: multiple definition of `pStateRecord'; w_customer_address.o:/<snip>/git/dsb/code/tools/w_customer_address.c:61: first defined here
/usr/bin/ld: w_household_demographics.o:/<snip>/git/dsb/code/tools/w_household_demographics.c:62: multiple definition of `pDemoRecord'; w_customer_demographics.o:/<snip>/git/dsb/code/tools/w_customer_demographics.c:54: first defined here
/usr/bin/ld: w_warehouse.o:/<snip>/git/dsb/code/tools/w_warehouse.c:54: multiple definition of `g_w_warehouse'; s_warehouse.o:/<snip>/git/dsb/code/tools/s_warehouse.c:51: first defined here
/usr/bin/ld: w_web_site.o:/<snip>/git/dsb/code/tools/w_web_site.c:60: multiple definition of `g_w_web_site'; s_web_site.o:/<snip>/git/dsb/code/tools/s_web_site.c:51: first defined here
collect2: error: ld returned 1 exit status
make: *** [makefile:233: dsdgen] Error 1

The same environment, the same issue.

Wan Shen Lim · Answer 4 · Sat Jun 22 2024 15:13:57 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.

Yibo Wang · Answer 5 · Sat Jun 22 2024 15:39:36 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.
@lmwnshn Excellent! Many thanks to your pr! I have just solved this problem by degrading gcc-11 to gcc-9.
gregrahn/tpcds-kit#57 (comment)

Bailu Ding · Answer 6 · Tue Jun 25 2024 04:15:25 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.

Thanks for the response!

Bailu Ding · Answer 7 · Tue Jun 25 2024 04:22:16 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.
@lmwnshn Excellent! Many thanks to your pr! I have just solved this problem by degrading gcc-11 to gcc-9.
gregrahn/tpcds-kit#57 (comment)

It is great to hear you have a workaround for this issue. Just curious, did you also try to compile / build original TPC-DS code (https://www.tpc.org/tpcds/) in the environment as well? Did it build successfully?

Yibo Wang · Answer 8 · Tue Jun 25 2024 16:44:30 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.
@lmwnshn Excellent! Many thanks to your pr! I have just solved this problem by degrading gcc-11 to gcc-9.
gregrahn/tpcds-kit#57 (comment)

It is great to hear you have a workaround for this issue. Just curious, did you also try to compile / build original TPC-DS code (https://www.tpc.org/tpcds/) in the environment as well? Did it build successfully?

I just tried to compile / build original TPC-DS code (TPC-DS 3.2.0), and I succeed in this environment:

lsb_release -d
Ubuntu 22.04.3 LTS
yacc --version
bison (GNU Bison) 3.8.2
uname -r
5.10.134-16.3.al8.x86_64
gcc --version
gcc (Ubuntu 9.5.0-1ubuntu1~22.04) 9.5.0
g++ --version
g++ (Ubuntu 9.5.0-1ubuntu1~22.04) 9.5.0

I also tried gcc (Ubuntu 10.5.0-1ubuntu1~22.04) 10.5.0 and gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, the same issue happens:

gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DYYDEBUG  -DLINUX -g -Wall  -o dsdgen s_brand.o s_customer_address.o s_call_center.o s_catalog.o s_catalog_order.o s_catalog_order_lineitem.o s_catalog_page.o s_catalog_promotional_item.o s_catalog_returns.o s_category.o s_class.o s_company.o s_customer.o s_division.o s_inventory.o s_item.o s_manager.o s_manufacturer.o s_market.o s_pline.o s_product.o s_promotion.o s_purchase.o s_reason.o s_store.o s_store_promotional_item.o s_store_returns.o s_subcategory.o s_subclass.o s_warehouse.o s_web_order.o s_web_order_lineitem.o s_web_page.o s_web_promotinal_item.o s_web_returns.o s_web_site.o s_zip_to_gmt.o w_call_center.o w_catalog_page.o w_catalog_returns.o w_catalog_sales.o w_customer_address.o w_customer.o w_customer_demographics.o w_datetbl.o w_household_demographics.o w_income_band.o w_inventory.o w_item.o w_promotion.o w_reason.o w_ship_mode.o w_store.o w_store_returns.o w_store_sales.o w_timetbl.o w_warehouse.o w_web_page.o w_web_returns.o w_web_sales.o w_web_site.o dbgen_version.o address.o build_support.o date.o decimal.o dist.o driver.o error_msg.o genrand.o join.o list.o load.o misc.o nulls.o parallel.o permute.o pricing.o print.o r_params.o StringBuffer.o tdef_functions.o tdefs.o text.o scd.o scaling.o release.o sparse.o validate.o -lm
/usr/bin/ld: s_purchase.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_purchase.c:55: multiple definition of `nItemIndex'; s_catalog_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order.c:56: multiple definition of `nItemIndex'; s_catalog_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order_lineitem.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order_lineitem.c:54: multiple definition of `g_s_web_order_lineitem'; s_web_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order.c:54: first defined here
/usr/bin/ld: w_catalog_page.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_catalog_page.c:52: multiple definition of `g_w_catalog_page'; s_catalog_page.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_page.c:51: first defined here
/usr/bin/ld: w_warehouse.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_warehouse.c:53: multiple definition of `g_w_warehouse'; s_warehouse.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_warehouse.c:51: first defined here
/usr/bin/ld: w_web_site.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_web_site.c:59: multiple definition of `g_w_web_site'; s_web_site.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_site.c:51: first defined here
collect2: error: ld returned 1 exit status
make: *** [makefile:233: dsdgen] Error 1

All in all, gcc-9 still works well, while the other versions of gcc fails to build TPC-DS.

blding · Answer 9 · Thu Jun 27 2024 04:12:44 GMT+0800 (China Standard Time)

Hi @Wangyibo321 , try merging #5 into your own branch.
@lmwnshn Excellent! Many thanks to your pr! I have just solved this problem by degrading gcc-11 to gcc-9.
gregrahn/tpcds-kit#57 (comment)

It is great to hear you have a workaround for this issue. Just curious, did you also try to compile / build original TPC-DS code (https://www.tpc.org/tpcds/) in the environment as well? Did it build successfully?

I just tried to compile / build original TPC-DS code (TPC-DS 3.2.0), and I succeed in this environment:

lsb_release -d
Ubuntu 22.04.3 LTS
yacc --version
bison (GNU Bison) 3.8.2
uname -r
5.10.134-16.3.al8.x86_64
gcc --version
gcc (Ubuntu 9.5.0-1ubuntu1~22.04) 9.5.0
g++ --version
g++ (Ubuntu 9.5.0-1ubuntu1~22.04) 9.5.0

I also tried gcc (Ubuntu 10.5.0-1ubuntu1~~22.04) 10.5.0 and gcc (Ubuntu 11.4.0-1ubuntu1~~22.04) 11.4.0, the same issue happens:

gcc -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -DYYDEBUG  -DLINUX -g -Wall  -o dsdgen s_brand.o s_customer_address.o s_call_center.o s_catalog.o s_catalog_order.o s_catalog_order_lineitem.o s_catalog_page.o s_catalog_promotional_item.o s_catalog_returns.o s_category.o s_class.o s_company.o s_customer.o s_division.o s_inventory.o s_item.o s_manager.o s_manufacturer.o s_market.o s_pline.o s_product.o s_promotion.o s_purchase.o s_reason.o s_store.o s_store_promotional_item.o s_store_returns.o s_subcategory.o s_subclass.o s_warehouse.o s_web_order.o s_web_order_lineitem.o s_web_page.o s_web_promotinal_item.o s_web_returns.o s_web_site.o s_zip_to_gmt.o w_call_center.o w_catalog_page.o w_catalog_returns.o w_catalog_sales.o w_customer_address.o w_customer.o w_customer_demographics.o w_datetbl.o w_household_demographics.o w_income_band.o w_inventory.o w_item.o w_promotion.o w_reason.o w_ship_mode.o w_store.o w_store_returns.o w_store_sales.o w_timetbl.o w_warehouse.o w_web_page.o w_web_returns.o w_web_sales.o w_web_site.o dbgen_version.o address.o build_support.o date.o decimal.o dist.o driver.o error_msg.o genrand.o join.o list.o load.o misc.o nulls.o parallel.o permute.o pricing.o print.o r_params.o StringBuffer.o tdef_functions.o tdefs.o text.o scd.o scaling.o release.o sparse.o validate.o -lm
/usr/bin/ld: s_purchase.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_purchase.c:55: multiple definition of `nItemIndex'; s_catalog_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order.c:56: multiple definition of `nItemIndex'; s_catalog_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_order.c:56: first defined here
/usr/bin/ld: s_web_order_lineitem.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order_lineitem.c:54: multiple definition of `g_s_web_order_lineitem'; s_web_order.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_order.c:54: first defined here
/usr/bin/ld: w_catalog_page.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_catalog_page.c:52: multiple definition of `g_w_catalog_page'; s_catalog_page.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_catalog_page.c:51: first defined here
/usr/bin/ld: w_warehouse.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_warehouse.c:53: multiple definition of `g_w_warehouse'; s_warehouse.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_warehouse.c:51: first defined here
/usr/bin/ld: w_web_site.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/w_web_site.c:59: multiple definition of `g_w_web_site'; s_web_site.o:/home/0ACAEA96-9CDA-4895-845C-C2C226A9B79A-TPC-DS-Tool/DSGen-software-code-3.2.0rc1/tools/s_web_site.c:51: first defined here
collect2: error: ld returned 1 exit status
make: *** [makefile:233: dsdgen] Error 1

All in all, gcc-9 still works well, while the other versions of gcc fails to build TPC-DS.

Thanks for the confirmation! The DSB code is adapted from the TPC-DS code, and it seems that the compatibility issue comes from the TPC-DS source code. I will look into this.