error while reading libtch

[luxiant]

I followed the installation guide in the readme, and also installed c++ libtorch by downloading file according to this link (https://pytorch.org/cppdocs/installing.html) and run setup-libtorch.sh.

when I tried to run my main.go, the console shows countless errors while reading libtch

"(luxiant) luxiant@localhost:~/Documents/projects/KoBERT> go run main.go
github.com/sugarme/gotch/libtch
In file included from /usr/include/torch/nn/module.h:6,
from /usr/include/torch/nn/cloneable.h:3,
from /usr/include/torch/nn.h:3,
from /usr/include/torch/all.h:15,
from /usr/include/torch/torch.h:3,
from torch_api.cpp:5:
/usr/include/torch/ordered_dict.h:13:7: error: redefinition of ‘class torch::OrderedDict<Key, Value>’
13 | class OrderedDict {
| ^~~~~~~~~~~
In file included from /usr/include/torch/csrc/jit/api/module.h:12,
from /usr/include/torch/csrc/jit/passes/fixup_trace_scope_blocks.h:3,
from torch_api.cpp:3:
/usr/include/torch/csrc/api/include/torch/ordered_dict.h:13:7: note: previous definition of ‘class torch::OrderedDict<Key, Value>’
13 | class OrderedDict {
| ^~~~~~~~~~~
/usr/include/torch/ordered_dict.h:194:32: error: redefinition of ‘class torch::OrderedDict<Key, Value>::Item’
194 | class OrderedDict<Key, Value>::Item {
| ^~~~
/usr/include/torch/csrc/api/include/torch/ordered_dict.h:194:32: note: previous definition of ‘class torch::OrderedDict<Key, Value>::Item’
194 | class OrderedDict<Key, Value>::Item {
| ^~~~
/usr/include/torch/ordered_dict.h:258:1: error: redefinition of ‘torch::OrderedDict<Key, Value>::OrderedDict(std::string)’
258 | OrderedDict<Key, Value>::OrderedDict(std::string key_description)
| ^~~~~~~~~~~~~~~~~~~~~~~
/usr/include/torch/csrc/api/include/torch/ordered_dict.h:248:1: note: ‘torch::OrderedDict<Key, Value>::OrderedDict(std::string)’ previously declared here"

...

The series of errors looks like I have missed something while installing libtorch. Can you help me?

[sugarme]

@luxiant ,

Please do:

• Read the installation guide in the README.md carefully. There are some options and steps for you to do. What options have you chosen? What steps caused the issue???
• Search for closed issues to see how people solve installation issues
• If you customize the installation to fit your own computer, please let's us know your computer detail and reason you try differently.

We don't know anything about your main.go so it's hard to guess. Perhaps, you better do

1. Follow the instruction in README.md to install

• Step1: Install libtorch for CPU or CUDA (by executing the script, you don't have to manually download anything)
• Step 2: Install gotch

2. If there's no errors after following the above, a simple example to test gotch is:

mkdir test-gotch
cd test-gotch
go mod init "github.com/YOUR_ACCOUNT/foo"
go get "github.com/sugarme/gotch@v0.7.0"
touch main.go

Then edit main.go as

package main

import (
	"fmt"

	"github.com/sugarme/gotch"
	"github.com/sugarme/gotch/ts"
)

func main() {

	x := ts.MustRandn([]int64{3, 4, 5}, gotch.Float, gotch.CPU)

	fmt.Printf("%0.3f", x)
}

Finally, run go run ., you should see something like this in your console:

go run .

(1,.,.) =
-0.680  -0.445  0.392   0.113   -2.019  
-1.062  -2.622  -1.193  -0.780  -0.560  
-1.403  -0.499  -1.426  1.095   -0.081  
-0.840  0.328   1.592   1.049   -0.014  

(2,.,.) =
0.138   -0.884  0.102   0.097   0.750   
1.239   0.009   -0.770  0.311   0.028   
-0.241  0.164   1.282   -1.514  0.962   
1.399   1.108   1.595   1.703   1.317   

(3,.,.) =
-0.092  0.748   0.305   0.935   0.243   
-0.765  -1.552  -0.242  0.384   -1.012  
-0.060  1.029   -0.082  1.520   0.849   
1.541   -0.532  -0.558  -0.547  -0.052  

[sugarme]

@luxiant,

If you read the closed issues, you will see someone has reported this issue and solved. It's because somehow your system did not pickup your option CUDA_VER=cpu or have you set it up before running setup-gotch.sh?

As by default, cgo flags in gotch are set for CUDA support when you choose to use CPU, the setup-gotch.sh script will update file cgo flags of gotch package before the linking process. The error you had was because cgo flags were still set for CUDA and linker followed such instruction and looked for cuda files in libtorch while you installed libtorch version for CPU in previous step.

A quick way to fix is manually updating file lib.go. You can do as following:

1. cd $GOPATH/pkg/mod/github.com/sugarme/gotch@v0.7.0/libtch
2. Open lib.go file with your editor of choice and update it with

package libtch

// #cgo CFLAGS: -I${SRCDIR} -O3 -Wall -Wno-unused-variable -Wno-deprecated-declarations -Wno-c++11-narrowing -g -Wno-sign-compare -Wno-unused-function
// #cgo CFLAGS: -I/usr/local/include
// #cgo CFLAGS: -D_GLIBCXX_USE_CXX11_ABI=1
// #cgo LDFLAGS: -lstdc++ -ltorch -lc10 -ltorch_cpu -L/lib64
// #cgo CXXFLAGS: -std=c++17 -I${SRCDIR} -g -O3
// #cgo CFLAGS: -I${SRCDIR}/libtorch/lib -I${SRCDIR}/libtorch/include -I${SRCDIR}/libtorch/include/torch/csrc/api/include -I${SRCDIR}/libtorch/include/torch/csrc
// #cgo LDFLAGS: -L${SRCDIR}/libtorch/lib
// #cgo CXXFLAGS: -I${SRCDIR}/libtorch/lib -I${SRCDIR}/libtorch/include -I${SRCDIR}/libtorch/include/torch/csrc/api/include -I${SRCDIR}/libtorch/include/torch/csrc
import "C"

Then try to run the setup example that I mentioned previously.
Hope that works for you. Let's me know the result. Ta.

[luxiant]

@sugarme

Excuse me for the late reply. I repeated installation but failed several times. But I tried in the different machine with docker environment on WSL ubuntu and it worked well. So I can proceed my project. I assume that the cause may be related to the conflict of the two libtorch setups, since that is the only difference compared to former trials.

By the way, is it possible to run this package on ROCM? I made it on cpu, but I haven't tried yet. I'm using AMD graphics card, so I would like to try this.

[sugarme]

@luxiant,

Thanks for your feedback and glad that you can run gotch in your box.
With regard to ROCm, gotch has not supported yet.

I am closing this issue for now.