Error when runing the python script.
jianlao8 opened this issue · comments
Hi,
I ran the script and then get the following error. I follow the suggestion to set the environment variable "export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True" at the terminal session, but it got the same error.
return self.df["dividend"][-1]
Progress |███████------------------------------------------------| 13.04% [Elapsed: 0:00:21 ETA: 0:02:24] Portfolio Val: 169,231.522024-02-25 19:00:23,447: root: ERROR: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 3.94 GiB of which 21.06 MiB is free. Including non-PyTorch memory, this process has 3.88 GiB memory in use. Of the allocated memory 3.41 GiB is allocated by PyTorch, and 13.19 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
2024-02-25 19:00:23,451: root: ERROR: Traceback (most recent call last):
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 856, in run
self._run_trading_session()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 807, in _run_trading_session
self._on_trading_iteration()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 276, in func_output
result = func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 300, in func_output
result = func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 87, in func_output
return func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 378, in _on_trading_iteration
on_trading_iteration()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/tools/decorators.py", line 62, in func_output
frame, result = call_function_get_frame(func_input, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/tools/decorators.py", line 30, in call_function_get_frame
result = func(*args, **kwargs)
File "/home/kenlao/python/Code Trading/tradingbot.py", line 50, in on_trading_iteration
probability, sentiment = self.get_sentiment()
File "/home/kenlao/python/Code Trading/tradingbot.py", line 45, in get_sentiment
probability, sentiment = estimate_sentiment(news)
File "/home/kenlao/python/Code Trading/finbert_utils.py", line 14, in estimate_sentiment
result = model(tokens["input_ids"], attention_mask=tokens["attention_mask"])[
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
outputs = self.bert(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
encoder_outputs = self.encoder(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
layer_outputs = layer_module(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
self_attention_outputs = self.attention(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
self_outputs = self.self(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 3.94 GiB of which 21.06 MiB is free. Including non-PyTorch memory, this process has 3.88 GiB memory in use. Of the allocated memory 3.41 GiB is allocated by PyTorch, and 13.19 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
Exception in thread MLTrader:
Traceback (most recent call last):
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 856, in run
self._run_trading_session()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 807, in _run_trading_session
self._on_trading_iteration()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 276, in func_output
result = func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 300, in func_output
result = func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 87, in func_output
return func_input(self, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 378, in _on_trading_iteration
on_trading_iteration()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/tools/decorators.py", line 62, in func_output
frame, result = call_function_get_frame(func_input, *args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/tools/decorators.py", line 30, in call_function_get_frame
result = func(*args, **kwargs)
File "/home/kenlao/python/Code Trading/tradingbot.py", line 50, in on_trading_iteration
probability, sentiment = self.get_sentiment()
File "/home/kenlao/python/Code Trading/tradingbot.py", line 45, in get_sentiment
probability, sentiment = estimate_sentiment(news)
File "/home/kenlao/python/Code Trading/finbert_utils.py", line 14, in estimate_sentiment
result = model(tokens["input_ids"], attention_mask=tokens["attention_mask"])[
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1564, in forward
outputs = self.bert(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 1013, in forward
encoder_outputs = self.encoder(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 607, in forward
layer_outputs = layer_module(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 497, in forward
self_attention_outputs = self.attention(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 427, in forward
self_outputs = self.self(
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl
return forward_call(*args, **kwargs)
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/transformers/models/bert/modeling_bert.py", line 325, in forward
attention_scores = torch.matmul(query_layer, key_layer.transpose(-1, -2))
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacity of 3.94 GiB of which 21.06 MiB is free. Including non-PyTorch memory, this process has 3.88 GiB memory in use. Of the allocated memory 3.41 GiB is allocated by PyTorch, and 13.19 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation. See documentation for Memory Management (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/home/kenlao/miniconda3/envs/py310/lib/python3.10/site-packages/lumibot/strategies/strategy_executor.py", line 869, in run
raise RuntimeError("Exception encountered, stopping BackTest.") from e
RuntimeError: Exception encountered, stopping BackTest.
Creating trades plot...
Creating indicators plot...
Creating tearsheet...
(py310) (base) kenlao@ken
you're getting a CUDA out-of-memory error, use a GPU server or switch to Google Colab instead will solve the issue