Error. nthreads cannot be larger than environment variable "NUMEXPR_MAX_THREADS" (64)
szf1993325 opened this issue · comments
This error appear when i finish setup, i don't how to resolve. Anyone have suggestion?
So this happens before even running?
Every time. even if i'm not input data. just write the code "respect -h". If i give input data the other error appear, like this "
Error. nthreads cannot be larger than environment variable "NUMEXPR_MAX_THREADS" (64)2023-07-05 23:34:49,825 INFO:Processing T80.fq.gz...
2023-07-05 23:35:12,501 INFO:compute_kmer_histogram finished in 13.619732141494751 seconds
2023-07-05 23:35:12,501 ERROR:Error occurred when processing /home/chrom/test/T80.fq.gz; it's skipped
Traceback (most recent call last):
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/respect_functions.py", line 247, in run_respect
parameter_estimator.set_kmer_histogram(args.threads, args.decomp)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/paramter_estimator.py", line 217, in set_kmer_histogram
self.compute_kmer_histogram(n_threads, decomp_util)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/timer.py", line 68, in wrapper_timer
return func(*args, **kwargs)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/paramter_estimator.py", line 174, in compute_kmer_histogram
profiler_output = kmer_profiler(self.input_file, self.sequence_type, self.output_name, self.tmp_dir,
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/profiling.py", line 120, in kmer_profiler
histo_stderr = check_output(["jellyfish", "histo", "-h", str(highest_histo_multiplicity), mercnt], stderr=STDOUT,
File "/home/soft/miniconda3/lib/python3.9/subprocess.py", line 420, in check_output
return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
File "/home/soft/miniconda3/lib/python3.9/subprocess.py", line 524, in run
raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['jellyfish', 'histo', '-h', '100000000', '/home/chrom/test/tmp/T80.fq.gz/T80.fq.gz.jf']' returned non-zero exit status 1.
Traceback (most recent call last):
File "/home/soft/miniconda3/bin/respect", line 33, in
sys.exit(load_entry_point('respect==1.3.0', 'console_scripts', 'respect')())
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/main.py", line 139, in main
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/respect_functions.py", line 262, in run_respect
File "/home/soft/miniconda3/lib/python3.9/multiprocessing/context.py", line 119, in Pool
return Pool(processes, initializer, initargs, maxtasksperchild,
File "/home/soft/miniconda3/lib/python3.9/multiprocessing/pool.py", line 205, in init
raise ValueError("Number of processes must be at least 1")
ValueError: Number of processes must be at least 1"
It seems you have many cores available on your machine and RESPECT is trying to use all of them but there is a limit set by the system. Can you try setting the number of requested cores to a smaller number? For example by adding --threads 8 option when running RESPECT?
This error have different when i set '--threads 8'. but i just set '--threads 2' before
Error. nthreads cannot be larger than environment variable "NUMEXPR_MAX_THREADS" (64)2023-07-05 23:56:29,115 INFO:Processing T80.fq.gz...
2023-07-05 23:58:30,303 INFO:compute_kmer_histogram finished in 112.02907919883728 seconds
2023-07-05 23:58:30,725 INFO:Starting iterations to estimate parameters of T80.fq.gz
2023-07-05 23:58:31,093 INFO:Set parameter Username
2023-07-05 23:58:31,199 INFO:estimate_genome_skim_parameters finished in 0.844153881072998 seconds
2023-07-05 23:58:31,199 ERROR:Error occurred when estimating parameters for /home/chrom/T80.fq.gz; it's skipped
Traceback (most recent call last):
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/paramter_estimator.py", line 344, in call
return self.estimate_genomic_parameters(*args, **kwargs)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/paramter_estimator.py", line 326, in estimate_genomic_parameters
self.estimate_genome_skim_parameters(spectra_number, error_norm, iterations_number, min_r1l, temperature)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/timer.py", line 68, in wrapper_timer
return func(*args, **kwargs)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/paramter_estimator.py", line 281, in estimate_genome_skim_parameters
optimizer.run_simulated_annealing(iterations_number, min_r1l, temperature)
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/optimizer.py", line 395, in run_simulated_annealing
repeat_spectra_next = self.estimate_repeat_spectra(o[1:], poisson_matrix_next[1:, :])
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/optimizer.py", line 335, in estimate_repeat_spectra
spectral_residuals = [1.0 * constrained_spectra[i] / norm(constrained_spectra[i:], ord=1) for i in range(
File "/home/soft/miniconda3/lib/python3.9/site-packages/respect-1.3.0-py3.9.egg/respect/optimizer.py", line 335, in
spectral_residuals = [1.0 * constrained_spectra[i] / norm(constrained_spectra[i:], ord=1) for i in range(
TypeError: 'NoneType' object is not subscriptable
2023-07-05 23:58:31,202 ERROR:Error occurred while trying to get estimated parameters for a sample
2023-07-05 23:58:31,209 INFO:Writing the results to the output files...
I just changed the version requirements for some of the prerequisites and made a commit to the master branch. Can you clone/download the repo again and install RESPECT into a new conda environment and see if the error is resolved?
Thank you very much. I try again, the similar error appear "Error. nthreads cannot be larger than environment variable "NUMEXPR_MAX_THREADS" (64)" (this conda enverioment version was 23.1.0), but when i run my data not other error appear. I think maybe the conda version was too newly, because when i install download package in older conda version 4.8.4, no error appear and quickly get output file. In all, i think RESPECT was a convenient and quickly software.
Okay I'm glad that it worked. I think the error is related to newer versions of pandas or numpy and I changed the requirements so that only versions that I had tested with are installed and not the latest.