ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 2.

[EdelineMD]

Hi @firasmidani

Thanks so much for this creating this and continuously improving it.

I am trying to run a test fit command on one plate of data. However I am running into a ValueError issue that ends the fitting process.

`ValueError: setting an array element with a sequence. The requested array would exceed the maximum number of dimension of 2.`

Fitting ER1_PM1-1
Fitting ER1_PM1-1	A1
Traceback (most recent call last):
(List of tracebacks) 

I am getting this error after running a basic fit function (both my own data and the examples data files, which produce similar errors and the tracebacks) in the amiga folder (as below) as well as using an alias for amiga.py:

python amiga.py fit -i </absolute/file/ref.txt> -t 1800 --verbose

It seems to be getting stuck at the diauxie detection (line 93, from traceback).
My understanding of the dimension limit is limited, but my input data file is Well_ID and the timecourse of OD readings. I have also tried filling in all empty well IDs and 0 values (even though a previous issue you tested showed it shouldn't affect the fit running).

Any advice would be greatly appreciated!

P.S.
if it helps - when I tried converting the ret[:,4:6] to ret[:,4] (to remove the sequence from the array, thinking it might reduce dimensions). On re-running, the maximum number of dimensions went to 1.
ret[:,4:6] = [[y1[l-1],y1[r-1]] for l,r in bounds] # growth rate at both bounds

[Firas Midani]

Thanks for the detailed issue. I just cloned the current amiga repository and ran a test example and did not have any issue. So, if possible, can you share an example file that gives you this error?

Also, if you re-run your command with the --verbose argument, this may point out the first well that prompts this error. Is there anything unique about the growth data for that particular well? Feel free to just share the data for the problematic well in addition to data for at least one other well that amiga is able to process without any errors.

[EdelineMD]

So I had to manually install all the requirements as the pip install -r requirements.txt didn't work for me and spit out ~2700 lines of error text. I've been able to run the summarize command, but not fit. I don't think there is anything particularly weird about the growth data, because when I use summarize the growth curves plots are fine and produce figure and summary files (attached).

I did run it all with the --verbose argument, and it halts on the first well with any data in both my test data (B3) as well as the first exemplar plate (A1; ER1_PM1-1.txt)

I have attached a picture of the summarize plots as well as the test data file I used to obtain them.

amiga summarize -i /Users/EdelineMD/data_AMiGA/data/3test49_1_Da.txt --verbose -t 1800

3test49_1_Da.pdf
3test49_1_Da.txt
3test49_1_Da.txt

[Firas Midani]

This is interesting. I just tested the file that you shared on AMiGA and it ran without any errors, so as you point out there does not seem to be anything weird about the growth data. I even recreated a new conda environment for this test run and I did not have any problems. So, there could be a mismatch between your AMiGA version and the python package requirements. So here are a few question that can help narrow it down:

1. Are you using the most recent version of AMiGA? If you cloned AMiGA with git, then you can run git show | head -3. The most recent version of AMiGA is from June of 2022. Is that what you have? If not, please re-download AMiGA and re-try your analysis.
2. What is your operating system?
3. Are you using Anaconda or Miniconda? Which version? Try conda --version if unsure.
4. What is your Python version? Try python --version if unsure.
5. Can you share the versions of the packages used by your python environments? If you are using conda, you can run conda list. If not, try pip list.

[EdelineMD]

Thanks for trying it out and helping me figure this out! To answer your questions:

1. I am using the most recent version, downloaded this week manually, and then just now I tried to clone it again using git (I wasn't able to do this before because I didn't have Xcode installed/updated), but neither of them are able to run the fit command still (summarize still working fine even with the version cloned from git).

I have pasted the values returned from git show | head -3

commit f5cde51
Author: Firas Said Midani [e-mail]
Date: Fri Jun 3 17:00:48 2022 -0500

2. MacOS Monterey (12.6)
3. Conda 22.11.1 but I'm running this within the Terminal itself (in an amiga venv)
4. Python 3.9.7
5. package version list
   (amiga_venv) (base) EdelineMD@EdelineMD ~ % pip list

Package Version

---

contourpy 1.0.7
cycler 0.11.0
Cython 0.29.33
decorator 5.1.1
fonttools 4.38.0
GPy 1.10.0
kiwisolver 1.4.4
matplotlib 3.6.3
numpy 1.24.1
packaging 23.0
pandas 1.5.2
paramz 0.9.5
pi 0.1.2
Pillow 9.4.0
pip 22.3.1
pyparsing 3.0.9
python-dateutil 2.8.2
pytz 2022.7.1
scipy 1.10.0
seaborn 0.12.2
setuptools 57.4.0
six 1.16.0
tabulate 0.9.0

[Firas Midani]

It seems that lots of your packages have versions much newer than recommended in the requirements.txt file. Because you were manually installing requirements, you may not have been specifying the versions for these packages. So, can you try to downgrade some of those packages as follows:

pip install numpy==1.18.1 pandas==0.25.3 matplotlib==3.1.3 scipy==1.4.1

Then, please re-analyze your data. Does it work now? If not, try to re-install all of your requirements but specify the versions based on the requirements.txt file as I do here.

[EdelineMD]

Hi @firasmidani

I did try downgrading the relevant packages, and rerunning the fit command for a test subset of data, but still get the same traceback and error message as above.

However, I have managed to get the program running using a conda command, instead of the python one, directly in my Terminal.

conda create -n amiga_venv instead of python -m venv /Users/EdelineMD/amiga_venv). After activating conda activate amiga_venv I also did conda config --addchannels conda-forge and finally conda install --file requirements.txt which is now working perfectly.

Thank you for your help troubleshooting! Hope the info above is helpful to anyone else having a similar problems.