Date
1 - 20 of 25
Error on start capture - on a linux machine
|
Errors on two of my machines, running commit e2e198948d35e6e3e09b0bc9e06fdc0cbd77b48e. Grateful for any clues.
These are multi-camera linux machines. If you need to update the RMS config file, you can do it now. Any changes to the config file will be read only after this script is started again or the Pi is rebooted. RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/au0004/source/RMS/RMS/StartCapture.py", line 36, in <module> from Utils.LiveViewer import LiveViewer File "/home/au0004/source/RMS/Utils/LiveViewer.py", line 13, in <module> import cv2 File "/home/au0004/vRMS/lib/python3.9/site-packages/cv2/__init__.py", line 89, in <module> bootstrap() File "/home/au0004/vRMS/lib/python3.9/site-packages/cv2/__init__.py", line 79, in bootstrap import cv2 ImportError: numpy.core.multiarray failed to import And also If you need to update the RMS config file, you can do it now. Any changes to the config file will be read only after this script is started again or the Pi is rebooted. No `name` configuration, performing automatic discovery running build_ext skipping '/home/au000u/.pyxbld/temp.linux-x86_64-cpython-39/pyrex/RMS/Routines/MorphCy.c' Cython extension (up-to-date) skipping 'RMS.Routines.MorphCy' extension (up-to-date) Traceback (most recent call last): File "/home/au000u/vRMS/lib/python3.9/site-packages/pyximport/pyximport.py", line 216, in load_module mod = imp.load_dynamic(name, so_path) File "/usr/lib/python3.9/imp.py", line 342, in load_dynamic return _load(spec) File "RMS/Routines/MorphCy.pyx", line 2, in init RMS.Routines.MorphCy import numpy as np ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.9/runpy.py", line 197, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.9/runpy.py", line 87, in _run_code exec(code, run_globals) File "/home/au000u/source/RMS/RMS/StartCapture.py", line 36, in <module> from Utils.LiveViewer import LiveViewer File "/home/au000u/source/RMS/Utils/LiveViewer.py", line 26, in <module> from RMS.Routines.Image import loadImage File "/home/au000u/source/RMS/RMS/Routines/Image.py", line 40, in <module> import RMS.Routines.MorphCy as morph File "/home/au000u/vRMS/lib/python3.9/site-packages/pyximport/pyximport.py", line 459, in load_module module = load_module(fullname, self.path, File "/home/au000u/vRMS/lib/python3.9/site-packages/pyximport/pyximport.py", line 231, in load_module raise exc.with_traceback(tb) File "/home/au000u/vRMS/lib/python3.9/site-packages/pyximport/pyximport.py", line 216, in load_module mod = imp.load_dynamic(name, so_path) File "/usr/lib/python3.9/imp.py", line 342, in load_dynamic return _load(spec) File "RMS/Routines/MorphCy.pyx", line 2, in init RMS.Routines.MorphCy import numpy as np ImportError: Building module RMS.Routines.MorphCy failed: ['ValueError: numpy.ndarray size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject\n'] |
|
|
|
Mark McIntyre
The version of numpy or python has changed, but the linked cython
(C to python) modules have not been recompiled. Try running "python setup.py install" in the RMS source folder.
If the error persists, run "pip uninstall numpy" followed by "pip
install -r requirements.txt" to remove and then reinstall numpy.
The numpy version must be 1.24 or lower. If that fails, you may
need to backup the config, then delete and reinstall RMS from
scratch. Note also that i don't think RMS is fully compatible with Python
3.9 or later, and i personally recommend sticking with 3.8 or 3.7.
There are binary incompatabilities in particular in the cython
code which seem to cause RMS to fall over. I'm running RMS on
Ubuntu 20.04 using Python 3.8 (in docker containers and on Hyper-V
containers). On 22/03/2023 12:24, David Rollinson
wrote:
Errors on two of my machines, running commit e2e198948d35e6e3e09b0bc9e06fdc0cbd77b48e. Grateful for any clues. |
|
|
|
The machine which is working is running Python 3.7.3. The machines which are not working are running 3.9.2.
|
|
|
|
pip install numpy --upgrade seems to be fixing the problem.
I end up with version 1.24.2 |
|
|
|
Mark McIntyre
good to hear it. I usually prefer to remove and reinstall, just
to make sure ! On 22/03/2023 13:06, David Rollinson
wrote:
pip install numpy --upgrade seems to be fixing the problem. |
|
|
|
Mark McIntyre
.... by the way, the risk of doing pip --upgrade is that you may
end up with a version thats not compatible with RMS for other
reasons (eg a deprecated function that RMS still uses but which
has now been remoevd from numpy). The requirements file constrains
the versions to those known to work, to avoid these issues. If it does work properly afterwards, please can you do python -V > /tmp/versions.txt and email me the output file? I can hopefully use it as a basis
to update the readme and requirements. On 22/03/2023 13:06, David Rollinson
wrote:
pip install numpy --upgrade seems to be fixing the problem. |
|
|
|
All good points - just trying to get it to run before I hit the sack!
|
|
|
|
peschman2001
I hit the same error yesterday on Ubuntu 20.04 system running three cameras.
I disabled RMS_Update on the other Linux multicamera systems until I can sort out the problem, so their captures are still working. However, even though last night's capture worked, when I try to ShowLiveStream I see
this:
$ ~/source/RMS/Scripts/MultiCamLinux/LiveStream.sh US0001
Starting RMS live stream...
RuntimeError: module compiled against API version 0x10 but this version of numpy is 0xe
Traceback (most recent call last):
File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/home/pi/vRMS/lib/python3.8/site-packages/RMS-0.1-py3.8-linux-x86_64.egg/Utils/ShowLiveStream.py", line 5, in <module>
import cv2
File "/home/pi/vRMS/lib/python3.8/site-packages/cv2/__init__.py", line 89, in <module>
bootstrap()
File "/home/pi/vRMS/lib/python3.8/site-packages/cv2/__init__.py", line 79, in bootstrap
import cv2
ImportError: numpy.core.multiarray failed to import
(vRMS) pi@US0001-02:~/source/RMS$
This morning I hit the error on a Bullseye RMS on RPi 4 with single camera I notice that the Bullseye RMS is now up to Python 3.9.2, so that will need to be downgraded.
pip install numpy --upgrade
seems to have fixed the error on the system that failed yesterday, so I will proceed with caution on the other Linux systems, and regroup on the Bullseye RMS on RPi 4.
Peter E.
From: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> on behalf of David Rollinson via groups.io <g7gpr@...>
Sent: Wednesday, March 22, 2023 7:06 AM To: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> Subject: Re: [GlobalMeteorNetwork] Error on start capture - on a linux machine pip install numpy --upgrade seems to be fixing the problem.
I end up with version 1.24.2 |
|
|
|
Mark McIntyre
This is one reason that its important to use a virtualenv. If you rely on the operating system's version of Python, you will get surprises !
toggle quoted message
Show quoted text
On 22/03/2023 14:30, peschman2001 wrote:
the Bullseye RMS is now up to Python 3.9.2, so that will need to be downgraded. |
|
|
|
peschman2001
This Bullseye RMS is on a RPi4. It is part of the effort to roll out a Bullseye RMS to replace Buster RMS as a distributed image for RPi.
Peter E.
From: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> on behalf of Mark McIntyre via groups.io <mark.jm.mcintyre@...>
Sent: Wednesday, March 22, 2023 8:32 AM To: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> Subject: Re: [GlobalMeteorNetwork] Error on start capture - on a linux machine This is one reason that its important to use a virtualenv. If you rely
on the operating system's version of Python, you will get surprises ! On 22/03/2023 14:30, peschman2001 wrote: > the Bullseye RMS is now up to Python 3.9.2, so that will need to be > downgraded. |
|
|
|
Yesterday I limited the version of numpy: numpy>=1.21.0,<1.22.0 Newer versions removed np.bool and the imreg_dft library doesn't work. We should look into either forking imreg_dft and fixing this, then changing the install to use the github repository, or use an already fixed community branch. Perhaps these systems were running a newer version which would blow up either way when it ran the imreg_dft library. Cheers, Denis On Wed, Mar 22, 2023 at 10:35 AM peschman2001 <eschman@...> wrote:
|
|
|
|
peschman2001
Denis,
At the bottom of a private msg to you yesterday, I said I tried this:
cd ~/source/RMS
pip uninstall numpy
pip install -r requirements.txt
and this did not work to fix us0008-09.
Now that I've done
pip install numpy --upgrade
StartCapture starts with no errors. I will not know until tomorrow morning if all processing works or not. Is there something I could do to test things now?
$ pip install numpy --upgrade
Requirement already satisfied: numpy in /home/pi/vRMS/lib/python3.8/site-packages (1.21.6)
Collecting numpy
Downloading numpy-1.24.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.3 MB)
???????????????????????????????????????? 17.3/17.3 MB 9.5 MB/s eta 0:00:00
Installing collected packages: numpy
Attempting uninstall: numpy
Found existing installation: numpy 1.21.6
Uninstalling numpy-1.21.6:
Successfully uninstalled numpy-1.21.6
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
rms 0.1 requires numpy<1.22.0,>=1.21.0, but you have numpy 1.24.2 which is incompatible.
Successfully installed numpy-1.24.2
___
Peter E.
From: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> on behalf of Denis Vida via groups.io <denis.vida@...>
Sent: Wednesday, March 22, 2023 8:44 AM To: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> Subject: Re: [GlobalMeteorNetwork] Error on start capture - on a linux machine Yesterday I limited the version of numpy:
numpy>=1.21.0,<1.22.0
Newer versions removed np.bool and the imreg_dft library doesn't work. We should look into either forking imreg_dft and fixing this, then changing the install to use the github repository, or use an already fixed community branch.
Perhaps these systems were running a newer version which would blow up either way when it ran the imreg_dft library.
Cheers,
Denis
On Wed, Mar 22, 2023 at 10:35 AM peschman2001 <eschman@...> wrote:
|
|
|
|
Mark McIntyre
you have an out of date version of RMS. The requirements file states
toggle quoted message
Show quoted text
numpy>=1.21.0,<1.24.0 ; python_version>='3.6' However, as noted in another thread, I've never tested RMS on python 3.9 so its likely it will require a different version of Numpy. As requested to Dave Rollinson can you do python -V pip list and paste the output into a file or email so that i can do some testing and validation. The risk of using a later version of Numpy is that there may be other deprecated or incompatible functions, which may not show up till later in the data processing pipe (possibly not till tomorrow morning!). However if it works, then we should document the versions and adapt the requirements file accordingly. On 22/03/2023 15:24, peschman2001 wrote:
rms 0.1 requires numpy<1.22.0,>=1.21.0, but you have numpy 1.24.2 which is incompatible. |
|
|
|
Mark McIntyre
oh, I missed this :) I think this may have other consequences elsewhere for non-Pi
users but I am pretty sure i have imreg_dft working ok here, so i
will check the versions and report back. On 22/03/2023 14:44, Denis Vida wrote:
|
|
|
|
Mark McIntyre
aha its me that has an out of date one, missing Denis change from yesterday :)
toggle quoted message
Show quoted text
On 22/03/2023 18:20, Mark McIntyre wrote:
you have an out of date version of RMS. The requirements file states |
|
|
|
peschman2001
Regarding numpy 1.24.2 (along with python 3.8.10) after running RMS_Update then
pip install numpy --upgrade
StartCapture and ShowLiveStream do work correctly. I ran a reprocessing job on
an earlier directory and all processing seems to have worked correctly. So numpy 1.24.2 might be OK.
That being said, what I'm actually rolling out to the RMS Linux systems I manage is
RMS_update
pip install numpy==1.23.4
python setup.py install
Until the requirements file is changed, these steps will have to be done after RMS_Update.
I hope this helps,
Peter E.
From: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> on behalf of Mark McIntyre via groups.io <mark.jm.mcintyre@...>
Sent: Wednesday, March 22, 2023 12:22 PM To: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> Subject: Re: [GlobalMeteorNetwork] Error on start capture - on a linux machine aha its me that has an out of date one, missing Denis change from yesterday :) On 22/03/2023 18:20, Mark McIntyre wrote: > you have an out of date version of RMS. The requirements file states > > numpy>=1.21.0,<1.24.0 ; python_version>='3.6' > > However, as noted in another thread, I've never tested RMS on python > 3.9 so its likely it will require a different version of Numpy. As > requested to Dave Rollinson can you do > > python -V > pip list > > and paste the output into a file or email so that i can do some > testing and validation. > > The risk of using a later version of Numpy is that there may be other > deprecated or incompatible functions, which may not show up till later > in the data processing pipe (possibly not till tomorrow morning!). > However if it works, then we should document the versions and adapt > the requirements file accordingly. > > On 22/03/2023 15:24, peschman2001 wrote: >> rms 0.1 requires numpy<1.22.0,>=1.21.0, but you have numpy 1.24.2 >> which is incompatible. |
|
|
|
Mark McIntyre
Thanks Pete. When you get a chance can you also run the two
commands i mentioned and send me the output? For what its worth, on Ubuntu 20.04 i am running Python 3.8.10
And all seems ok, no issues. On 22/03/2023 19:06, peschman2001
wrote:
|
|
|
|
I returned the old version limits now, the imreg_dft errors can only appear when using numpy 1.24.0 or newer. I'm not sure why it didn't work for me locally with 1.23.x versions, perhaps the install wasn't applied properly. Sorry for the mess! Cheers, Denis On Wed, Mar 22, 2023 at 4:10 PM Mark McIntyre <mark.jm.mcintyre@...> wrote:
|
|
|
|
As predicted by others, my arrangement has fallen over in the morning.
AttributeError: module 'numpy' has no attribute 'bool'. `np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: At least I got all the .fits files. |
|
|
|
peschman2001
David,
Now that RMS_Update works again, subsequent restarts should work correctly.
At least, they do for the Linux systems I' m managing.
Peter E.
From: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> on behalf of David Rollinson via groups.io <g7gpr@...>
Sent: Wednesday, March 22, 2023 4:01 PM To: main@globalmeteornetwork.groups.io <main@globalmeteornetwork.groups.io> Subject: Re: [GlobalMeteorNetwork] Error on start capture - on a linux machine [EXTERNAL] As predicted by others, my arrangement has fallen over in the morning.
AttributeError: module 'numpy' has no attribute 'bool'. `np.bool` was a deprecated alias for the builtin `bool`. To avoid this error in existing code, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at: At least I got all the .fits files. |
|
|