hi ferdinand,
Just resumed trying out throttle for a bit on MacOS and ran into the
following:
$ /opt/homebrew/opt/throttle-cli/bin/throttle-server --LOGLEVEL DEBUG
Traceback (most recent call last):
File "/opt/homebrew/opt/throttle-cli/bin/throttle-server", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/cli_server.py", line 25, in main
socketpath = Path(BaseDirectory.get_runtime_dir()) / "throttle.sock"
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/xdg/BaseDirectory.py", line 141, in get_runtime_dir
return os.environ['XDG_RUNTIME_DIR']
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
File "<frozen os>", line 685, in __getitem__
KeyError: 'XDG_RUNTIME_DIR'
sure, we need a XDG_RUNTIME_DIR, somehow pyxdg's hardcoded fallback to
/tmp does not work.
Incidentally, I did some looking around trying to understand what other
projects do when XDG_* vars are undefined. It seems the platformdirs
pypi package has platform specific fallbacks and may be a more portable
alternative to pyxdg?
Setting it manually:
$ XDG_RUNTIME_DIR=/Users/mya/Library/Caches/TemporaryItems
/opt/homebrew/opt/throttle-cli/bin/throttle-server --LOGLEVEL DEBUG
Traceback (most recent call last):
File "/opt/homebrew/opt/throttle-cli/bin/throttle-server", line 8, in <module>
sys.exit(main())
^^^^^^
File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/cli_server.py", line 29, in main
start_server(socketpath, loglevel)
File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/server.py", line 53, in start_server
p_ipc.start()
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/context.py", line 289, in _Popen
return Popen(process_obj)
^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
AttributeError: Can't pickle local object 'start_server.<locals>.handleMsg'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 122, in spawn_main
exitcode = _main(fd, parent_sentinel)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 132, in _main
self = reduction.pickle.load(from_parent)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/synchronize.py", line 115, in __setstate__
self._semlock = _multiprocessing.SemLock._rebuild(*state)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory
This no longer seems related to the XDG_* vars, but to pickling. Any
suggestions?
kind regards, Maarten
Hey Maarten,
On Tue Apr 09, 2024 at 12:02, Maarten Aertsen <maarten@nlnetlabs.nl> wrote:
> hi ferdinand,
Bence :)
>> Just resumed trying out throttle for a bit on MacOS and ran into the > following:>> $ /opt/homebrew/opt/throttle-cli/bin/throttle-server --LOGLEVEL DEBUG> Traceback (most recent call last):> File "/opt/homebrew/opt/throttle-cli/bin/throttle-server", line 8, in <module>> sys.exit(main())> ^^^^^^> File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/cli_server.py", line 25, in main> socketpath = Path(BaseDirectory.get_runtime_dir()) / "throttle.sock"> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/xdg/BaseDirectory.py", line 141, in get_runtime_dir> return os.environ['XDG_RUNTIME_DIR']> ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^> File "<frozen os>", line 685, in __getitem__> KeyError: 'XDG_RUNTIME_DIR'>> sure, we need a XDG_RUNTIME_DIR, somehow pyxdg's hardcoded fallback to > /tmp does not work. >> Incidentally, I did some looking around trying to understand what other > projects do when XDG_* vars are undefined. It seems the platformdirs > pypi package has platform specific fallbacks and may be a more portable > alternative to pyxdg?
Thanks for looking into this, this seems like a sane alternative.
>> Setting it manually:>> $ XDG_RUNTIME_DIR=/Users/mya/Library/Caches/TemporaryItems > /opt/homebrew/opt/throttle-cli/bin/throttle-server --LOGLEVEL DEBUG > Traceback (most recent call last):> File "/opt/homebrew/opt/throttle-cli/bin/throttle-server", line 8, in <module>> sys.exit(main())> ^^^^^^> File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/cli_server.py", line 29, in main> start_server(socketpath, loglevel)> File "/opt/homebrew/Cellar/throttle-cli/0.3.0/libexec/lib/python3.12/site-packages/throttle_cli/server.py", line 53, in start_server> p_ipc.start()> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/process.py", line 121, in start> self._popen = self._Popen(self)> ^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/context.py", line 224, in _Popen> return _default_context.get_context().Process._Popen(process_obj)> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/context.py", line 289, in _Popen> return Popen(process_obj)> ^^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 32, in __init__> super().__init__(process_obj)> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_fork.py", line 19, in __init__> self._launch(process_obj)> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/popen_spawn_posix.py", line 47, in _launch> reduction.dump(process_obj, fp)> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/reduction.py", line 60, in dump> ForkingPickler(file, protocol).dump(obj)> AttributeError: Can't pickle local object 'start_server.<locals>.handleMsg'> Traceback (most recent call last):> File "<string>", line 1, in <module>> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 122, in spawn_main> exitcode = _main(fd, parent_sentinel)> ^^^^^^^^^^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/spawn.py", line 132, in _main> self = reduction.pickle.load(from_parent)> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^> File "/opt/homebrew/Cellar/python@3.12/3.12.2_1/Frameworks/Python.framework/Versions/3.12/lib/python3.12/multiprocessing/synchronize.py", line 115, in __setstate__> self._semlock = _multiprocessing.SemLock._rebuild(*state)> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^> FileNotFoundError: [Errno 2] No such file or directory>> This no longer seems related to the XDG_* vars, but to pickling. Any > suggestions?>> kind regards, Maarten
I always run into this then forget, linux forks by default, and mac and windows
spawn (and the latter can't fork).
The quickfix is probably adding this, before the server is passed off to multiprocessing:
multiprocessing.set_start_method("fork")
Probably would be better to make it work with spawn, I'll send patches for both
issues sometimes later. In the meanwhile can you check if setting the start
method explicitly to fork makes the error go away?
Best,
Bence
--
+36305425054
bence.ferdinandy.com
On Tue Apr 9, 2024 at 12:29 CEST, Bence Ferdinandy wrote:
> On Tue Apr 09, 2024 at 12:02, Maarten Aertsen <maarten@nlnetlabs.nl> > wrote:> > hi ferdinand,>> Bence :)
As I said on IRC, apologies Bence, that was that clumsy.
> > This no longer seems related to the XDG_* vars, but to pickling. Any > > suggestions?>> I always run into this then forget, linux forks by default, and mac and windows> spawn (and the latter can't fork).>> The quickfix is probably adding this, before the server is passed off to multiprocessing:>> multiprocessing.set_start_method("fork")>> Probably would be better to make it work with spawn, I'll send patches for both> issues sometimes later. In the meanwhile can you check if setting the start> method explicitly to fork makes the error go away?
Yes, I can confirm this fixes the issue. It starts up now. Thank you.
Happy to try your patches in due time.
kind regards, Maarten