Austreelis: 1
Use relative imports when importing offpunk's own modules
5 files changed, 11 insertions(+), 11 deletions(-)
Actually, that's not an issue with this architecture, that's an issue
with the assumptions I made when making the patches :) When importing
its own modules offpunk looks for a package named "offpunk". This
assumes you either have it somewhere in your PYTHONPATH or in the pwd.
Running script from anywhere isn't something I anticipated - coming
from NixOS, and having only ever developped python in the PEP517 era
where you always install modules you run, this is something I basically
never consider.
I have a patch ready that solves this by using relative imports
instead, I think this should solve it without breaking anything
packaging-related, but I'm still testing this.
Forgot to say it: the previous patch should be applied atop the 2 previous
patches I sent (with great struggle). If that's an issue I can resend the
whole patch set, but I still haven't found why I the first of the patches I
send wasn't correctly being sent so I'll likely just create a new thread if
so.
I'd rather keep offpunk's development content in offpunk-devel mailing list
than offload it to my fork. I only made a fork to have a git server
somewhere people can fetch the current work from (without me sending a 300MB
email[1] with each commit), not to hack on.
If you want to avoid making a bunch of noise in offpunk-devel, just send me the
patches somehow (email, pull access to a git server, ...). Anything on my
gitlab account may get nuked in the following months when I'll finally decide to
migrate off gitlab.
[1]: Because my change imply modifying and renaming most of offpunk's existing
python script, even the patience diff algorithm generates diff of my branch
squashed with old files deleted (whole) and new files added (whole), instead
of renamed + changed. That diff is 300MB+
( https://gitlab.com/austreelis/offpunk/-/commit/d3e827751fc5dd019169e1ba5554cdd907ba33fe.patch )
So I deemed it easier to publish a fork than send a full patch set unsquashed
every time (or 300MB emails).
I’ve tested this patch and this doesn’t seem to solve the problem of
running offpunk locally.
But ansicat, netcache and opnk are working fine.
I wonder if it would make more sense to follow the same approche for
offpunk. A simple offpunk.py at the root would solve all of our
problems.
Also, I’m not really comfortable with the "from .". I wonder how they
might behave when installed.
This essentially means "look for modules in the top-level package's
directory" (i.e. __main__), instead of "look in sys.path". Since it's
not fixing much (if anything). I don't think the patch is still needed
(especially if you're not comfortable with it.
I’m testing with an offpunk.py at the root and "from offpunk" in the
import and it seems to work fine for me. Maybe this is the way to go ?
What’s wrong with this approach?
Thanks for that patch. I applied it on top of the two others, like
instructed.
I don’t understand what this is supposde to change or how offpunk can be
launched locally.
BTW, you were right: the netcache|ansicat pipeline is now working as
expected. Thanks for that !
Oh, that's right, you still can't just `python -m <some random dir>`.
Bah, my bad, it doesn't actually solve your issue with launching
offpunk. Sorry about the blunder.
I think a solution to run some `what/ever/offpunk.py` might be needed
anyway, but this might get a bit hacky. Just as a reminder, nothing
forbids putting an `offpunk.py` in the repo's root just like the other
tools, it just makes flit complain about it and throw its arms up. So
I'll try to work around that.
I see several ways:
- alias to `PYTHONPATH=/path/to/repo/ python -m offpunk` instead of
`/path/to/repo/offpunk.py` (old) or `python -m /path/to/repo/offpunk`
(nonsensical). That's not a fix, that's just breaking users' spacebar
heating (insert xkcd/1172).
- Somehow get flit to ignore an `offpunk.py` in the repo's route,
hopefully without too much black magic, nor an upstream change.
Notably I haven't tested if adding the glue scripts to
pyproject.toml's "exclude" setting actually solves this.
- Use another name for the local script, e.g. `offpunk_local.py` or
`loffpunk.py` if amount of bytes is the almighty metric. It would be
disappointing to resort to this imo.
- Add a build step moving away/hiding glue scripts from flit. It should
Maybe this is too bad for your taste, but would running a
`~/dev/offpunk/main.py` would rub the wrong way ? If the script to run
offpunk needs to be called `offpunk.py`, then I fear we either need to
break something (be it user workflow/scripts or packaging) or ~~break~~
patch flit.
NB: Sorry for double-posting, I'm kind of writing as I poke at things.
break all packaging workflows, which I dislike just as much as the
first item. And I'm not sure I can help patch say arch's PKGBUILD,
so I would avoid this if possible.
Welp it doesn't. Maybe I can ask flit if that's something they would
consider changing, but that still isn't ideal for packaging (which will
be broken until such a change lands in the package repos.
------- Original Message -------
I'll do a one-file patch :)
As a heads-up: I have a working-ish setup.py and setuptools workflow, but I had
to change the files layout *again* because when installing locally via
setuptools through pip (`pip install .`, which eventually just runs setup.py),
it confuses offpunk.py (the script) and offpunk (the package/module/folder).
Adopting a src-layout (which is the recommeeded thing to do these days) did
allow to tell setuptools to look into src/, and only there, but this is getting
a bit hairy (and tiring :P). This also means running offpunk locally straight
from the repo still works, but only when offpunk modules use relative imports
(which is another change I had to make).
I have definitely not yet gotten to the bottom of this, so I need to keep
exploring solutions to see if there's a better compromise. Either way:
- setuptools-based tools works great regardless of the file layout issue thing.
Somehow none of `python setup.py install`, `python -m build sdist` and co.
fail, only pip does. This is still bad as it breaks e.g. nix packaging
(an probably more).
- Running (unlik installing) locally still works great.
- Adopting the src layout should be the solution.
To be clear, this means I now have on my branch something like:
src
└── offpunk
├── __init__.py
├── __main__.py
├── ansicat.py
├── cache_migration.py
├── netcache.py
├── opnk.py
├── themes.py
└── utils.py
and script shims in the repo root that `from src.offpunk.ansicat import main`
(in the case of ansicat).
TL;DR: Setuptools (when used through pip) still has the same issue of not being
able to tell `offpunk.py` and `offpunk/__init__.py` apart. That is, unless I
missed something, which I'm still investigating. Meanwhile I have a working
setup by using relative imports and a src/ layout, but I'd like to see if
there's a way to deviate "as little as possible" from current trunk's way of
doing things.
On 23/09/19 03:47, David Zaslavsky wrote:
Tbh, I'm not familiar with it and I've already spend enough time on this
that I'm not ready to learn tox' workflow. I don't have any real opinion on
tox, but to me this is out of scope of what I'm trying to get merged.
NB: I've published a personal fork of offpunk at
https://gitlab.com/austreelis/offpunk.git , feel free to add it as a remote if
you want to peek at my work (in the `main` branch) before I send the finalized
patch :)
If I may: in the original Mastodon discussion that prompted my thread
about packaging, I had raised the idea of using tox for testing. If
you want to do things like checking whether piping netcache to ansicat
works, tox would be a good way to set up the environment for that test
such that it'd be easy for other developers to run the tests without
having to configure anything or run any obscure commands on their side.
Hey,
Thanks for the recommendation (and all your valuable input to the
discussion). What’s funny is that, for me, "tox" is quite esoteric ;-)
My requirements to merge the tests is to not add any dependency for
running offpunk. So tests can be safely ignored by everybody who doesn’t
care about it.
In previous projects, I always had a simple "run.sh" to launch the
program and "run_test.sh" to launch the tests.
David
------- Original Message -------
This is a great idea. Feeling useless myself, I decided to write down my
thoughts on the subject.
https://ploum.net/2023-10-01-future-of-offpunk-packaging-hell.html
Thanks both of you for your help and support
Thanks for sharing!
I had a thought... Austreelis, would it help if I pushed some of my
suggestions to a branch on your repository? For some of these things it
might be more efficient to base the discussion on actual code. If you
think that'd be useful, feel free to give my Gitlab account
(https://gitlab.com/diazona) the necessary permissions. (Or I could
always make a fork or something if you prefer)
David