~skeeto/public-inbox

4 2

Configuring w64devkit.exe to add more directories to the PATH

Details
Message ID
<a2720b0e7adc3391376ca571e9c7fdb4@snopyta.org>
DKIM signature
pass
Download raw message
Hi,
Recently, I have been examining how I could set up for myself a totally 
"portable" development environment, so I could use it easily at school.
w64devkit was basically perfect for this, except I had a need to add 
directories containing other tools to the PATH, and also set a HOME 
directory there, so tools like vim and git wouldn't attempt to place or 
search for config files anywhere on C:\.

I have attached the patch to w64devkit.c I wrote to do this. I suspect 
I'm probably not the only person who would want to do this, so I'll 
leave it as reference for anyone else who might want to do the same.

I think it is worth considering perhaps supporting having a file next to 
w64devkit, filled with key/value pairs that the environment will 
inherit, to make this easier. But currently, this seems to behave 
exactly how I wanted. A perfect example of how free software works!

Best wishes,
Peter D.
Details
Message ID
<20220721205204.6d74uppkls3fekvc@nullprogram.com>
In-Reply-To
<a2720b0e7adc3391376ca571e9c7fdb4@snopyta.org> (view parent)
DKIM signature
missing
Download raw message
Interesting, Peter, thanks! I'd been considering adding environment 
variable arguments to w64devkit.exe, a la make and configure scripts:

    w64devkit.exe HOME=C:\path\to\home PATH+=c:\git\cmd

You'd include such arguments in a shortcut target, then launch w64devkit 
from that shortcut. There would also be syntax to compute an absolute path 
from a path relative to w64devkit.exe (a la %~dp0), though I haven't 
worked that out. That would be essential for your use case.

If one of the arguments is "-" (or maybe "-i"?) then it would delete all 
environment variables, a la the "env" command, giving you a clean slate. 
Though it might leave a couple special, soft-required variables behind 
like SYSTEMROOT and TEMP, since many programs break badly when these are 
unset. (There are enough of these special variables that it might not be 
worth having the option.)

Caveat: The Windows shortcut UI is incredibly restricted, limited, and 
finicky — essentially unchanged over the past quarter century — and it 
does not permit relative targets. So I don't know how well it works in a 
portable setup. Considering this, I prefer your idea of an adjacent 
configuration file, and I'm considering it. It would need a %~dp0-style 
feature, and perhaps it would just be worth simply expanding environment 
variables as a general feature.

Alternatively, you could launch w64devkit from a batch script, setting 
variables before starting w64devkit.exe via the "start" command (to 
jettison the cmd.exe process):

    @echo off
    HOME=%~dp0\homedir
    PATH=%~dp0\git\cmd;%PATH%
    start "w64devkit" w64devkit\w64devkit.exe

(At this point you could skip w64devkit.exe altogether, add w64devkit's 
bin to PATH in the batch script, and then run "busybox sh -l" directly, 
still via "start". Though you miss out on my lovely icon and the two 
w64devkit environment variables!)

However, the only variable that truly matters is HOME. Everything else can 
be set inside a .profile at that location, which, besides, is a better 
place to configure the environment. That's my recommendation, and you only 
need a good way to set HOME first, such as by batch script (or in the 
future, an adjacent environment configuration).

By the way, per my examples, better to put PortableGit's cmd/ in PATH 
rather than its bin/. The bin directory has its own MSYS2 sh.exe which you 
don't want in PATH (when using w64devkit), plus cmd/ contains extra Git 
goodies like gitk.exe.

Here's my own w64devkit .profile in case there are any ideas worth 
harvesting for your portable setup:

https://github.com/skeeto/dotfiles/blob/master/w64devkit.profile
Details
Message ID
<6d536308195886c0ad182582f3dc758a@snopyta.org>
In-Reply-To
<20220721205204.6d74uppkls3fekvc@nullprogram.com> (view parent)
DKIM signature
pass
Download raw message
On 2022-07-21 23:52, Christopher Wellons wrote:
> Interesting, Peter, thanks! I'd been considering adding environment
> variable arguments to w64devkit.exe, a la make and configure scripts:
> 
>    w64devkit.exe HOME=C:\path\to\home PATH+=c:\git\cmd
> 
> You'd include such arguments in a shortcut target, then launch
> w64devkit from that shortcut. There would also be syntax to compute an
> absolute path from a path relative to w64devkit.exe (a la %~dp0),
> though I haven't worked that out. That would be essential for your use
> case.
> 
> If one of the arguments is "-" (or maybe "-i"?) then it would delete
> all environment variables, a la the "env" command, giving you a clean
> slate. Though it might leave a couple special, soft-required variables
> behind like SYSTEMROOT and TEMP, since many programs break badly when
> these are unset. (There are enough of these special variables that it
> might not be worth having the option.)
I can't really think of why this would be useful, if the system is
broken so badly that unsetting all the environment variables is useful,
then I imagine it would already have existing problems. I would be
interested to understand the use cases though, if you could explain 
some.
> 
> Caveat: The Windows shortcut UI is incredibly restricted, limited, and
> finicky — essentially unchanged over the past quarter century — and it
> does not permit relative targets. So I don't know how well it works in
> a portable setup. Considering this, I prefer your idea of an adjacent
> configuration file, and I'm considering it. It would need a
> %~dp0-style feature, and perhaps it would just be worth simply
> expanding environment variables as a general feature.
I've never used shortcuts, but I should elaborate on what I had in mind.
Since, as you pointed out, the only environment variable that really
can't be set outside of a shell config is $HOME, here's what I propose:

Have a text file, maybe "w64devkithome.txt", that you can set the
value of the home dir in. If it starts with X:\, treat it as an absolute
path, and if it doesn't, treat it as a path relative to w64devkit.exe.
> 
> Alternatively, you could launch w64devkit from a batch script, setting
> variables before starting w64devkit.exe via the "start" command (to
> jettison the cmd.exe process):
I have never written any cmd scripts of any real complexity or length,
it was ironically easier for me to figure out patching w64devkit.exe for
this purpose. But, I will keep it in mind that I can.
> 
>    @echo off
>    HOME=%~dp0\homedir
>    PATH=%~dp0\git\cmd;%PATH%
>    start "w64devkit" w64devkit\w64devkit.exe
> 
> (At this point you could skip w64devkit.exe altogether, add
> w64devkit's bin to PATH in the batch script, and then run "busybox sh
> -l" directly, still via "start". Though you miss out on my lovely icon
> and the two w64devkit environment variables!)
I agree, the icon is very nice.
If this was the case, then what is the purpose of w64devkit at all? Does
the issue of the "misbehaving monitor cmd.exe" not apply here?
> 
> However, the only variable that truly matters is HOME. Everything else
> can be set inside a .profile at that location, which, besides, is a
> better place to configure the environment. That's my recommendation,
> and you only need a good way to set HOME first, such as by batch
> script (or in the future, an adjacent environment configuration).
Well, at least not all of what I did was redundant then. I will move the
prepending to the PATH to .profile later.
> 
> By the way, per my examples, better to put PortableGit's cmd/ in PATH
> rather than its bin/. The bin directory has its own MSYS2 sh.exe which
> you don't want in PATH (when using w64devkit), plus cmd/ contains
> extra Git goodies like gitk.exe.
Noted, thanks for telling me, I will switch. I also wonder, what parts
of PortableGit are actually strictly needed to function? Is it possible
to remove some parts of it? I doubt you know the answer, I'm probably
asking the wrong person, but it seems suspicious I need perl and the
whole MSYS2 shell.
> 
> Here's my own w64devkit .profile in case there are any ideas worth
> harvesting for your portable setup:
> 
> https://github.com/skeeto/dotfiles/blob/master/w64devkit.profile
Thanks, I wasn't aware of this. Certainly, I think it would be nice to
add go and gpg to my setup. Have you managed to set up gpg for signing
git commits within w64devkit yourself?

Thanks for the very informative and useful reply, it has been helpful.

Regards,
Peter D.
Details
Message ID
<20220723190547.czyu5pk4kdiplspm@nullprogram.com>
In-Reply-To
<6d536308195886c0ad182582f3dc758a@snopyta.org> (view parent)
DKIM signature
missing
Download raw message
> I can't really think of why this would be useful

I occasionally use invocations like "env - PATH=/usr/bin" on Linux, and 
similar on Windows with w64devkit in order to establish a clean(er) 
environment for testing and reproduction. I may even set a temporary HOME 
to keep (most) programs from reading custom configs. In particular, some 
application defaults are poor or newbie-oriented, the default can be 
overridden by an environment variable — directly or indirectly — so I do 
so. Then I eventually forget that I'm not running the default. It's an 
easy way to wipe the slate clean.

For Windows this also means clearing out junk added to the environment by 
various installers, which may interfere with issue reproduction. (A stray 
PYTHONHOME or JAVA_HOME, etc.)

> Does the issue of the "misbehaving monitor cmd.exe" not apply here?

To use unix shell nomenclature, running a command with "start" effectively 
starts it as a background job, then detaches the process. For console 
subsystem programs like w64devkit.exe, it spawns a fresh console. The 
cmd.exe is not monitoring the new console, and it's free to exit.

It's not ideal since launching via batch script will create a temporary 
console for the cmd.exe, which flashes in and out of existence (animations 
and all in more recent versions of Windows). It's worse for slow batch 
scripts, such as the case for Visual Studio's vcvars.bat. (Visual Studio 
has become so bloated and slow that it takes several seconds just to set 
some environment variables. Contrast with Visual Studio 2008, when it was 
instantaneous.)

By the way, the "start" command — a cmd.exe built-in — is actually pretty 
useful in general, which is why I've aliased it in my own .profile. Point 
it at a URL to open it in the default browser (note: sort of, Firefox 
doesn't work due to a long-standing profile management bug), at a 
directory to open the file explorer (esp. "start ."), at a file to open it 
with its default association, etc.

> Is it possible to remove some parts of it?

I've never had the need to strip it down, but if you ever find out I'd 
like to know as well. The embedded MSYS2 is quite inconvenient when you 
don't want it. It comes with its own version of Vim, which is what you'll 
normally get instead of w64devkit's Vim (I use EDITOR=vim.bat to force the 
one in w64devkit). Even still, when it invokes the editor, it puts all its 
MSYS2 binaries at the front of PATH, so I can't reliably access non-MSYS2 
commands. The MSYS2 programs aren't truly native — they run in a virtual 
filesystem, etc. — so they don't interact properly with my non-MSYS2 
tools. I wish the situation was better.

> Have you managed to set up gpg for signing git commits within w64devkit 
> yourself?

In case you weren't aware, I maintain builds of GnuPG, built in Docker 
just like w64devkit, since I don't like the official Windows builds:

https://github.com/skeeto/gnupg-windows-build

Part of my motivation is legitimizing my use of OpenPGP signatures on my 
software releases. Not even Windows users can complain that verifying my 
signatures is too cumbersome. Cool fact: gpgv.exe is standalone — IMHO, 
the only GnuPG component that escapes its insane daemon architecture — so 
if you only care about verifying signatures, you could grab that binary 
and skip the rest. It even works with Git (see gpg.program) with small 
helper script.

Per the MSYS2 discussion, MSYS2 Git also embeds its own GnuPG. This GnuPG, 
not being Windows-native, looks in the wrong place for the keyring. A 
couple of options:

* Maintain a MSYS2 keyring for use by Git. You can access that version of 
GnuPG through the bash.exe shell. If this is the only place you care about 
accessing GnuPG then that's probably the most convenient option.

* In Git, set gpg.program to your native Windows GnuPG, such as the one I 
distribute. Git can use it effectively despite not being MSYS2. Then you 
can have one GnuPG keyring common to Git and elsewhere.

Either works correctly as far as my testing goes.

> Certainly, I think it would be nice to add go and gpg to my setup.

In case you didn't know, it's trivial to bootstrap Go from source using 
w64devkit, which is where I get my Go toolchain. Compile bootstrap Go 1.4 
with w64devkit (note: this is an updated 1.4 release still maintained by 
the Go team), then use that to build the latest Go. It only takes a few 
minutes. After that you can continue upgrading Go from source. The built 
toolchain is "portable" so you can toss it alongside everything else in 
your setup.
Details
Message ID
<20220803215749.cdcusm3idxnaie66@nullprogram.com>
In-Reply-To
<6d536308195886c0ad182582f3dc758a@snopyta.org> (view parent)
DKIM signature
missing
Download raw message
I have a small surprise for you, Peter. Instead of "w64devkithome.txt" I 
have opted for "w64devkit.ini" which currently lives on the "ini" branch:

https://github.com/skeeto/w64devkit/commit/8db776f

It supports variable expansion and paths relative to the .ini file. You'd 
use "home = ..\homedir" in your case. I went for an .ini file since I 
figured this would grow more options in the future.

I carefully read the .ini file as UTF-8 and pass wide paths to Windows, 
though unfortunately busybox-w32 remains limited by the narrow API.

I'm going to dogfood this for awhile, in case I notice issues and to 
incorporate feedback you or others might have.
Reply to thread Export thread (mbox)