Hi,
Recently, I have been examining how I could set up for myself a totally 
"portable" development environment, so I could use it easily at school.
w64devkit was basically perfect for this, except I had a need to add 
directories containing other tools to the PATH, and also set a HOME 
directory there, so tools like vim and git wouldn't attempt to place or 
search for config files anywhere on C:\.

I have attached the patch to w64devkit.c I wrote to do this. I suspect 
I'm probably not the only person who would want to do this, so I'll 
leave it as reference for anyone else who might want to do the same.

I think it is worth considering perhaps supporting having a file next to 
w64devkit, filled with key/value pairs that the environment will 
inherit, to make this easier. But currently, this seems to behave 
exactly how I wanted. A perfect example of how free software works!

Best wishes,
Peter D.

Interesting, Peter, thanks! I'd been considering adding environment 
variable arguments to w64devkit.exe, a la make and configure scripts:

    w64devkit.exe HOME=C:\path\to\home PATH+=c:\git\cmd

You'd include such arguments in a shortcut target, then launch w64devkit 
from that shortcut. There would also be syntax to compute an absolute path 
from a path relative to w64devkit.exe (a la %~dp0), though I haven't 
worked that out. That would be essential for your use case.

If one of the arguments is "-" (or maybe "-i"?) then it would delete all 
environment variables, a la the "env" command, giving you a clean slate. 
Though it might leave a couple special, soft-required variables behind 
like SYSTEMROOT and TEMP, since many programs break badly when these are 
unset. (There are enough of these special variables that it might not be 
worth having the option.)

Caveat: The Windows shortcut UI is incredibly restricted, limited, and 
finicky — essentially unchanged over the past quarter century — and it 
does not permit relative targets. So I don't know how well it works in a 
portable setup. Considering this, I prefer your idea of an adjacent 
configuration file, and I'm considering it. It would need a %~dp0-style 
feature, and perhaps it would just be worth simply expanding environment 
variables as a general feature.

Alternatively, you could launch w64devkit from a batch script, setting 
variables before starting w64devkit.exe via the "start" command (to 
jettison the cmd.exe process):

    @echo off
    HOME=%~dp0\homedir
    PATH=%~dp0\git\cmd;%PATH%
    start "w64devkit" w64devkit\w64devkit.exe

(At this point you could skip w64devkit.exe altogether, add w64devkit's 
bin to PATH in the batch script, and then run "busybox sh -l" directly, 
still via "start". Though you miss out on my lovely icon and the two 
w64devkit environment variables!)

However, the only variable that truly matters is HOME. Everything else can 
be set inside a .profile at that location, which, besides, is a better 
place to configure the environment. That's my recommendation, and you only 
need a good way to set HOME first, such as by batch script (or in the 
future, an adjacent environment configuration).

By the way, per my examples, better to put PortableGit's cmd/ in PATH 
rather than its bin/. The bin directory has its own MSYS2 sh.exe which you 
don't want in PATH (when using w64devkit), plus cmd/ contains extra Git 
goodies like gitk.exe.

Here's my own w64devkit .profile in case there are any ideas worth 
harvesting for your portable setup:

https://github.com/skeeto/dotfiles/blob/master/w64devkit.profile

On 2022-07-21 23:52, Christopher Wellons wrote:
> Interesting, Peter, thanks! I'd been considering adding environment
> variable arguments to w64devkit.exe, a la make and configure scripts:
> 
>    w64devkit.exe HOME=C:\path\to\home PATH+=c:\git\cmd
> 
> You'd include such arguments in a shortcut target, then launch
> w64devkit from that shortcut. There would also be syntax to compute an
> absolute path from a path relative to w64devkit.exe (a la %~dp0),
> though I haven't worked that out. That would be essential for your use
> case.
> 
> If one of the arguments is "-" (or maybe "-i"?) then it would delete
> all environment variables, a la the "env" command, giving you a clean
> slate. Though it might leave a couple special, soft-required variables
> behind like SYSTEMROOT and TEMP, since many programs break badly when
> these are unset. (There are enough of these special variables that it
> might not be worth having the option.)
I can't really think of why this would be useful, if the system is
broken so badly that unsetting all the environment variables is useful,
then I imagine it would already have existing problems. I would be
interested to understand the use cases though, if you could explain 
some.
> 
> Caveat: The Windows shortcut UI is incredibly restricted, limited, and
> finicky — essentially unchanged over the past quarter century — and it
> does not permit relative targets. So I don't know how well it works in
> a portable setup. Considering this, I prefer your idea of an adjacent
> configuration file, and I'm considering it. It would need a
> %~dp0-style feature, and perhaps it would just be worth simply
> expanding environment variables as a general feature.
I've never used shortcuts, but I should elaborate on what I had in mind.
Since, as you pointed out, the only environment variable that really
can't be set outside of a shell config is $HOME, here's what I propose:

Have a text file, maybe "w64devkithome.txt", that you can set the
value of the home dir in. If it starts with X:\, treat it as an absolute
path, and if it doesn't, treat it as a path relative to w64devkit.exe.
> 
> Alternatively, you could launch w64devkit from a batch script, setting
> variables before starting w64devkit.exe via the "start" command (to
> jettison the cmd.exe process):
I have never written any cmd scripts of any real complexity or length,
it was ironically easier for me to figure out patching w64devkit.exe for
this purpose. But, I will keep it in mind that I can.
> 
>    @echo off
>    HOME=%~dp0\homedir
>    PATH=%~dp0\git\cmd;%PATH%
>    start "w64devkit" w64devkit\w64devkit.exe
> 
> (At this point you could skip w64devkit.exe altogether, add
> w64devkit's bin to PATH in the batch script, and then run "busybox sh
> -l" directly, still via "start". Though you miss out on my lovely icon
> and the two w64devkit environment variables!)
I agree, the icon is very nice.
If this was the case, then what is the purpose of w64devkit at all? Does
the issue of the "misbehaving monitor cmd.exe" not apply here?
> 
> However, the only variable that truly matters is HOME. Everything else
> can be set inside a .profile at that location, which, besides, is a
> better place to configure the environment. That's my recommendation,
> and you only need a good way to set HOME first, such as by batch
> script (or in the future, an adjacent environment configuration).
Well, at least not all of what I did was redundant then. I will move the
prepending to the PATH to .profile later.
> 
> By the way, per my examples, better to put PortableGit's cmd/ in PATH
> rather than its bin/. The bin directory has its own MSYS2 sh.exe which
> you don't want in PATH (when using w64devkit), plus cmd/ contains
> extra Git goodies like gitk.exe.
Noted, thanks for telling me, I will switch. I also wonder, what parts
of PortableGit are actually strictly needed to function? Is it possible
to remove some parts of it? I doubt you know the answer, I'm probably
asking the wrong person, but it seems suspicious I need perl and the
whole MSYS2 shell.
> 
> Here's my own w64devkit .profile in case there are any ideas worth
> harvesting for your portable setup:
> 
> https://github.com/skeeto/dotfiles/blob/master/w64devkit.profile
Thanks, I wasn't aware of this. Certainly, I think it would be nice to
add go and gpg to my setup. Have you managed to set up gpg for signing
git commits within w64devkit yourself?

Thanks for the very informative and useful reply, it has been helpful.

Regards,
Peter D.

> I can't really think of why this would be useful

I occasionally use invocations like "env - PATH=/usr/bin" on Linux, and 
similar on Windows with w64devkit in order to establish a clean(er) 
environment for testing and reproduction. I may even set a temporary HOME 
to keep (most) programs from reading custom configs. In particular, some 
application defaults are poor or newbie-oriented, the default can be 
overridden by an environment variable — directly or indirectly — so I do 
so. Then I eventually forget that I'm not running the default. It's an 
easy way to wipe the slate clean.

For Windows this also means clearing out junk added to the environment by 
various installers, which may interfere with issue reproduction. (A stray 
PYTHONHOME or JAVA_HOME, etc.)

> Does the issue of the "misbehaving monitor cmd.exe" not apply here?

To use unix shell nomenclature, running a command with "start" effectively 
starts it as a background job, then detaches the process. For console 
subsystem programs like w64devkit.exe, it spawns a fresh console. The 
cmd.exe is not monitoring the new console, and it's free to exit.

It's not ideal since launching via batch script will create a temporary 
console for the cmd.exe, which flashes in and out of existence (animations 
and all in more recent versions of Windows). It's worse for slow batch 
scripts, such as the case for Visual Studio's vcvars.bat. (Visual Studio 
has become so bloated and slow that it takes several seconds just to set 
some environment variables. Contrast with Visual Studio 2008, when it was 
instantaneous.)

By the way, the "start" command — a cmd.exe built-in — is actually pretty 
useful in general, which is why I've aliased it in my own .profile. Point 
it at a URL to open it in the default browser (note: sort of, Firefox 
doesn't work due to a long-standing profile management bug), at a 
directory to open the file explorer (esp. "start ."), at a file to open it 
with its default association, etc.

> Is it possible to remove some parts of it?

I've never had the need to strip it down, but if you ever find out I'd 
like to know as well. The embedded MSYS2 is quite inconvenient when you 
don't want it. It comes with its own version of Vim, which is what you'll 
normally get instead of w64devkit's Vim (I use EDITOR=vim.bat to force the 
one in w64devkit). Even still, when it invokes the editor, it puts all its 
MSYS2 binaries at the front of PATH, so I can't reliably access non-MSYS2 
commands. The MSYS2 programs aren't truly native — they run in a virtual 
filesystem, etc. — so they don't interact properly with my non-MSYS2 
tools. I wish the situation was better.

> Have you managed to set up gpg for signing git commits within w64devkit 
> yourself?

In case you weren't aware, I maintain builds of GnuPG, built in Docker 
just like w64devkit, since I don't like the official Windows builds:

https://github.com/skeeto/gnupg-windows-build

Part of my motivation is legitimizing my use of OpenPGP signatures on my 
software releases. Not even Windows users can complain that verifying my 
signatures is too cumbersome. Cool fact: gpgv.exe is standalone — IMHO, 
the only GnuPG component that escapes its insane daemon architecture — so 
if you only care about verifying signatures, you could grab that binary 
and skip the rest. It even works with Git (see gpg.program) with small 
helper script.

Per the MSYS2 discussion, MSYS2 Git also embeds its own GnuPG. This GnuPG, 
not being Windows-native, looks in the wrong place for the keyring. A 
couple of options:

* Maintain a MSYS2 keyring for use by Git. You can access that version of 
GnuPG through the bash.exe shell. If this is the only place you care about 
accessing GnuPG then that's probably the most convenient option.

* In Git, set gpg.program to your native Windows GnuPG, such as the one I 
distribute. Git can use it effectively despite not being MSYS2. Then you 
can have one GnuPG keyring common to Git and elsewhere.

Either works correctly as far as my testing goes.

> Certainly, I think it would be nice to add go and gpg to my setup.

In case you didn't know, it's trivial to bootstrap Go from source using 
w64devkit, which is where I get my Go toolchain. Compile bootstrap Go 1.4 
with w64devkit (note: this is an updated 1.4 release still maintained by 
the Go team), then use that to build the latest Go. It only takes a few 
minutes. After that you can continue upgrading Go from source. The built 
toolchain is "portable" so you can toss it alongside everything else in 
your setup.

I have a small surprise for you, Peter. Instead of "w64devkithome.txt" I 
have opted for "w64devkit.ini" which currently lives on the "ini" branch:

https://github.com/skeeto/w64devkit/commit/8db776f

It supports variable expansion and paths relative to the .ini file. You'd 
use "home = ..\homedir" in your case. I went for an .ini file since I 
figured this would grow more options in the future.

I carefully read the .ini file as UTF-8 and pass wide paths to Windows, 
though unfortunately busybox-w32 remains limited by the narrow API.

I'm going to dogfood this for awhile, in case I notice issues and to 
incorporate feedback you or others might have.

> what other options do you have potentially in mind?

* Tell w64devkit.exe to start a new console window instead of reusing the 
one it was given.

* The "clear the environment" idea I mentioned since it's difficult to 
accomplish in the .profile. Could help normalize the environment across 
different machines by keeping the local junk out of the way.

* CreateProcess options. For example, starting the shell under a new job 
object with a "kill on job close" flag. I've been experimenting with 
running w64devkit, including the home directory, from a thumb drive using 
the new .ini configuration. I've found GnuPG Agent even more irritating 
than usual. It prevents me from ejecting the drive until I manually kill 
the process. It might be nice if closing the console killed every process 
started from it.

* An alternate shell path.

> this path in he INI can't contain any non-ascii characters?

Mostly, but it's not quite so strict. You can use any code point from your 
system's active code page. For example, if configured for CP-1252, as is 
often the case for western languages, then you could use the non-ASCII 
word "naïve" in your home path. You'd encode this path with UTF-8 in 
w64devkit.ini, and then, via the narrow API, busybox-w32 would receive and 
use the CP-1252 encoding of that path without loss.

https://en.wikipedia.org/wiki/Windows-1252

(With CP-1252 as the active code page, you can type ï into the BusyBox 
shell with alt+0239. Useful for testing. In Vim the digraph is ":i".)

> for my use case, things are a bit less clean on .profile

Remember that HOME is converted to an absolute path before starting the 
shell, and that absolute paths can have relative components like "..", so 
you don't need that realpath. Just use HOME as an anchor. This will work 
fine:

PATH="$PATH;$HOME/../PortableGit/cmd"

That becomes "E:/compilers/homedir/../PortableGit/cmd" in your PATH, which 
does what you'd expect. This sort of thing only becomes a problem if 
you're pushing the (meager on Windows) PATH length limitation.

> get the drive letter of the drive that w64devkit.exe is located on

Unfortunately it's not that simple. For example, if you run it from a 
network drive then there is no drive letter. However, W64DEVKIT_HOME is 
guaranteed to be an absolute path, and if you're confident it's not a 
network path then you could "cut -b1-2" on this variable to extract a 
drive. (Thought: What is HOMEDRIVE supposed to be when the user profile is 
on a network drive? I can't figure this out, or if any convention exists 
in the first place.)

> w64devkit is really serving me well.

Great to hear!

Aug 4, 2022 12:57:52 AM Christopher Wellons <wellons@nullprogram.com>:

> I have a small surprise for you, Peter. Instead of "w64devkithome.txt" I have opted for "w64devkit.ini" which currently lives on the "ini" branch:
>
> https://github.com/skeeto/w64devkit/commit/8db776f
>
> It supports variable expansion and paths relative to the .ini file. You'd use "home = ..\homedir" in your case. I went for an .ini file since I figured this would grow more options in the future.
Looks great, what other options do you have potentially in mind?
>
> I carefully read the .ini file as UTF-8 and pass wide paths to Windows, though unfortunately busybox-w32 remains limited by the narrow API.
So, I suppose this means that this path in he INI can't contain any
non-ascii characters? It's out of your hands anyway, I suppose.
>
> I'm going to dogfood this for awhile, in case I notice issues and to incorporate feedback you or others might have.

I'm going to do the same, thanks for telling 
me. I've tried it out, and
it works as intended for me.

However, for my use case, things are a bit less clean on .profile
My setup looks something like this:
E:/compilers/homedir
E:/compilers/w64devkit
E:/compilers/PortableGit
..etc

I had to write in my .profile:
COMPILERSDIRPATH=`realpath ..`
PATH="$PATH;$COMPLERSDIRPATH/PortableGit/cmd"
...
unset COMPILERSDIRPATH

It might be nice to provide to users a way to get the drive letter of
the drive that w64devkit.exe is located on, so I don't have to do this
On some of my school computers, they have DVD drives, and others don't,
so the drive letter it's located on changes.

Back when I used the git bash as a shell of choice, I had to do some
awful hack that involved code that looked for a file on the root of
every single drive letter (and on top of it, it only worked because the
homedir was shared between every computer, I could not figure out how to
tell it where to look for the shell profile)
So, I think provi
ding a variable DRIVELETTER for this purpose would be cool.

Thanks for your time, w64devkit is really serving me well.

Regards,
Peter D.