~skeeto/public-inbox

5 2

Threads in w64devkit

Harris Snyder
Details
Message ID
<CAKF5_T=SN4tAsLfp1pWOfN5wr7cd1GNA6uo8qSroY9jg9i_cqA@mail.gmail.com>
DKIM signature
pass
Download raw message
Hi Chris (and anyone else who's interested in this topic),

Thanks again for w64devkit - it's a wonderful tool. I'm curious about
using it for multithreaded programming. I see that the distribution
does not come with support for C11 threads (I gather this is a
mingw/gcc issue), but does come with a pthread implementation. Do you
think using the included pthread implementation is the right way to
go, or should we prefer to use the Windows native API for threading?
Or, use an open source C11 threads implementation such as
https://github.com/tinycthread/tinycthread ? I like language
standards, so I really want to use C11 threads, but I also want things
to work correctly - sometimes it's better to have a windows-specific
code path rather than using an API that "almost" works the same way
everywhere...

Thanks very much,
Harris
Details
Message ID
<20210827234205.ajovvd6shilsnewa@nullprogram.com>
In-Reply-To
<CAKF5_T=SN4tAsLfp1pWOfN5wr7cd1GNA6uo8qSroY9jg9i_cqA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
Interesting topic, Harris!

> (I gather this is a mingw/gcc issue)

Yup, it's not (yet?) implemented by Mingw-w64. Though, there's little 
point to doing so anyway. C11 threads have completely failed to achieve 
their one purpose: a portable thread interface across implementations that 
support threads. This is mostly Microsoft's fault as the main holdout. For 
decades we've had pthreads as a standard, mature, portable threading API 
available everywhere threads are available… except MSVC where you have to 
use Microsoft's proprietary thread APIs.

C11 threads were meant to address this sort of thing, but Microsoft isn't 
interested in implementing that API either. Anywhere you might find C11 
threads you can already access the superior pthreads. You're better off 
just using pthreads in those cases.

In contrast, their counterpart in C++11 has worked out well. Since they 
weren't optional, everyone had to support them, including Microsoft, so 
they're actually useful as a portable threading interface.

> should we prefer to use the Windows native API for threading?

It depends on your goals. If you don't care about MSVC, then just use 
pthreads. You can still target Windows via Mingw-w64, and pthreads works 
everywhere else.

Microsoft may have neglected pthreads and C11 threads, but MSVC supports 
OpenMP 2.0. It's a bit outdated and the implementation in MSVC is pretty 
limited, but still surprisingly useful if your goal is parallelism rather 
than concurrency. Most of the time it's sufficient for my goals — I really 
just need to parallelize a loop — and so I can cover all implementations I 
care about, including MSVC, by targeting that standard. With care, it even 
covers platforms without threading or OpenMP for free since my program is 
still valid (if slower) when ignoring the omp pragmas.

Those two cover at least 95% of my own cases.

If neither apply, I write a wrapper around pthreads and Win32 threads (or 
even just MSVC threads) as a platform layer implementing exactly what I 
need and no more. The tricky part is that the start_routine function 
prototype differs between thread APIs.

An advantage of a custom platform layer is that it can combine multiple 
features into the simpler interface needed by the application. For 
instance, on Windows I might implement a "poll for input events" with 
WaitForMultipleObjects() and a couple of threads, but on POSIX I might 
implement the same feature with just ioctl(2) and poll(2). If those are 
the only threads I need then the threading stuff need not be portable 
anyway. Or I might even hide the threads altogether and provide a generic 
job queue for the application. That nicely covers up the differences 
between threading APIs.

> Or, use an open source C11 threads implementation

If you're porting a program that already makes heavy use of C11 threads 
then this is a reasonable way to go about it. You can just plug it in and 
be done with it. However, writing a platform layer with thread support 
isn't so complex as to warrant a dependency on a greenfield project.

That being said, I'm impressed with TinyCThread. It's well-written and 
designed for embedding — i.e. ready to drop into any project. It's quite 
reasonable as far as dependencies go.
Harris Snyder
Details
Message ID
<CAKF5_Tkv7z=918T4jAKXXY1zR_V+yuZf0O9qLG9reZaz9-+-_g@mail.gmail.com>
In-Reply-To
<20210827234205.ajovvd6shilsnewa@nullprogram.com> (view parent)
DKIM signature
pass
Download raw message
Thanks for the detailed reply! FWIW, microsoft claims that adding C11
thread support is "on their roadmap" as of 2020 (see
https://devblogs.microsoft.com/cppblog/c11-and-c17-standard-support-arriving-in-msvc/),
but I'm not holding my breath either. Based on what you're saying
here, though, it sounds like for Unix-first greenfield projects,
pthreads is the way to go when OpenMP isn't sufficient.

The actual use case I had in mind was tutoring a windows user in
programming! (while, admittedly, trying to expose them to the
unix-style toolchain via w64devkit). Threading is hardly the first
topic I'm covering, but I've been planning some longer-term learning
projects that will eventually require some concurrency.

--Harris

On Fri, Aug 27, 2021 at 7:42 PM Christopher Wellons
<wellons@nullprogram.com> wrote:
>
> Interesting topic, Harris!
>
> > (I gather this is a mingw/gcc issue)
>
> Yup, it's not (yet?) implemented by Mingw-w64. Though, there's little
> point to doing so anyway. C11 threads have completely failed to achieve
> their one purpose: a portable thread interface across implementations that
> support threads. This is mostly Microsoft's fault as the main holdout. For
> decades we've had pthreads as a standard, mature, portable threading API
> available everywhere threads are available… except MSVC where you have to
> use Microsoft's proprietary thread APIs.
>
> C11 threads were meant to address this sort of thing, but Microsoft isn't
> interested in implementing that API either. Anywhere you might find C11
> threads you can already access the superior pthreads. You're better off
> just using pthreads in those cases.
>
> In contrast, their counterpart in C++11 has worked out well. Since they
> weren't optional, everyone had to support them, including Microsoft, so
> they're actually useful as a portable threading interface.
>
> > should we prefer to use the Windows native API for threading?
>
> It depends on your goals. If you don't care about MSVC, then just use
> pthreads. You can still target Windows via Mingw-w64, and pthreads works
> everywhere else.
>
> Microsoft may have neglected pthreads and C11 threads, but MSVC supports
> OpenMP 2.0. It's a bit outdated and the implementation in MSVC is pretty
> limited, but still surprisingly useful if your goal is parallelism rather
> than concurrency. Most of the time it's sufficient for my goals — I really
> just need to parallelize a loop — and so I can cover all implementations I
> care about, including MSVC, by targeting that standard. With care, it even
> covers platforms without threading or OpenMP for free since my program is
> still valid (if slower) when ignoring the omp pragmas.
>
> Those two cover at least 95% of my own cases.
>
> If neither apply, I write a wrapper around pthreads and Win32 threads (or
> even just MSVC threads) as a platform layer implementing exactly what I
> need and no more. The tricky part is that the start_routine function
> prototype differs between thread APIs.
>
> An advantage of a custom platform layer is that it can combine multiple
> features into the simpler interface needed by the application. For
> instance, on Windows I might implement a "poll for input events" with
> WaitForMultipleObjects() and a couple of threads, but on POSIX I might
> implement the same feature with just ioctl(2) and poll(2). If those are
> the only threads I need then the threading stuff need not be portable
> anyway. Or I might even hide the threads altogether and provide a generic
> job queue for the application. That nicely covers up the differences
> between threading APIs.
>
> > Or, use an open source C11 threads implementation
>
> If you're porting a program that already makes heavy use of C11 threads
> then this is a reasonable way to go about it. You can just plug it in and
> be done with it. However, writing a platform layer with thread support
> isn't so complex as to warrant a dependency on a greenfield project.
>
> That being said, I'm impressed with TinyCThread. It's well-written and
> designed for embedding — i.e. ready to drop into any project. It's quite
> reasonable as far as dependencies go.
Details
Message ID
<20210926005739.ejx4oxw2pzwo3f33@nullprogram.com>
In-Reply-To
<CAKF5_Tkv7z=918T4jAKXXY1zR_V+yuZf0O9qLG9reZaz9-+-_g@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
> tutoring a windows user in programming! (while, admittedly, trying to 
> expose them to the unix-style toolchain via w64devkit)

This sounds interesting! I'm curious about your approach and ideas if 
you'd like to elaborate.
Harris Snyder
Details
Message ID
<CAKF5_T=kNAHZyW1rJV6ZFaKtWjWxqJ4Mxk-9cpXP+GYocXS2rA@mail.gmail.com>
In-Reply-To
<20210926005739.ejx4oxw2pzwo3f33@nullprogram.com> (view parent)
DKIM signature
pass
Download raw message
We are just starting, so we'll see if the plan survives contact with
reality, but my planned approach is roughly as follows. First
establish decent proficiency with C, using only the standard library
and self-written code, and then have a session where I introduce a
large range of topics like object oriented programming, sockets and
network programming, GUIs, graphics, databases, etc, and let the
student choose the order in which to cover them. Other languages would
be introduced as appropriate for those topics, but always with a nod
to how the same thing would be accomplished in C, since that's a good
way to understand what's under the hood. After covering a few of these
topics, I expect the student will start to form a clearer sense of
what he's interested in, and we'll reshape things in accordance with
that.

Along the way, there are some additions to existing open source
software that I need some help implementing, where most of what's
needed is a large volume of fairly simple work - things that would be
interesting challenges for someone who's learning, but become "just
work" once you're more experienced. The plan is to use those projects
as learning vehicles, so the student can see their work actually be
used by real people, even while still learning the ropes.

The basic theory underlying this is that the things you have to learn
in order to become proficient in C form the foundation for
understanding everything else in computer programming. There are
certain specializations that are exceptions to that, of course, but I
think in general it is true.

In terms of toolchain, right now it's just w64devkit, though I will
need to add git and an ssh client fairly soon. Part of the reason for
this choice, over visual studio, is because I think it's a good idea
to learn to be effective in just a terminal, since that way you can
work over ssh, etc, but also because I'm more comfortable with the
unix tools than with the windows ones, so it's easier for me to teach
that way.

--Harris

On Sat, Sep 25, 2021 at 8:57 PM Christopher Wellons
<wellons@nullprogram.com> wrote:
>
> > tutoring a windows user in programming! (while, admittedly, trying to
> > expose them to the unix-style toolchain via w64devkit)
>
> This sounds interesting! I'm curious about your approach and ideas if
> you'd like to elaborate.
Details
Message ID
<20210927221638.zkcdgg23yuyp2w7j@nullprogram.com>
In-Reply-To
<CAKF5_T=kNAHZyW1rJV6ZFaKtWjWxqJ4Mxk-9cpXP+GYocXS2rA@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
Sounds exciting! I like your approach, and I agree with your basic theory. 
Thanks for sharing the details.

> git and an ssh client fairly soon

As part of dogfooding and testing w64devkit, I've set up a development 
environment from scratch in a fresh install many times. The "Portable Git" 
distribution — a self-extracting 7zip archive — has been far superior to 
the standard installer. The installer tries to be "friendly" by asking 
lots of annoying questions that simply don't need to be asked, with more 
questions added from time to time. The portable distribution doesn't need 
any of this information, so obviously it's inessential. Like w64devkit, 
"uninstall" is just a matter of deleting one directory and leaves behind 
no mess.

I unzip the portable distribution to $HOME/git and add $HOME/git/cmd to my 
PATH. One way to do this, of course, is via .profile in w64devkit, though 
I personally set it in Windows so that cmd.exe prompts can see it, too.

Since I had missed this news myself for a long time: In case you didn't 
know, Windows 10 comes with an SSH client, so if you're on that version 
then you can already just "ssh" at any time.

Though building PuTTY from source using w64devkit could be an interesting 
exercise (it's not at all difficult). My first student — colleague by this 
point — had an issue where PuTTY was stealing a keybinding he wanted in 
Emacs. So he patched PuTTY, cross-compiled it, and has been using his 
custom version since. It was a perfect demonstration for how open source 
ought to work.
Reply to thread Export thread (mbox)