~sircmpwn/sr.ht-discuss

9 2

[builds.sr.ht] Feature request: artifacts globbing

Details
Message ID
<20211115182358.2b524845@lambda>
DKIM signature
pass
Download raw message
Sometimes the name of an artifact can depend on some data such as the
project's version (eg. somepackage-1.0.deb).

In this case, when that data is updated, the build manifest has to be
updated too.

Other times, the exact number of artifacts may not even be known in
advance.

It'd be nice if the artifacts field allowed basic (* and ?) globbing,
so one could write:
  artifacts: somepackage-*.deb
Details
Message ID
<CFQJ388MVIVK.3HY7LN5U1HTTV@taiga>
In-Reply-To
<20211115182358.2b524845@lambda> (view parent)
DKIM signature
fail
Download raw message
DKIM signature: fail
This has been requested before. Are you not able to solve your problem
by, for example, creating a symlink from the inconsistent filename to a
more consistent path?
Details
Message ID
<20211115183518.321c31ce@lambda>
In-Reply-To
<CFQJ388MVIVK.3HY7LN5U1HTTV@taiga> (view parent)
DKIM signature
pass
Download raw message
Misclicked "Reply" instead of "reply all", sorry for the double mail

On Mon, 15 Nov 2021 18:27:15 +0100
"Drew DeVault" <sir@cmpwn.com> wrote:

> This has been requested before. Are you not able to solve your problem
> by, for example, creating a symlink from the inconsistent filename to
> a more consistent path?

That's what I'm doing, but there are two issues with it:

* The artifact gets renamed, and sometimes the name is important (like
  in the .deb example, having a versionless name is confusing)
* It isn't possible to produce a variable number of artifacts (this can
  be pretty useful for generic manifest files)
Details
Message ID
<CFR08GDS66KO.2QXOINF2JU3ES@taiga>
In-Reply-To
<20211115183518.321c31ce@lambda> (view parent)
DKIM signature
pass
Download raw message
I'm open to this idea, but I'm not in love with it. It will make things
more complicated.
Details
Message ID
<20211116120523.590e54c5@lambda>
In-Reply-To
<CFR08GDS66KO.2QXOINF2JU3ES@taiga> (view parent)
DKIM signature
pass
Download raw message
For reference both GitHub Actions and GitLab CI support it:

https://github.com/actions/upload-artifact#upload-using-a-wildcard-pattern
https://docs.gitlab.com/ee/ci/yaml/index.html#artifactspaths

Another option could be to allow specifying directories of artifacts.
Details
Message ID
<CFR5MEJE6X0R.2GJKLMOWDEQZO@taiga>
In-Reply-To
<20211116120523.590e54c5@lambda> (view parent)
DKIM signature
pass
Download raw message
On Tue Nov 16, 2021 at 12:05 PM CET, Francesco Gazzetta wrote:
> For reference both GitHub Actions and GitLab CI support it:
>
> https://github.com/actions/upload-artifact#upload-using-a-wildcard-pattern
> https://docs.gitlab.com/ee/ci/yaml/index.html#artifactspaths
>
> Another option could be to allow specifying directories of artifacts.

The main issue is finding a way of executing the glob pattern to find
the list of artifacts in a means which is portable across all of our
images.
Details
Message ID
<20211116154357.5aeb554b@lambda>
In-Reply-To
<CFR5MEJE6X0R.2GJKLMOWDEQZO@taiga> (view parent)
DKIM signature
pass
Download raw message
On Tue, 16 Nov 2021 12:06:42 +0100
"Drew DeVault" <sir@cmpwn.com> wrote:

> The main issue is finding a way of executing the glob pattern to find
> the list of artifacts in a means which is portable across all of our
> images.

Find patterns should be portable enough, though it's an extra package
to require for every distro.
Alternatively, sh could be used.

The only exception would be 9front, that needs walk and rc calls
instead, but I guess that image is already special cased?
Details
Message ID
<CFRAA750YL4T.30CBI9QPX0SFN@taiga>
In-Reply-To
<20211116154357.5aeb554b@lambda> (view parent)
DKIM signature
pass
Download raw message
On Tue Nov 16, 2021 at 3:43 PM CET, Francesco Gazzetta wrote:
> Find patterns should be portable enough, though it's an extra package
> to require for every distro.
> Alternatively, sh could be used.

tar + sh might work. Kind of messy.

> The only exception would be 9front, that needs walk and rc calls
> instead, but I guess that image is already special cased?

Don't worry about Plan 9 for this one.
Details
Message ID
<20211116162208.2f5e2c94@lambda>
In-Reply-To
<CFRAA750YL4T.30CBI9QPX0SFN@taiga> (view parent)
DKIM signature
pass
Download raw message
On Tue, 16 Nov 2021 15:45:51 +0100
"Drew DeVault" <sir@cmpwn.com> wrote:

> tar + sh might work. Kind of messy.

Yeah, making the runner extract the tar doesn't sound too good...
What about first obtaining the list of files though said sh call, then
transferring them one by one like it's done now with multiple artifacts?
Details
Message ID
<CFRB452F9RYO.CTF43PQH65Z1@taiga>
In-Reply-To
<20211116162208.2f5e2c94@lambda> (view parent)
DKIM signature
pass
Download raw message
On Tue Nov 16, 2021 at 4:22 PM CET, Francesco Gazzetta wrote:
> Yeah, making the runner extract the tar doesn't sound too good...
> What about first obtaining the list of files though said sh call, then
> transferring them one by one like it's done now with multiple artifacts?

That seems agreeable to me. Need to make sure that we correctly deal
with things like spaces in artifact filenames.
Reply to thread Export thread (mbox)