~rjarry/dlrepo

This thread contains a patchset. You're looking at the original emails, but you may wish to use the patch review UI. Review patch
5 3

[PATCH dlrepo 1/2] container: fix push/pull with podman

Details
Message ID
<20230331230016.235805-1-robin@jarry.cc>
DKIM signature
missing
Download raw message
Patch: +16 -6
podman uses the oci image formats. Make sure to handle them properly and
return the correct content type with manifest responses.

Signed-off-by: Robin Jarry <robin@jarry.cc>
---
 dlrepo/views/container.py | 22 ++++++++++++++++------
 1 file changed, 16 insertions(+), 6 deletions(-)

diff --git a/dlrepo/views/container.py b/dlrepo/views/container.py
index ed5c1fd4187f..94b07b38c581 100644
--- a/dlrepo/views/container.py
+++ b/dlrepo/views/container.py
@@ -3,6 +3,7 @@
# SPDX-License-Identifier: BSD-3-Clause

from http import HTTPStatus
import json

from aiohttp import web

@@ -61,8 +62,6 @@ class CatalogView(ContainerView):

# --------------------------------------------------------------------------------------
class ManifestReadOnlyView(ContainerView):
    CONTENT_TYPE = "application/vnd.docker.distribution.manifest.v2+json"

    @classmethod
    def urls(cls):
        yield "/v2/{branch}/{job}/manifests/{reference}"
@@ -103,15 +102,21 @@ class ManifestReadOnlyView(ContainerView):
        return path, digest

    async def head(self):
        self.get_manifest()
        return self.response()
        return await self.get()

    async def get(self):
        path, digest = self.get_manifest()
        try:
            manifest = json.loads(path.read_text())
            media_type = manifest["mediaType"]
        except (ValueError, KeyError) as e:
            raise errors.ManifestInvalid() from e
        if self.request.method == "HEAD":
            path = None
        return self.response(
            path=path,
            headers={
                "Content-Type": self.CONTENT_TYPE,
                "Content-Type": media_type,
                "Docker-Content-Digest": digest,
            },
        )
@@ -119,6 +124,11 @@ class ManifestReadOnlyView(ContainerView):

# --------------------------------------------------------------------------------------
class ManifestView(ManifestReadOnlyView):
    CONTENT_TYPES = (
        "application/vnd.oci.image.manifest.v1+json",
        "application/vnd.docker.distribution.manifest.v2+json",
    )

    @classmethod
    def urls(cls):
        yield "/v2/{branch}/{job}/manifests/{reference}"
@@ -140,7 +150,7 @@ class ManifestView(ManifestReadOnlyView):
            manifest = await self.request.json()
            if not isinstance(manifest, dict):
                raise ValueError()
            if manifest.get("mediaType") != self.CONTENT_TYPE:
            if manifest.get("mediaType") not in self.CONTENT_TYPES:
                raise errors.Unsupported()
            digest = self.registry().new_manifest(job, manifest)
        except PermissionError as e:
-- 
2.39.2

[PATCH dlrepo 2/2] container: fix http return codes for blob upload

Details
Message ID
<20230331230016.235805-2-robin@jarry.cc>
In-Reply-To
<20230331230016.235805-1-robin@jarry.cc> (view parent)
DKIM signature
missing
Download raw message
Patch: +21 -3
Fix this error when pushing a tag with podman:

 Error: writing blob: uploading layer to http://.../blobs/uploads/...:
 received unexpected HTTP status: 204 No Content

This is incorrect, that PUT request should return 201 created. docker
seems to not care about the return codes. podman does.

Also fix the return code for PATCH, DELETE and add the missing GET
handler.

Link: https://docs.docker.com/registry/spec/api/#put-blob-upload
Signed-off-by: Robin Jarry <robin@jarry.cc>
---
 dlrepo/views/container.py | 24 +++++++++++++++++++++---
 1 file changed, 21 insertions(+), 3 deletions(-)

diff --git a/dlrepo/views/container.py b/dlrepo/views/container.py
index 94b07b38c581..5712880a7bf4 100644
--- a/dlrepo/views/container.py
+++ b/dlrepo/views/container.py
@@ -231,6 +231,24 @@ class BlobsUploadsView(ContainerView):
        yield "/v2/{branch}/{job}/blobs/uploads/{uuid}"
        yield "/v2/u/{user}/{branch}/{job}/blobs/uploads/{uuid}"

    async def get(self):
        uuid = self.request.match_info["uuid"]
        try:
            path = self.repo().upload_path(uuid)
            data_size = path.stat().st_size
        except PermissionError as e:
            raise errors.Denied(str(e)) from e
        except FileNotFoundError as e:
            raise errors.BlobUploadUnknown() from e
        return self.response(
            status=HTTPStatus.NO_CONTENT,
            headers={
                "Location": self.request.path,
                "Range": f"0-{data_size - 1}",
                "Docker-Upload-Uuid": uuid,
            },
        )

    async def patch(self):
        uuid = self.request.match_info["uuid"]
        try:
@@ -240,7 +258,7 @@ class BlobsUploadsView(ContainerView):
        except FileNotFoundError as e:
            raise errors.BlobUploadUnknown() from e
        return self.response(
            status=HTTPStatus.NO_CONTENT,
            status=HTTPStatus.ACCEPTED,
            headers={
                "Location": self.request.path,
                "Range": f"0-{data_size - 1}",
@@ -265,7 +283,7 @@ class BlobsUploadsView(ContainerView):
        except FileNotFoundError as e:
            raise errors.BlobUploadUnknown() from e
        return self.response(
            status=HTTPStatus.NO_CONTENT,
            status=HTTPStatus.CREATED,
            headers={
                "Location": self.request.path,
                "Docker-Content-Digest": digest,
@@ -277,7 +295,7 @@ class BlobsUploadsView(ContainerView):
            self.repo().cancel_upload(self.request.match_info["uuid"])
        except FileNotFoundError as e:
            raise errors.BlobUploadUnknown() from e
        return self.response()
        return self.response(status=HTTPStatus.NO_CONTENT)


# --------------------------------------------------------------------------------------
-- 
2.39.2

[dlrepo/patches/.build.yml] build success

builds.sr.ht <builds@sr.ht>
Details
Message ID
<CRKXYGPV8314.3ACZUJ2E52H0D@cirno2>
In-Reply-To
<20230331230016.235805-2-robin@jarry.cc> (view parent)
DKIM signature
missing
Download raw message
dlrepo/patches/.build.yml: SUCCESS in 1m37s

[container: fix push/pull with podman][0] from [Robin Jarry][1]

[0]: https://lists.sr.ht/~rjarry/dlrepo/patches/40110
[1]: robin@jarry.cc

✓ #966198 SUCCESS dlrepo/patches/.build.yml https://builds.sr.ht/~rjarry/job/966198
Details
Message ID
<CAC2aAUTL1eqn5+O6=BrKAAs97a-Zxc4F_acDA++oGQ8ZWw5m=w@mail.gmail.com>
In-Reply-To
<20230331230016.235805-1-robin@jarry.cc> (view parent)
DKIM signature
missing
Download raw message
Le sam. 1 avr. 2023 à 01:00, Robin Jarry <robin@jarry.cc> a écrit :
>
> podman uses the oci image formats. Make sure to handle them properly and
> return the correct content type with manifest responses.

Tested for non-regression with docker.

> Signed-off-by: Robin Jarry <robin@jarry.cc>

Acked-by: Julien Floret <julien.floret@6wind.com>

Re: [PATCH dlrepo 2/2] container: fix http return codes for blob upload

Details
Message ID
<CAC2aAUSMQekZLQ90qZAfsCiTthFfbwMr9ampMoLW7-tJmp=+7g@mail.gmail.com>
In-Reply-To
<20230331230016.235805-2-robin@jarry.cc> (view parent)
DKIM signature
missing
Download raw message
Le sam. 1 avr. 2023 à 01:00, Robin Jarry <robin@jarry.cc> a écrit :
>
> Fix this error when pushing a tag with podman:
>
>  Error: writing blob: uploading layer to http://.../blobs/uploads/...:
>  received unexpected HTTP status: 204 No Content
>
> This is incorrect, that PUT request should return 201 created. docker
> seems to not care about the return codes. podman does.
>
> Also fix the return code for PATCH, DELETE and add the missing GET
> handler.
>

Tested for non regression with docker.

> Link: https://docs.docker.com/registry/spec/api/#put-blob-upload
> Signed-off-by: Robin Jarry <robin@jarry.cc>

Acked-by: Julien Floret <julien.floret@6wind.com>

Re: [PATCH dlrepo 2/2] container: fix http return codes for blob upload

Details
Message ID
<CROTXIT6QYB4.3HJWN586LXYW0@ringo>
In-Reply-To
<CAC2aAUSMQekZLQ90qZAfsCiTthFfbwMr9ampMoLW7-tJmp=+7g@mail.gmail.com> (view parent)
DKIM signature
missing
Download raw message
Julien Floret, Apr 04, 2023 at 14:23:
> Tested for non regression with docker.
>
> > Link: https://docs.docker.com/registry/spec/api/#put-blob-upload
> > Signed-off-by: Robin Jarry <robin@jarry.cc>
>
> Acked-by: Julien Floret <julien.floret@6wind.com>

Applied. Thanks!
Reply to thread Export thread (mbox)