~cnx/misc

This thread contains a patchset. You're looking at the original emails, but you may wish to use the patch review UI. Review patch
3 2

[PATCH fead v2] Add option to skip errors

Details
Message ID
<20230830041240.14065-1-huyngo@disroot.org>
DKIM signature
missing
Download raw message
Patch: +18 -3
Previous one would generate an error because I forgot to return

---
 src/fead.py | 21 ++++++++++++++++++---
 1 file changed, 18 insertions(+), 3 deletions(-)

diff --git a/src/fead.py b/src/fead.py
index 370b623..15bca4c 100755
--- a/src/fead.py
+++ b/src/fead.py
@@ -185,9 +185,19 @@ async def fetch(raw_url):
                        response.getheaders(), response)


async def fetch_all(urls):
async def fetch_skip_error(url):
    try:
        return await fetch(url)
    except Exception as e:
        warn(f'fail to fetch {url}: {e}', type('ErrorWarning', (Warning,), {}))


async def fetch_all(urls, skip_error):
    """Fetch all given URLs asynchronously and return them parsed."""
    tasks = gather(*map(fetch, urls))
    if skip_error:
        tasks = gather(*map(fetch_skip_error, urls))
    else:
        tasks = gather(*map(fetch, urls))
    try:
        return await tasks
    except:
@@ -234,11 +244,16 @@ def main():
    parser.add_argument('-o', '--output', metavar='PATH',
                        type=FileType('w'), default=stdout,
                        help='output file (default to stdout)')
    parser.add_argument('-s', '--skip-error', action='store_true',
                        default=False,
                        help="errors not causing failure but logged")
    args = parser.parse_args()

    template = args.template.read()
    args.template.close()
    for ad in select(args.count, (ad for feed in run(fetch_all(args.feeds))
    for ad in select(args.count, (ad for feed in run(fetch_all(args.feeds,
                                                               args.skip_error))
                                  if feed is not None
                                  for ad in select(args.per_feed, feed))):
        args.output.write(template.format(**truncate(ad, args.len)._asdict()))
    args.output.close()
-- 
2.40.1
Details
Message ID
<CV5LRIXZ7CUZ.A4FF627VB366@guix>
In-Reply-To
<20230830041240.14065-1-huyngo@disroot.org> (view parent)
DKIM signature
missing
Download raw message
> -async def fetch_all(urls):
> +async def fetch_skip_error(url):
> +    try:
> +        return await fetch(url)
> +    except Exception as e:
> +        warn(f'fail to fetch {url}: {e}', type('ErrorWarning', (Warning,), {}))

Please return None explicitly.  Would be nice if thou canst come up
with something less ironic than ErrorWarning but I can rename it
before application.

> +async def fetch_all(urls, skip_error):
>      """Fetch all given URLs asynchronously and return them parsed."""
> -    tasks = gather(*map(fetch, urls))
> +    if skip_error:
> +        tasks = gather(*map(fetch_skip_error, urls))
> +    else:
> +        tasks = gather(*map(fetch, urls))
>      try:
>          return await tasks

Please filter the None results here.
Details
Message ID
<9f519fa8d9600c10b63e9bcedb8f0915@disroot.org>
In-Reply-To
<CV5LRIXZ7CUZ.A4FF627VB366@guix> (view parent)
DKIM signature
missing
Download raw message
On 2023-08-30 11:42, Nguyễn Gia Phong wrote:
> Would be nice if thou canst come up
> with something less ironic than ErrorWarning but I can rename it
> before application.

How about FailureWarning huh? I already uses "fail" in the previous 
phrase.

(resent because previous one was not plain text and not delivered)
Details
Message ID
<CV5POWH8EKVZ.3HGB5LOOO6N9P@guix>
In-Reply-To
<9f519fa8d9600c10b63e9bcedb8f0915@disroot.org> (view parent)
DKIM signature
missing
Download raw message
On 2023-08-30 at 14:39+07:00, Huy wrote:
> On 2023-08-30 11:42, Nguyễn Gia Phong wrote:
> > Would be nice if thou canst come up
> > with something less ironic than ErrorWarning but I can rename it
> > before application.
>
> How about FailureWarning huh? I already uses "fail" in the previous 
> phrase.

Eh I have a very curesed idea of extracting the name from the exception.
I will implement the magic once I get home.

On 2023-08-30 at 14:39+07:00, Huy wrote:
> (resent because previous one was not plain text and not delivered)

It did LMFAO:

> Date: Wed, 30 Aug 2023 14:35:08 +0700
> Message-ID: <e2d2ecd29d6e3fe3b8e7de151f94aabf@disroot.org> 
Reply to thread Export thread (mbox)