~cadence/tube-devel

5 3

YouTube rate limiting

Details
Message ID
<C5V3YQIKSL4T.2RLPS41CAI88P@libricia-thinkcentre>
DKIM signature
pass
Download raw message
For the last few days, when I try to watch any video on
tube.cadence.moe, I get

```
Error
Could not extract video info. Instance is likely blocked.

RATE_LIMITED_BY_YOUTUBE

That error was generated by https://second.cadence.moe.

Watch on YouTube →
```

Shutting down second.cadence.moe so that users stop making requests
until your IP address gets unblocked might fix this in the short term.

In the long term, spreading the load across multiple instances of Second
and/or Invidious might prevent this from happening.  Having to switch
between different frontend instances creates friction for users
(in the form of migrating subscriptions, preferences, etc.), but unless
there is a reason CloudTube needs to be running on the same server as
Second, perhaps CloudTube instances could rotate between a list of
Second and/or Invidious instances as their backend to spread the load.
When a Second or Invidious instance gets blocked, CloudTube instances
could fallback to a non-blocked instance and avoid using the blocked
instance for a certain period of time to allow it to come back online.
Details
Message ID
<08cd8ca9-9140-4511-9707-c8eac2871b87@disroot.org>
In-Reply-To
<C5V3YQIKSL4T.2RLPS41CAI88P@libricia-thinkcentre> (view parent)
DKIM signature
pass
Download raw message
People can actually already manually change the backend Second or 
Invidious instance that the CloudTube frontend uses. Second and Invidious 
are almost completely compatible (you may notice small differences in 
number formatting, quality sorting, or missing video durations, but these 
can be addressed in the frontend). The instance that is used can be 
configured in the settings page. I plan to design a special rate limited 
error message page that explains the situation and directs the person to 
the settings page to select a new instance.

I also plan to add support for anti-captcha to Second so that it can 
automatically unblock itself.

Automatically switching instances may be possible, and I will consider 
this after I complete anti-captcha.
Details
Message ID
<C5YEM49C7G97.1RUXYF1EHOOA9@libricia-thinkcentre>
In-Reply-To
<08cd8ca9-9140-4511-9707-c8eac2871b87@disroot.org> (view parent)
DKIM signature
pass
Download raw message
On Sat Sep 26, 2020 at 4:43 PM PDT, Cadence Ember wrote:
> People can actually already manually change the backend Second or
> Invidious instance that the CloudTube frontend uses.

Nice!  I hadn't noticed that, but it's helpful.  Thanks.

> I plan to design a special rate limited error message page that
> explains the situation and directs the person to the settings page to
> select a new instance.

It might be a little smoother if from the error message page the user
could select and switch to a known working instance and immediately
start watching the video they were looking for using the new instance.
Cadence instances could have an internal list of known Invidious and
Second instances, and periodically check to see which are currently rate
limited by attempting to pull the information for a test video.

> I also plan to add support for anti-captcha to Second so that it can
> automatically unblock itself.

anti-captcha isn't automatic.  It is an online CAPTCHA sweatshop that
has human workers solve CAPTCHAs manually.  Some users and/or instance
admins may not be comfortable with this, so it would be nice to have
some alternative options as well.  Automatic instance switching to
spread the load across instances might reduce the frequency with which
instances get rate limited, and perhaps users could have the option of
occasionally solving a CAPTCHA or performing some action to attempt
getting an instance unblocked, similar to a script I recall being
prompted to run when encountering a blocked Bibliogram instance.

> Automatically switching instances may be possible, and I will consider
> this after I complete anti-captcha.

Thanks, and thanks for all your hard work.
Details
Message ID
<ca51f257-aa43-41b9-a1a6-d5b7957fe402@disroot.org>
In-Reply-To
<C5YEM49C7G97.1RUXYF1EHOOA9@libricia-thinkcentre> (view parent)
DKIM signature
pass
Download raw message
Great reply.

Invidious uses anti-captcha. I don't have anything else to say about the 
matter.
Details
Message ID
<C661GQHI7C2D.SURD2J38BTD6@libricia-thinkcentre>
In-Reply-To
<ca51f257-aa43-41b9-a1a6-d5b7957fe402@disroot.org> (view parent)
DKIM signature
pass
Download raw message
FYI, tube.cadence.moe is rate limited again.

I've thought of another suggestion which may be a little less ideal than
automatic instance rotation, but potentially simpler to implement:  In
addition to a default Second/Invidious instance, also define a fallback
instance to attempt when the default instance does not work (e.g. due to
rate limiting).  For now a relatively reliable Invidious instance like
invidious.snopyta.org might be a reasonable choice, and the user could
customize their fallback instance on the settings page like they can
with their default instance.
Cadence Ember
Details
Message ID
<20201018224930.617afbd8c04148b210f20fc3@vivaldi.net>
In-Reply-To
<08cd8ca9-9140-4511-9707-c8eac2871b87@disroot.org> (view parent)
DKIM signature
pass
Download raw message
I've added what I hope is a much more descriptive notice which
explains the situation and what users can do about the instance being
blocked.
Reply to thread Export thread (mbox)