~sircmpwn/sr.ht-dev

This thread contains a patchset. You're looking at the original emails, but you may wish to use the patch review UI. Review patch
4 3

[PATCH meta.sr.ht] billing: PaymentIntents proof of concept

Details
Message ID
<20210530191556.1026511-1-tim@t8w.de>
DKIM signature
pass
Download raw message
Patch: +656 -106
DO NOT APPLY IN PRODUCTION!
Also, non-destructive database modifications ahead.

Requires the patch "billing: Expand sources to show payment methods".

Intended and implemented approach:
(Implemented for Stripe)

Add User.payment_processor attribute (which will eventually obsolete
User.stripe_customer), then add additional data structures for each
payment processor and a module that initiates the first payment
in-session and then has a function to be called every day after the next
payment is due and till it is paid.

First payment:
Client: Submit payment method to server
Server: Create customer, set payment method
Server: Create payment intent
Server: Return result to customer, change acc stat if successful
[if no action required]
  Client: Display result
[otherwise]
  Client: Handle action (e.g. credit cards 3d secure)
  [if unsuccessful]
    Client: Display result
  [otherwise]
    Client: Request acc stat update  << not implemented in demo
    Server: Check on intent, update acc stat
    Client: Display success
  [end]
[end]

Further payments (all on server):
Check daily if payment is due, if not continue
Initiate payment, unless already done
Check on payment (fail if initiated more than 2 weeks ago)
On success/failure: Update acc stat and due date and send mail

Result (may contain traces of bugs):

That approach works pretty well (though Stripes API makes this workflow
a bit cumbersome at times) and should be well adaptable to other
payment processors.

Requesting an account status update after successfully handling an
action was tested manually and worked fine. Adding/Changing/Removing
payment methods was not implemented, would be quite simple though. And,
atleast for Stripe, we can still use the exact same frontend code for
that.

Also, client-side has a lot more JS now and is right now a bit alerting
(also a bit more generalized for other payment processors). I don't
think that adds any additional requirements regarding browser support
than Stripe already adds, but with a lot of effor, that could be further
reduced.

Even then there are a lot of todos, quite a few of which are questions,
where I'd like some more thoughts[1] -- feedback is requested everywhere
of course, but there it's a bit more required ;)

Regarding implementing this properly: I don't know yet how much time I
will need for that / have much time I will have the next few months, so
this might be done in another few weekends or autumn, no idea.

[1]:
metasrht/blueprints/move_to_gql/__init__.py
40:    # todo: this transition makes it possible to remain indefinitely

metasrht/blueprints/move_to_gql/stripe.py
156:                # todo: provide following optional info?

metasrht/templates/new-payment.html
97:// todo: allow user to choose currency (unless its sepa, thats eur only)
114:  // todo: display amount accordingly (show only in terms selection?)
245:      name: "{{name}}",  // todo: right now thats the username -- banks might

metasrht/templates/billing.html
170:      <!-- todo: list given payment methods (sub-templates possible?) -->
---
 metasrht-daily                              |  13 +-
 metasrht/alembic/versions/7bcec2a5cfa4_.py  |  57 ++++
 metasrht/billing.py                         |   4 +-
 metasrht/blueprints/billing.py              |  84 +++---
 metasrht/blueprints/move_to_gql/__init__.py |  74 ++++++
 metasrht/blueprints/move_to_gql/stripe.py   | 226 ++++++++++++++++
 metasrht/templates/billing.html             |   1 +
 metasrht/templates/new-payment.html         | 271 +++++++++++++++-----
 metasrht/types/payment_processors.py        |  22 ++
 metasrht/types/user.py                      |  10 +-
 10 files changed, 656 insertions(+), 106 deletions(-)
 create mode 100644 metasrht/alembic/versions/7bcec2a5cfa4_.py
 create mode 100644 metasrht/blueprints/move_to_gql/__init__.py
 create mode 100644 metasrht/blueprints/move_to_gql/stripe.py
 create mode 100644 metasrht/types/payment_processors.py

diff --git a/metasrht-daily b/metasrht-daily
index 48d9a12..906032c 100755
--- a/metasrht-daily
+++ b/metasrht-daily
@@ -18,11 +18,22 @@ if cfg("meta.sr.ht::billing", "enabled") == "yes":
    print("Running billing")
    from metasrht.billing import charge_user, ChargeResult
    users = (User.query
        .filter(User.payment_cents != 0)
        .filter(User.payment_cents != 0)  # todo: user_type active_paying or active_provisional?
        .filter(User.payment_due < datetime.utcnow())
    ).all()
    ncharges = 0
    for user in users:
        # todo: call the payment processors handle_term function
        #       if PaymentTransition.successful, send success mail
        #       if PaymentTransition.failed, send failure mail
        #       otherwise, noop

        if user.id == 2:  # todo: remove manual test
            from metasrht.blueprints.move_to_gql.stripe import handle_term
            print(user, handle_term(user))
        continue


        print(f"Billing ~{user.username} ({ncharges+1}/{len(users)})")
        result, error = charge_user(user)
        db.session.commit()
diff --git a/metasrht/alembic/versions/7bcec2a5cfa4_.py b/metasrht/alembic/versions/7bcec2a5cfa4_.py
new file mode 100644
index 0000000..2fcaedf
--- /dev/null
+++ b/metasrht/alembic/versions/7bcec2a5cfa4_.py
@@ -0,0 +1,57 @@
"""empty message

Revision ID: 7bcec2a5cfa4
Revises: 3e4e74262480
Create Date: 2021-05-29 11:03:11.409112

"""

# revision identifiers, used by Alembic.
revision = '7bcec2a5cfa4'
down_revision = '3e4e74262480'

from alembic import op
import sqlalchemy as sa
import sqlalchemy_utils as sau
from metasrht.types.user import PaymentProcessor

def upgrade():
    op.create_table('payment_processor_stripe_customers',
        sa.Column('user_id', sa.Integer,
            sa.ForeignKey('user.id'), primary_key=True),
        sa.Column('customer', sa.String(256), nullable=False))

    op.create_table('payment_processor_stripe_payments',
        sa.Column('intent_id', sa.String(256), primary_key=True),
        sa.Column('user_id', sa.Integer, sa.ForeignKey('user.id')),
        sa.Column('created', sa.DateTime),
        sa.Column('due', sa.DateTime),
        sa.Column('done', sa.Boolean))

    op.add_column('user', sa.Column(
        'payment_processor',
        sau.ChoiceType(PaymentProcessor, impl=sa.String()),
        nullable=False,
        server_default=PaymentProcessor.none.value))

    op.execute('''
        update "user"
        set payment_processor = 'stripe'
        where stripe_customer is not null''')
    op.execute('''
        insert into payment_processor_stripe_customers
        select id, stripe_customer
        from "user"
        where stripe_customer is not null''')

    # todo: drop user.stripe_customer


def downgrade():
    op.drop_column('user', 'payment_processor')

    # todo: add user.stripe_customer
    # todo: fill data

    # op.drop_table('payment_processor_stripe_customers')
    # op.drop_table('payment_processor_stripe_payments')
diff --git a/metasrht/billing.py b/metasrht/billing.py
index 23e7e2f..b1d3579 100644
--- a/metasrht/billing.py
+++ b/metasrht/billing.py
@@ -29,7 +29,6 @@ def charge_user(user):
        amount = user.payment_cents
        if user.payment_interval == PaymentInterval.yearly:
            amount = amount * 10 # Apply yearly discount
        # TODO: Multiple currencies
        charge = stripe.Charge.create(
            amount=amount,
            currency="usd",
@@ -56,7 +55,8 @@ def charge_user(user):
        invoice.source = charge.source.stripe_id
    db.session.add(invoice)
    if user.payment_interval == PaymentInterval.monthly:
        invoice.valid_thru = datetime.utcnow() + timedelta(days=30)
        invoice.valid_thru = datetime.utcnow() + timedelta(
            days=30, hours=10, minutes=30)
        user.payment_due = invoice.valid_thru
    else:
        invoice.valid_thru = datetime.utcnow()
diff --git a/metasrht/blueprints/billing.py b/metasrht/blueprints/billing.py
index 687c5fe..a51e0cc 100644
--- a/metasrht/blueprints/billing.py
+++ b/metasrht/blueprints/billing.py
@@ -1,12 +1,12 @@
import stripe
from datetime import datetime, timedelta
from flask import Blueprint, render_template, request, redirect
from flask import url_for, abort, Response
from jinja2 import escape
from metasrht.audit import audit_log
from metasrht.billing import charge_user
from metasrht.types import User, UserType, PaymentInterval, Invoice
from metasrht.types import User, UserType, PaymentInterval, Invoice, PaymentProcessor
from metasrht.webhooks import deliver_profile_update
from metasrht.blueprints import move_to_gql
from sqlalchemy import and_
from srht.config import cfg
from srht.database import db
@@ -25,10 +25,7 @@ def billing_GET():
    if message:
        del session["message"]
    customer = None
    if current_user.stripe_customer:
        customer = stripe.Customer.retrieve(
            current_user.stripe_customer,
            expand=["sources"])
    # todo: get payment methods from payment processor
    total_users = (User.query
            .filter(User.user_type != UserType.unconfirmed)
            .filter(User.user_type != UserType.suspended)).count()
@@ -71,17 +68,18 @@ def billing_initial_POST():
        return "Invalid form submission", 400
    current_user.payment_cents = amount
    db.session.commit()
    if current_user.stripe_customer:
    if current_user.payment_processor != PaymentProcessor.none:
        return redirect(url_for("billing.billing_chperiod_GET"))
    return redirect(url_for("billing.new_payment_GET"))

@billing.route("/billing/change-period")
@loginrequired
def billing_chperiod_GET():
    if not current_user.stripe_customer:
    if current_user.payment_processor == PaymentProcessor.none:
        return redirect(url_for("billing.new_payment_GET"))
    return render_template("billing-change-period.html")

# todo: update
@billing.route("/billing/change-period", methods=["POST"])
def billing_chperiod_POST():
    if not current_user.stripe_customer:
@@ -107,51 +105,45 @@ def new_payment_GET():
    if not current_user.payment_cents:
        return redirect(url_for("billing.billing_initial_GET"))
    return render_template("new-payment.html",
            amount=current_user.payment_cents)
            amount=current_user.payment_cents,
            name=current_user.username,
            email=current_user.email)


@billing.route("/billing/new-payment", methods=["POST"])
@loginrequired
def new_payment_POST():
    valid = Validation(request)
    term = valid.require("term")
    token = valid.require("stripe-token")
    processor = valid.require("processor")
    if not valid.ok:
        return "Invalid form submission", 400
    if not current_user.stripe_customer:
        new_customer = True
        try:
            customer = stripe.Customer.create(
                    description="~" + current_user.username,
                    email=current_user.email,
                    card=token)
            current_user.stripe_customer = customer.id
            current_user.payment_due = datetime.utcnow() + timedelta(minutes=-5)
        except stripe.error.CardError as e:
            details = e.json_body["error"]["message"]
            return render_template("new-payment.html",
                    amount=current_user.payment_cents, error=details)
    else:
        new_customer = False
        if current_user.user_type != UserType.active_paying:
            current_user.payment_due = datetime.utcnow() + timedelta(minutes=-5)
        try:
            customer = stripe.Customer.retrieve(current_user.stripe_customer)
            source = customer.sources.create(source=token)
            customer.default_source = source.stripe_id
            customer.save()
        except stripe.error.CardError as e:
            details = e.json_body["error"]["message"]
            return render_template("new-payment.html",
                    amount=current_user.payment_cents, error=details)
    audit_log("billing", "New payment method handed")
        return {"error": "Invalid form submission"}

    current_user.payment_processor = processor

    # Ensure that the billing term starts before the first payment is created.
    current_user.payment_due = datetime.utcnow() - timedelta(seconds=5)

    current_user.payment_interval = PaymentInterval(term)
    success, details = charge_user(current_user)
    if not success:
        return render_template("new-payment.html",
                amount=current_user.payment_cents, error=details)
    db.session.commit()
    freshen_user()
    deliver_profile_update(current_user)

    # todo: add address to redirect client-side after success

    if processor == "stripe":
        method = valid.require("method")
        if not valid.ok:
            return {"error": "Payment method not set"}
        payment_initiated, retval = move_to_gql.stripe.init_payment(
            current_user, method)
    else:
        return {"error": "Unknown payment processor"}

    if payment_initiated:
        audit_log("billing", "New payment method handed")
        freshen_user()
        deliver_profile_update(current_user)

    return retval

    return_to = session.pop("return_to", None)
    if return_to:
@@ -161,6 +153,7 @@ def new_payment_POST():
    session["message"] = "Your payment method was updated."
    return redirect(url_for("billing.billing_GET"))

# todo: update
@billing.route("/billing/remove-source/<source_id>", methods=["POST"])
@loginrequired
def payment_source_remove(source_id):
@@ -173,6 +166,7 @@ def payment_source_remove(source_id):
    session["message"] = "Your payment method was removed successfully."
    return redirect(url_for("billing.billing_GET"))

# todo: update
@billing.route("/billing/set-default-source/<source_id>", methods=["POST"])
@loginrequired
def payment_source_make_default(source_id):
@@ -202,6 +196,7 @@ def cancel_POST():
    audit_log("billing", "Plan cancelled (will not renew)")
    return redirect(url_for("billing.billing_GET"))

# todo: update
@billing.route("/billing/invoice/<int:invoice_id>")
@loginrequired
def invoice_GET(invoice_id):
@@ -213,6 +208,7 @@ def invoice_GET(invoice_id):
        abort(401)
    return render_template("billing-invoice.html", invoice=invoice)

# todo: update
@billing.route("/billing/invoice/<int:invoice_id>", methods=["POST"])
@loginrequired
def invoice_POST(invoice_id):
diff --git a/metasrht/blueprints/move_to_gql/__init__.py b/metasrht/blueprints/move_to_gql/__init__.py
new file mode 100644
index 0000000..180b0b0
--- /dev/null
+++ b/metasrht/blueprints/move_to_gql/__init__.py
@@ -0,0 +1,74 @@
from datetime import datetime, timedelta
from enum import Enum
from metasrht.types import User, UserType, PaymentInterval, Invoice
from metasrht.blueprints.move_to_gql import stripe
from srht.database import db


def get_amount(user):
    amount = user.payment_cents
    if user.payment_interval == PaymentInterval.yearly:
        amount = amount * 10
    return amount

class PaymentStatus(Enum):
    unchanged = "unchanged"
    successful = "successful"
    processing = "processing"
    failed = "failed"

ut = UserType
ps = PaymentStatus

# {(status, transition): new_status}
# non existent or None transitions keep the current status
status_transitions = {
    # todo: uncofirmed users can't setup payments, right? otherwise add
    #       transitions.

    ("admin",     "*"): None,
    ("unknown",   "*"): None,
    ("suspended", "*"): None,

    ("*", ps.successful): ut.active_paying,

    (ut.active_provisional, ps.failed): ut.active_delinquent,
    (ut.active_paying,      ps.failed): ut.active_delinquent,

    (ut.active_non_paying, ps.processing): ut.active_provisional,

    # todo: this transition makes it possible to remain indefinitely
    #       provisional by adding new, non-working payment methods. suspend
    #       users who are provisional and made no payment in the last n weeks?
    #       remove this transition? if choosing the latter then you can't
    #       easily get your sr.ht account working again after switching bank
    #       accounts or whatever.
    (ut.active_delinquent, ps.processing): ut.active_provisional,
}


def status_update(user, transition):
    old_status = user.user_type

    # Use False as sentinel
    status = status_transitions.get((old_status, transition), False)
    if status is False:
        status = status_transitions.get((old_status, "*"), False)
    if status is False:
        status = status_transitions.get(("*", transition), False)
    if not status:
        return

    user.user_type = status

    if (
            transition == PaymentStatus.successful
            and user.payment_due < datetime.utcnow()):  # todo: not that great
        days = 365.2425  # Mean year
        if user.payment_interval == PaymentInterval.monthly:
            days /= 12
        user.payment_due += timedelta(days=days)

    print(user.user_type, user.payment_due)

    db.session.commit()
diff --git a/metasrht/blueprints/move_to_gql/stripe.py b/metasrht/blueprints/move_to_gql/stripe.py
new file mode 100644
index 0000000..a8b9db3
--- /dev/null
+++ b/metasrht/blueprints/move_to_gql/stripe.py
@@ -0,0 +1,226 @@
# Credit card test payment
#
# Use a test card number to try your integration. These card numbers work in
# test mode with any CVC, postal code, and future expiry date. Stripe also has
# a set of international test cards to test specific postal code formats (e.g.
# only allow numerical values for U.S. zip codes).
#
# 4242 4242 4242 4242  Payment succeeds immediately.
# 4000 0025 0000 3155  Payment requires authentication.
# 4000 0000 0000 9995  Payment is immediately declined.
#
# SEPA test payment
#                         From processing to...
# DE89370400440532013000  succeeded.
# DE08370400440532013003  succeeded after three minutes.
# DE62370400440532013001  requires_payment_method.
# DE78370400440532013004  requires_payment_method after three minutes.
# DE35370400440532013002  succeeded, but a dispute is immediately created.
#


import stripe
from datetime import datetime, timedelta
from srht.config import cfg
from srht.database import db
from metasrht.blueprints import move_to_gql
from srht.database import DbSession
from metasrht.types.payment_processors import (
    PaymentProcessorStripeCustomers, PaymentProcessorStripePayments)


def init_payment(user, method):
    """Initialize payment processor and create a first payment.

    If called again will request another payment if the first one failed.

    May only be used on-session due to it allowing for actions which the client
    must handle (e.g. 3d secure for credit cards).

    Args:
        user: sr.ht user
        method: stripe payment method

    Returns:
        Tuple[PaymentStatus, Dict]: The payment status / data to
            return to the client to be handled by stripe.
    """

    customer = PaymentProcessorStripeCustomers.query.get(user.id)
    if customer:
        customer = customer.customer
    else:
        customer = stripe.Customer.create(
                description="~" + user.username,
                email=user.email,
                payment_method=method,  # Necessary to set as default.
                invoice_settings={"default_payment_method": method}
                )
        customer_entry = PaymentProcessorStripeCustomers()
        customer_entry.user = user
        customer_entry.customer = customer.id
        db.session.commit()

    return _create_intent(
        user,
        customer=customer,
        method=method,
        fail_on_action=False)

def handle_term(user):
    """Handles the term payment.

    Must be called regularly (e.g. daily via cronjob) till the users due date
    is changed to the next term or the users type is changed to delinquent.

    (Either creates a new payment or checks if a payment exists) and reacts to
    that status.

    Args:
        user: sr.ht user

    Returns:
        PaymentStatus
    """

    return _create_intent(user)[0]

def _create_intent(user, customer=None, method=None, fail_on_action=True):
    """
    (Either creates a new payment intent or checks if a payment intent exists)
    and reacts to that status.

    Args:
        user: sr.ht user
        customer: Optional stripe customer or customer id.
        method: Optional stripe payment method or payment method id.

    Returns:
        Tuple[Optional[PaymentStatus], Dict]: The payment status (None if no) /
            data to return to the client to be handled by stripe.
    """

    if user.payment_due and user.payment_due > datetime.utcnow():
        return (move_to_gql.PaymentStatus.unchanged, {"error": "Not due."})

    payment = (PaymentProcessorStripePayments.query
        .filter(PaymentProcessorStripePayments.user_id == user.id)
        .filter(PaymentProcessorStripePayments.created > user.payment_due)
    ).first()

    if payment:
        _check_on_intent(user, payment)
        if payment.done:
            return (move_to_gql.PaymentStatus.unchanged, {
                "error": "Paid already."})
        return (
            move_to_gql.PaymentStatus.unchanged,
            {"error": "Another payment is processing and can't be cancelled."})

    amount = move_to_gql.get_amount(user)

    if customer is None or isinstance(customer, str):
        customer = stripe.Customer.retrieve(
            customer
            or PaymentProcessorStripeCustomers.query.get(user.id).customer,
            expand=["invoice_settings.default_payment_method"])
        if method is None or isinstance(method, str):
            method = customer.invoice_settings.default_payment_method
    if method is None or isinstance(method, str):
        method = stripe.PaymentMethod.retrieve(
            method
            or customer.invoice_settings.default_payment_method
            or customer.default_source  # todo: remove when all on new api
            )

    intent_data = {
        "amount": amount,
        "confirm": True,
        "customer": customer,
        "description": customer.description,
        "payment_method": method.id,
        "setup_future_usage": "off_session"
    }

    if method.type == "card":
        intent_data["currency"] = "USD"  # todo: allow other currencies
        intent_data["payment_method_types"] = ["card"]

    elif method.type == "sepa_debit":
        intent_data["currency"] = "EUR"
        intent_data["payment_method_types"] = ["sepa_debit"]
        intent_data["mandate_data"] = {
            "customer_acceptance": {
                "type": "offline",

                # todo: provide following optional info?
                #
                # accepted_at: time, when the mandate was acceptedn (right now)
                # online: {ip_address:, user_agent:}
                #
                # Stripe gets (could get) that data anyway and it might make
                # the customers bank more likely to accept instead of asking
                # the customer for confirmation.
                #
                # Based on personal experience: That happens quite seldom in
                # legitimate cases (read: once in the last decade for a pretty
                # suspicious double booking). Though I don't know if this
                # changed / will change with stricter implementations regarding
                # Strong Customer Authentication.
            }
        }

    else:
        return (False, {"error": "Unknown payment method"})

    intent = stripe.PaymentIntent.create(**intent_data)

    print(intent)  # todo: remove (audit log here?)
    payment = PaymentProcessorStripePayments()
    payment.intent_id = intent.id
    payment.created = datetime.utcnow()
    payment.due = payment.created + timedelta(weeks=2)
    payment.done = intent.status == "succeeded"
    payment.user = user
    db.session.commit()

    if intent.status == "requires_action" and not fail_on_action:
        return (move_to_gql.PaymentStatus.unchanged, {
            "status": "requires_action",
            "client_secret": intent.client_secret})

    _check_on_intent(user, payment, intent=None)

    if intent.status == "processing":
        return (move_to_gql.PaymentStatus.processing, {"status": "processing"})
    if intent.status == "requires_payment_method":  # failed
        return (move_to_gql.PaymentStatus.failed, {
            "status": "requires_payment_method"})
    if intent.status == "succeeded":
        return (move_to_gql.PaymentStatus.successful, {"status": "succeeded"})

    return (move_to_gql.PaymentStatus.failed, {"error": "Unexpected result"})

# todo: implement removing/adding/changing payment methods

def _check_on_intent(user, payment, intent=None):
    if not intent or isinstance(intent, str):
        intent = stripe.PaymentIntent.retrieve(intent or payment.intent_id)

    if intent.status == "succeeded":
        payment.done = True
        db.session.commit()
        status = move_to_gql.PaymentStatus.successful
    elif intent.status == "processing":
        if payment.due < datetime.utcnow:
            # todo: add the final status to the payment too?
            status = move_to_gql.PaymentStatus.failed
            payment.done = True
            db.session.commit()
        else:
            status = move_to_gql.PaymentStatus.processing
    else:
        status = move_to_gql.PaymentStatus.failed
    move_to_gql.status_update(user, status)

    return status
diff --git a/metasrht/templates/billing.html b/metasrht/templates/billing.html
index 60f05bd..882b1dd 100644
--- a/metasrht/templates/billing.html
+++ b/metasrht/templates/billing.html
@@ -167,6 +167,7 @@
    ] %}
    <h3>Payment methods</h3>
    <div class="event-list">
      <!-- todo: list given payment methods (sub-templates possible?) -->
      {% for source in customer.sources %}
      <div class="event row">
        <div class="col-md-8">
diff --git a/metasrht/templates/new-payment.html b/metasrht/templates/new-payment.html
index bb05363..198cc53 100644
--- a/metasrht/templates/new-payment.html
+++ b/metasrht/templates/new-payment.html
@@ -27,21 +27,8 @@
        complete and the rest of the site will work normally.
      </div>
    </noscript>
    <form method="POST" id="payment-form" style="display: none">
    <form id="payment-form">
      {{csrf_token()}}
      <div class="form-group">
        <label for="card-element" style="font-weight: bold">
          Payment details
        </label>
        <div id="card-element" class="form-control"></div>
        <div id="card-error" class="invalid-feedback">
        </div>
      </div>
      {% if error %}
      <div class="alert alert-danger">
        {{error}}
      </div>
      {% endif %}
      <fieldset style="margin-bottom: 1rem">
        <legend style="font-weight: bold">Payment term</legend>
        <div class="form-check form-check-inline">
@@ -68,62 +55,230 @@
          </label>
        </div>
      </fieldset>
      <input type="hidden" name="stripe-token" id="stripe-token" />
      <div class="form-group">
        {% if current_user.user_type == UserType.active_paying %}
        <button class="btn btn-primary" type="submit">
        <label for="payment-method" style="font-weight: bold">
          Payment method
        </label>
        <select id="payment-method" class="form-control">
          <option disabled selected hidden>Select a payment method</option>
          <option value="stripe-card">Credit Card</option>
          <option value="stripe-iban">IBAN</option>
        </select>
      </div>
      <div id="payment-details" class="form-group" style="display: none">
        <label for="card-element" style="font-weight: bold">
          Payment details
        </label>
        <div id="stripe-element" class="form-control payment-element"></div>
        <div id="error" class="invalid-feedback">
        </div>
      </div>
      {% if error %}
      <div class="alert alert-danger">
        {{error}}
      </div>
      {% endif %}
      <div class="form-group">
        <button id="submit" class="btn btn-primary" type="submit" disabled>
          {% if current_user.user_type == UserType.active_paying %}
          Add payment method
          {{icon('caret-right')}}
        </button>
        {% else %}
        <button class="btn btn-primary" type="submit">
          {% else %}
          Submit payment
          {% endif %}
          {{icon('caret-right')}}
        </button>
        {% endif %}
      </div>
      <p>
        Your payment is securely processed with
        <a href="https://stripe.com/">Stripe</a> over an encrypted connection.
        Your credit card details are never sent
        to {{cfg("sr.ht", "site-name")}} servers.
      </p>
    </form>
  </div>
</div>
<script src="https://js.stripe.com/v3/?advancedFraudSignals=false"></script>
<script>
document.getElementById('payment-form').style.display = 'block';
var stripe = Stripe('{{cfg("meta.sr.ht::billing", "stripe-public-key")}}');
var elements = stripe.elements();
var amount = {{amount}};
var card = elements.create('card');
card.mount('#card-element');
card.addEventListener('change', function(event) {
  var displayError = document.getElementById('card-error');
  var cardElement = document.getElementById('card-element');
  if (event.error) {
    displayError.textContent = event.error.message;
    cardElement.classList.add('is-invalid');
  } else {
    displayError.textContent = '';
    cardElement.classList.remove('is-invalid');

// todo: allow user to choose currency (unless its sepa, thats eur only)

function set_error(error) {
  document.getElementById("error").textContent = error ? error : ""
}

function set_submit_enabled(value) {
  document.getElementById("submit").disabled = !value
}

function submitting(value) {
  console.log(value)
  set_submit_enabled(!value)
  // todo: show loading indicator if true, hide otherwise
}

function set_currency(currency) {
  // todo: display amount accordingly (show only in terms selection?)
}

function submit(processor, data) {
  let form = new FormData()
  form.append(
    "_csrf_token", document.getElementsByName("_csrf_token")[0].value)
  form.append("term", term)
  form.append("processor", processor)
  Object.keys(data).forEach(key => form.append(key, data[key]))
  return fetch("/billing/new-payment", {method: "POST", body: form})
}

function stripe_method_changed(method) {
  let container = document.getElementById("stripe-element")
  let stripe = processors.stripe

  if(!stripe.elements)
    stripe.elements = stripe.stripe.elements()

  if(!stripe.created_elements[method]) {
    let element = stripe.elements.create(method, stripe.elements_args[method])
    stripe.created_elements[method] = element

    element.addEventListener('change', evt => {
      set_error(evt.error ? evt.error.message : "")
      if(evt.error)
        container.classList.add("is-invalid")
      else
        container.classList.remove("is-invalid")
      set_submit_enabled(evt.complete)
    })
  }

  stripe.current = {
    "type": method == "iban" ? "sepa_debit" : method,
    "element": stripe.created_elements[method]}
  stripe.current.element.mount(container)
}

function stripe_term_changed(term) {
  // noop
}

function stripe_other_method_selected() {
  set_error("")
  let stripe = processors.stripe
  if(stripe.current)
    stripe.current.element.unmount()
  stripe.current = null
  document.getElementById("stripe-element").innerHTML = ""
}

function stripe_submit() {
  submitting(true)

  let stripe = processors.stripe
  stripe.stripe.createPaymentMethod({
    "type": stripe.current.type,
    [stripe.current.type]: stripe.current.element,
    "billing_details": stripe.billing_details
  }).then(result => { // todo: doesn't js has async/await too? browser support?
    if(result.error) // todo: handle error properly
      alert(result.error.message)
    else {
      submit("stripe", {method: result.paymentMethod.id})
      .then(r => r.json())
      .then(stripe_result)
    }
  })
}

function stripe_result(result) { // todo: handle results properly
  if(result.error) {
    alert(result.error)
    return
  }
});
var form = document.getElementById('payment-form');
form.addEventListener('submit', function(e) {
  e.preventDefault();
  stripe.createToken(card).then(function(result) {
    if (result.error) {
      var errorElement = document.getElementById('card-error');
      var cardElement = document.getElementById('card-element');
      errorElement.textContent = result.error.message;
      cardElement.classList.add('is-invalid');
    } else {
      document.getElementById('stripe-token').value = result.token.id;
      form.submit();

  if(result.status == "succeeded") {
    alert("Your payment was successful! Your account is now activated.")
    return
  }

  if(result.status == "processing") {
    alert(
      "Your payment is successfully processing! Your account is "
      +"provisional for now and will be activated when your payment is done.")
    return
  }

  if(result.status == "requires_action") {
    let confirmPayment = processors.stripe.stripe.confirmCardPayment
    if(processors.stripe.current.type == "sepa_debit") {
      // todo: is this even possible for sepa?
      confirmPayment = processors.stripe.stripe.confirmSepaDebitPayment
    }
  });
});

    confirmPayment(result.client_secret)
    .then(result => {
      if(result.error) {
        alert(result.error.message)
        return
      }
      console.log(result)
      alert("")
      // todo: ask server to verify intents success and activate account
      //       only then show success
    })
  }

  if(result.status == "requires_payment_method") {
    alert("Your payment failed! Verify your data or choose another method.")
    return
  }
}

let term = "monthly"
let amount = {
  usd: {{amount}},
  eur: {{amount}}*0.82 // todo: provide current rate via template
}

let processors = {
  stripe: {
    method_changed: stripe_method_changed,
    term_changed: stripe_term_changed,
    other_method_selected: stripe_other_method_selected,
    submit: stripe_submit,

    stripe: Stripe('{{cfg("meta.sr.ht::billing", "stripe-public-key")}}'),
    billing_details: {
      name: "{{name}}",  // todo: right now thats the username -- banks might
      email: "{{email}}" //       require the account holders name.
    },
    elements_args: {
      card: {},
      iban: {
        supportedCountries: ["SEPA"]
      }
    },
    created_elements: {}
  },
  current: null
}

Array.from(document.getElementsByName("term")).forEach(el => {
  el.addEventListener("change", evt => {
    term = el.value
    if(processors.current)
      processors.current.term_changed(term)
  })
})

document.getElementById("payment-method").addEventListener("change", evt => {
  if(processors.current)
    processors.current.other_method_selected()

  document.getElementById("payment-details").style.display = "block"

  let values = document.getElementById("payment-method").value.split("-")
  processors.current = processors[values[0]]
  processors.current.method_changed(values[1])
})

document.getElementById("submit").addEventListener("click", evt => {
  evt.preventDefault()
  processors.current.submit()
})

</script>
{% endblock %}
diff --git a/metasrht/types/payment_processors.py b/metasrht/types/payment_processors.py
new file mode 100644
index 0000000..9c4674b
--- /dev/null
+++ b/metasrht/types/payment_processors.py
@@ -0,0 +1,22 @@
import sqlalchemy as sa
from srht.database import Base
from enum import Enum


# todo: updated (/created columns) ?

class PaymentProcessorStripeCustomers(Base):
    __tablename__ = 'payment_processor_stripe_customers'
    user_id = sa.Column(sa.Integer, sa.ForeignKey('user.id'), primary_key=True)
    user = sa.orm.relationship("User", backref="payment_data_stripe")
    customer = sa.Column(sa.String(256))


class PaymentProcessorStripePayments(Base):
    __tablename__ = 'payment_processor_stripe_payments'
    intent_id = sa.Column(sa.String(256), primary_key=True)
    user_id = sa.Column(sa.Integer, sa.ForeignKey('user.id'))
    user = sa.orm.relationship("User", backref="payment_processor_stripe_payments")
    created = sa.Column(sa.DateTime)
    due = sa.Column(sa.DateTime)
    done = sa.Column(sa.Boolean)
diff --git a/metasrht/types/user.py b/metasrht/types/user.py
index 59015e2..ea4967e 100644
--- a/metasrht/types/user.py
+++ b/metasrht/types/user.py
@@ -17,6 +17,11 @@ class UserNote(Base):
    user = sa.orm.relationship('User', backref=sa.orm.backref('notes'))
    note = sa.Column(sa.Unicode())

class PaymentProcessor(Enum):
    none = "none"
    admin = "admin"
    stripe = "stripe"

class PaymentInterval(Enum):
    monthly = "monthly"
    yearly = "yearly"
@@ -32,7 +37,10 @@ class User(Base, UserMixin):
    reset_expiry = sa.Column(sa.DateTime())
    invites = sa.Column(sa.Integer, server_default='0')
    "Number of invites this user can send"
    stripe_customer = sa.Column(sa.String(256))
    payment_processor = sa.Column(
            sau.ChoiceType(PaymentProcessor, impl=sa.String()),
            nullable=False,
            default=PaymentProcessor.none)
    payment_cents = sa.Column(
            sa.Integer, nullable=False, server_default='0')
    payment_interval = sa.Column(
-- 
2.31.1

[meta.sr.ht/patches] build failed

builds.sr.ht
Details
Message ID
<CBQTKVLJ0K8R.2EHKZY0VMGJNQ@cirno>
In-Reply-To
<20210530191556.1026511-1-tim@t8w.de> (view parent)
DKIM signature
missing
Download raw message
meta.sr.ht/patches: FAILED in 27s

[billing: PaymentIntents proof of concept][0] from [Tim Hallmann][1]

[0]: https://lists.sr.ht/~sircmpwn/sr.ht-dev/patches/23100
[1]: mailto:tim@t8w.de

✗ #516123 FAILED meta.sr.ht/patches/alpine.yml    https://builds.sr.ht/~sircmpwn/job/516123
✗ #516125 FAILED meta.sr.ht/patches/debian.yml    https://builds.sr.ht/~sircmpwn/job/516125
✗ #516124 FAILED meta.sr.ht/patches/archlinux.yml https://builds.sr.ht/~sircmpwn/job/516124
Details
Message ID
<CBYEVD1UNK40.1UCOS0ZUQ44K3@taiga>
In-Reply-To
<20210530191556.1026511-1-tim@t8w.de> (view parent)
DKIM signature
fail
Download raw message
DKIM signature: fail
This looks like a good start/research prototype. The real one will have
to be implemented closer to the model I explained earlier, with the bulk
of the work taking place on Go on the GQL side (you already know this
given the name of your blueprint, but worth clarifying for posterity).

On Sun May 30, 2021 at 3:15 PM EDT, Tim Hallmann wrote:
> diff --git a/metasrht/billing.py b/metasrht/billing.py
> index 23e7e2f..b1d3579 100644
> --- a/metasrht/billing.py
> +++ b/metasrht/billing.py
> @@ -29,7 +29,6 @@ def charge_user(user):
> amount = user.payment_cents
> if user.payment_interval == PaymentInterval.yearly:
> amount = amount * 10 # Apply yearly discount
> - # TODO: Multiple currencies
> charge = stripe.Charge.create(
> amount=amount,
> currency="usd",
> @@ -56,7 +55,8 @@ def charge_user(user):
> invoice.source = charge.source.stripe_id
> db.session.add(invoice)
> if user.payment_interval == PaymentInterval.monthly:
> - invoice.valid_thru = datetime.utcnow() + timedelta(days=30)
> + invoice.valid_thru = datetime.utcnow() + timedelta(
> + days=30, hours=10, minutes=30)
> user.payment_due = invoice.valid_thru

Can you explain this new interval?

> @billing.route("/billing/new-payment", methods=["POST"])
> @loginrequired
> def new_payment_POST():
> -%<-
> + return {"error": "Invalid form submission"}
> +
> + current_user.payment_processor = processor
> +
> + # Ensure that the billing term starts before the first payment is
> created.
> + current_user.payment_due = datetime.utcnow() - timedelta(seconds=5)
> +
> current_user.payment_interval = PaymentInterval(term)
> - success, details = charge_user(current_user)
> - if not success:
> - return render_template("new-payment.html",
> - amount=current_user.payment_cents, error=details)
> db.session.commit()
> - freshen_user()
> - deliver_profile_update(current_user)
> +
> + # todo: add address to redirect client-side after success
> +
> + if processor == "stripe":
> + method = valid.require("method")
> + if not valid.ok:
> + return {"error": "Payment method not set"}
> + payment_initiated, retval = move_to_gql.stripe.init_payment(
> + current_user, method)
> + else:
> + return {"error": "Unknown payment processor"}

This bit would be good to generalize a bit. Defining the interface
alone, in one file, would make it easier to reason about the logic.

IMO this patchset should begin with the new interface for payment
processing defined in isolation for its own independent review, so that
we can evaluate and validate the state flow in a smaller patch.

Maybe you can also sketch up a flowchart which describes the state
changes? This would be helpful to validate the implementation against.

> +def get_amount(user):
> + amount = user.payment_cents
> + if user.payment_interval == PaymentInterval.yearly:
> + amount = amount * 10
> + return amount
> +
> +class PaymentStatus(Enum):
> + unchanged = "unchanged"
> + successful = "successful"
> + processing = "processing"
> + failed = "failed"
> +
> +ut = UserType
> +ps = PaymentStatus
> +
> +# {(status, transition): new_status}
> +# non existent or None transitions keep the current status
> +status_transitions = {
> + # todo: uncofirmed users can't setup payments, right? otherwise add
> + # transitions.
> +
> + ("admin", "*"): None,
> + ("unknown", "*"): None,
> + ("suspended", "*"): None,
> +
> + ("*", ps.successful): ut.active_paying,
> +
> + (ut.active_provisional, ps.failed): ut.active_delinquent,
> + (ut.active_paying, ps.failed): ut.active_delinquent,
> +
> + (ut.active_non_paying, ps.processing): ut.active_provisional,
> +
> + # todo: this transition makes it possible to remain indefinitely
> + # provisional by adding new, non-working payment methods. suspend
> + # users who are provisional and made no payment in the last n weeks?
> + # remove this transition? if choosing the latter then you can't
> + # easily get your sr.ht account working again after switching bank
> + # accounts or whatever.
> + (ut.active_delinquent, ps.processing): ut.active_provisional,
> +}

I would like to see this state machine simplified a bit, or made more
robust with the benefit of a stronger type system like Go's. Invalid
states should be unrepresentable if possible. Unit tests against a mock
payment provider would also be very good here. I normally don't write
tests unless the cost:benefit ratio is good, and for a complex billing
system it'd be very good.

Speaking of tests, a mock provider would be good for *_test.go, but it
would also be cool to have a separate binary which can be manually run
to validate our flow against the Stripe sandbox API.

> <script>
> -document.getElementById('payment-form').style.display = 'block';
> -var stripe = Stripe('{{cfg("meta.sr.ht::billing",
> -%<-

As this script grows in size, it would make sense to move it into a
separate file. Style note: use semi-colons.

> + }).then(result => { // todo: doesn't js has async/await too? browser support?

Let's not use aysnc/await

> + if(result.error) // todo: handle error properly

Note: robust error handling is an acceptance requirement of the new
billing system.
Details
Message ID
<CC12DWWDYBOM.1YRWQ4ZT7PHU0@pc>
In-Reply-To
<CBYEVD1UNK40.1UCOS0ZUQ44K3@taiga> (view parent)
DKIM signature
pass
Download raw message
On Tue Jun 8, 2021 at 7:26 PM CEST, Drew DeVault wrote:
> This looks like a good start/research prototype.

Happy to hear that :)

> On Sun May 30, 2021 at 3:15 PM EDT, Tim Hallmann wrote:
> > diff --git a/metasrht/billing.py b/metasrht/billing.py
> > index 23e7e2f..b1d3579 100644
> > --- a/metasrht/billing.py
> > +++ b/metasrht/billing.py
> > @@ -29,7 +29,6 @@ def charge_user(user):
> > amount = user.payment_cents
> > if user.payment_interval == PaymentInterval.yearly:
> > amount = amount * 10 # Apply yearly discount
> > - # TODO: Multiple currencies
> > charge = stripe.Charge.create(
> > amount=amount,
> > currency="usd",
> > @@ -56,7 +55,8 @@ def charge_user(user):
> > invoice.source = charge.source.stripe_id
> > db.session.add(invoice)
> > if user.payment_interval == PaymentInterval.monthly:
> > - invoice.valid_thru = datetime.utcnow() + timedelta(days=30)
> > + invoice.valid_thru = datetime.utcnow() + timedelta(
> > + days=30, hours=10, minutes=30)
> > user.payment_due = invoice.valid_thru
>
> Can you explain this new interval?

One year (31 557 600 seconds) divided by 12. More tongue-in-cheek than
a necessary change.

> > @billing.route("/billing/new-payment", methods=["POST"])
> > @loginrequired
> > def new_payment_POST():
> > -%<-
> > + return {"error": "Invalid form submission"}
> > +
> > + current_user.payment_processor = processor
> > +
> > + # Ensure that the billing term starts before the first payment is
> > created.
> > + current_user.payment_due = datetime.utcnow() - timedelta(seconds=5)
> > +
> > current_user.payment_interval = PaymentInterval(term)
> > - success, details = charge_user(current_user)
> > - if not success:
> > - return render_template("new-payment.html",
> > - amount=current_user.payment_cents, error=details)
> > db.session.commit()
> > - freshen_user()
> > - deliver_profile_update(current_user)
> > +
> > + # todo: add address to redirect client-side after success
> > +
> > + if processor == "stripe":
> > + method = valid.require("method")
> > + if not valid.ok:
> > + return {"error": "Payment method not set"}
> > + payment_initiated, retval = move_to_gql.stripe.init_payment(
> > + current_user, method)
> > + else:
> > + return {"error": "Unknown payment processor"}
>
> This bit would be good to generalize a bit. Defining the interface
> alone, in one file, would make it easier to reason about the logic.

The interface for the payment processors is basically what the public
functions in move_to_gql.stripe (init_payment and handle_term) offer,
though functions to add, change or remove a payment method (and the
processor altogether) would of course be required too. I should have
made that a bit clearer in the proposal.

I am not sure what you want to generalize here, though. (Unless you
mean the validation for the method and how that is passed to
init_payment, thats not exactly great right now and could ideally be
handled within init_payment.) Other processors would implement an
init_payment as well, get whatever the client sends them, and return a
PaymentStatus and whatever they want to return to the client.

> IMO this patchset should begin with the new interface for payment
> processing defined in isolation for its own independent review, so that
> we can evaluate and validate the state flow in a smaller patch.
>
> Maybe you can also sketch up a flowchart which describes the state
> changes? This would be helpful to validate the implementation against.

Yeah, I'll create a more detailed definition for the interface the next
few days.

> > +def get_amount(user):
> > + amount = user.payment_cents
> > + if user.payment_interval == PaymentInterval.yearly:
> > + amount = amount * 10
> > + return amount
> > +
> > +class PaymentStatus(Enum):
> > + unchanged = "unchanged"
> > + successful = "successful"
> > + processing = "processing"
> > + failed = "failed"
> > +
> > +ut = UserType
> > +ps = PaymentStatus
> > +
> > +# {(status, transition): new_status}
> > +# non existent or None transitions keep the current status
> > +status_transitions = {
> > + # todo: uncofirmed users can't setup payments, right? otherwise add
> > + # transitions.
> > +
> > + ("admin", "*"): None,
> > + ("unknown", "*"): None,
> > + ("suspended", "*"): None,
> > +
> > + ("*", ps.successful): ut.active_paying,
> > +
> > + (ut.active_provisional, ps.failed): ut.active_delinquent,
> > + (ut.active_paying, ps.failed): ut.active_delinquent,
> > +
> > + (ut.active_non_paying, ps.processing): ut.active_provisional,
> > +
> > + # todo: this transition makes it possible to remain indefinitely
> > + # provisional by adding new, non-working payment methods. suspend
> > + # users who are provisional and made no payment in the last n weeks?
> > + # remove this transition? if choosing the latter then you can't
> > + # easily get your sr.ht account working again after switching bank
> > + # accounts or whatever.
> > + (ut.active_delinquent, ps.processing): ut.active_provisional,
> > +}
>
> I would like to see this state machine simplified a bit, or made more
> robust with the benefit of a stronger type system like Go's. Invalid
> states should be unrepresentable if possible. Unit tests against a mock
> payment provider would also be very good here. I normally don't write
> tests unless the cost:benefit ratio is good, and for a complex billing
> system it'd be very good.

Oh, for sure. Just didn't want to spend more effort on this prototype.
I mean, thats not how I would implement a proper state machine in
Python either.

> Speaking of tests, a mock provider would be good for *_test.go, but it
> would also be cool to have a separate binary which can be manually run
> to validate our flow against the Stripe sandbox API.

Depending on how extensive this test should be, this might not make
sense as a simple binary. If we want to test a single payment initation
without authentication (e.g. 3d secure) and renewal per payment
processor, sure, just a few GraphQL calls, but if we want tests with
authentication, declined payments and so on, then we need to run those
in the browser (or script those in a browser).

> > <script>
> > -document.getElementById('payment-form').style.display = 'block';
> > -var stripe = Stripe('{{cfg("meta.sr.ht::billing",
> > -%<-
>
> As this script grows in size, it would make sense to move it into a
> separate file. Style note: use semi-colons.

As a template or as <script src="...">? The latter needs proper cache
validation and so on, is that currently provided by flask?

> > + }).then(result => { // todo: doesn't js has async/await too? browser support?
>
> Let's not use aysnc/await

Alright.

> > + if(result.error) // todo: handle error properly
>
> Note: robust error handling is an acceptance requirement of the new
> billing system.

Yeah, I had no intention of showing alerts either.
Details
Message ID
<CC4BND2PKNAD.J6943Z5NT8RH@taiga>
In-Reply-To
<CC12DWWDYBOM.1YRWQ4ZT7PHU0@pc> (view parent)
DKIM signature
fail
Download raw message
DKIM signature: fail
On Fri Jun 11, 2021 at 4:17 PM EDT, Tim Hallmann wrote:
> I am not sure what you want to generalize here, though. (Unless you
> mean the validation for the method and how that is passed to
> init_payment, thats not exactly great right now and could ideally be
> handled within init_payment.) Other processors would implement an
> init_payment as well, get whatever the client sends them, and return a
> PaymentStatus and whatever they want to return to the client.

In retrospect my generalization comment probably won't apply when this
is rewritten for GQL.

> > As this script grows in size, it would make sense to move it into a
> > separate file. Style note: use semi-colons.
>
> As a template or as <script src="...">? The latter needs proper cache
> validation and so on, is that currently provided by flask?

We can incorporate it into our static asset pipeline: see the Makefiles
in *.sr.ht and core.sr.ht. These involve a step which hashes the file.
See static_resource in srht/flask.py as well.
Reply to thread Export thread (mbox)