Received: from wout2-smtp.messagingengine.com (wout2-smtp.messagingengine.com [64.147.123.25]) by mail-b.sr.ht (Postfix) with ESMTPS id 50AC3FF122 for <~euandreh/public-inbox@lists.sr.ht>; Tue, 1 Sep 2020 11:25:21 +0000 (UTC) Authentication-Results: mail-b.sr.ht; dkim=pass (1024-bit key) header.d=josephg.com header.i=@josephg.com header.b=O0rYtvTA; dkim=fail reason="key not found in DNS" (0-bit key) header.d=messagingengine.com header.i=@messagingengine.com header.b=f1mPF/1x Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id E4956E77; Tue, 1 Sep 2020 07:25:19 -0400 (EDT) Received: from imap35 ([10.202.2.85]) by compute1.internal (MEProxy); Tue, 01 Sep 2020 07:25:20 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=josephg.com; h= mime-version:message-id:in-reply-to:references:date:from:to :subject:content-type:content-transfer-encoding; s=mesmtp; bh=BO dAgwWsH67HEzbojnfogot17D0WKCAzyItvgTLmJCw=; b=O0rYtvTAoINKZTJObn vUnmONdChl7CPOEklbKnqUQ+Sk8qaAoCe0YyqwTLq0YZywk49u56Zwff5QaSJfLm SYA9bd2RUEagih/gPTR4BHJarqfWSHQeUsHsvMmVtj4w8jrJt261gcmh1Git8Jkt lJSUIZ9tpOecI/nFsnhxCleA0= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; bh=BOdAgwWsH67HEzbojnfogot17D0WKCAzyItvgTLmJ Cw=; b=f1mPF/1xX+9SrbduF4hyrnDZlojUnotl/ZuV6F+yuzbQlOHFTtLgMSJ5u ghKQ7iTGJdFGqxzM87y1JJLCWb6BZjbRw2TjhDK+z24JeMwOVOpZgDsZKZ4vWlHI x6C8Ca/TnlF8Glsyvepoczx5SP6kbYCOe/JrBGp3gTIfTGohNHuxCsxxHvW3gn35 hQLIa42DMt3y1p0q2rl4ltdrCYpCzQVvfbhKssX3AmCkwndYDhbP6l4jLnWH6jeM oBw77aR3keR94fhUnSCsnlX5ZSljRkIecR7Z9sGsR8KeB5j72A/ND3rncJyOOQ/e +DXRIAj+KVgqvbY2AimyrweAA9mQg== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduiedrudefjedggedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucenucfjughrpefofgggkfgjfhffhffvufgtgfesth hqredtreerjeenucfhrhhomhepfdfuvghphhcuifgvnhhtlhgvfdcuoehmvgesjhhoshgv phhhghdrtghomheqnecuggftrfgrthhtvghrnhepkeegtdfhgfdvheetveetleejgfdvhf egtdegfedvhfegveffgfdthfehgedukeegnecuffhomhgrihhnpehslhgrtghkrdgtohhm necuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepmhgvse hjohhsvghphhhgrdgtohhm X-ME-Proxy: Received: by mailuser.nyi.internal (Postfix, from userid 501) id 72BAB14C034C; Tue, 1 Sep 2020 07:25:17 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.3.0-248-gcd102cb-fm-20200901.001-gcd102cb9 Mime-Version: 1.0 Message-Id: <9e513706-7ebc-4a40-8e5e-a5f94d84bc77@www.fastmail.com> In-Reply-To: <87ft81zvi0.fsf@euandre.org> References: <87ft81zvi0.fsf@euandre.org> Date: Tue, 01 Sep 2020 21:24:55 +1000 From: "Seph Gentle" To: EuAndreh , ~euandreh/public-inbox@lists.sr.ht Subject: Re: Making a database Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: quoted-printable On Tue, Sep 1, 2020, at 7:39 PM, EuAndreh wrote: > "Seph Gentle" writes: >=20 > > I've been thinking along similar lines, and am talking to some folks= > > about building something on top of CRDTs. Modern CRDTs like automerg= e > > and y.js are just getting to the point where they're fast and small > > enough to work fine. I can see something really valuable that looks > > kind of like git, except where you can store arbitrary data. And > > live-sync peers - so if I make changes you see those changes live on= > > your computer too - without (necessarily) needing a centralized > > server. > > > > Love to chat if you & others are keen - I/we are still in the proces= s > > of figuring out what this thing will look like. But, I feel like the= > > database piece is really core here too, and I like a lot of your > > thinking in this space. >=20 > Thanks for the encouragement :) >=20 > I've looked at CRDTs, but I haven't done a deep dive yet, because my > first impression was that they generally choose to limit the types of > data structures and operations that you can perform on them in favour = of > being conflict-free. >=20 > That's why I didn't mention it on my article directly, but I admit to > having it as pre-requisite to making an informed decision on the > subject. >=20 > WDYT? Well, they=E2=80=99re conflict-free in the sense that (unlike Postgres/f= oundationdb/etc) the database doesn=E2=80=99t reject conflicting concurr= ent updates. Which is a property you need if you=E2=80=99re going to com= mit local writes while you=E2=80=99re offline. And if I=E2=80=99m readin= g your blog post right it sounds like that=E2=80=99s what you=E2=80=99re= going for. You kinda gave a good description of them in your post already. They wor= k by requiring all peers to resolve concurrent edits to an object the sa= me way. That could be last-writer-wins (based on time stamps like you sa= id in your article). Or it could store all conflicting versions together= to be resolved by the next reader (Eg riak). Or it could do something m= ore clever (like merge changes using automerge or y.js or the like). Git= is sort of a CRDT - it=E2=80=99s just it=E2=80=99s 3 way merge algorith= m is a bit of a dogs breakfast as far as these things go. Anyway, happy to chat about this stuff in more detail if you wanna zoom = or something. Or hop into the automerge slack (with pvh from ink&switch,= and Martin Kleppmann, and others) and we can talk about this stuff - ht= tps://join.slack.com/t/automerge/shared_invite/zt-e4p3760n-kKh7r3KRH1Yww= NfiZM8ktw As I said - I=E2=80=99m looking to build something like this too and it=E2= =80=99d be good to compare notes on our approaches!