Last active 2 days ago


Last active 6 days ago


Last active a month ago


Last active a month ago


Last active 2 months ago


Last active 3 months ago


Last active 3 months ago


Last active 3 months ago


Last active 3 months ago


Last active 3 months ago
View more

Recent activity

Re: [PATCH gddo] gddo-server: Fetch modules in the background a day ago

From Adnan Maolood to ~sircmpwn/godocs.io

On Sun Jun 20, 2021 at 7:50 PM EDT, Drew DeVault wrote:
> Can we estimate this based on the number of packages in the production
> db today? Come up with some kind of model for predicting its behavior?

As of right now, there are 15022 unique packages that are imported by
packages in the database. Of these 15022 packages, 1693 (about 11.3%)
are not already in the database. Now, we can't tell how many packages
these 1693 import without adding them to the database or fetching them
from the module proxy.

To retrieve the number of unique imported packages in the database, this
command was used:

[PATCH gddo] gddo-server: Fix error fetching stdlib a day ago

From Adnan Maolood to ~sircmpwn/godocs.io

Using the version "latest" caused problems when fetching the standard
library. Use the resolved latest version instead.
 gddo-server/fetch.go | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/gddo-server/fetch.go b/gddo-server/fetch.go
index ba795b7..39a0218 100644
--- a/gddo-server/fetch.go
+++ b/gddo-server/fetch.go
@@ -93,6 +93,9 @@ func (s *Server) fetch(ctx context.Context, modulePath, version string) error {
		if err := s.db.PutModule(ctx, mod); err != nil {
			return err
[message trimmed]

[PATCH gddo] gddo-server: Fetch modules in the background a day ago

From Adnan Maolood to ~sircmpwn/godocs.io

In a separate goroutine, continuously fetch the oldest module in the
database from the module proxy and update it if necessary. Note that the
--crawl-interval flag must be specified to enable background fetching.

A few remaining questions:

- Should we crawl the imports of packages (to discover new packages not
  in the database)? If we do, perhaps this should be opt-in as this
  could cause the database to grow exponentially, which wouldn't be
  desirable for small installations.
- Is PostgreSQL's ORDER BY fast enough for large databases? Or is speed
  not really a concern?
- Should we log FETCH messages, or should we keep silent except when
[message trimmed]

[PATCH gddo] gddo-server: Clean up module fetching code 2 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

 gddo-server/{crawl.go => fetch.go} | 156 ++++++++++++++++-------------
 gddo-server/gemini.go              |   5 +-
 gddo-server/http.go                |   5 +-
 gddo-server/server.go              |  16 ++-
 4 files changed, 98 insertions(+), 84 deletions(-)
 rename gddo-server/{crawl.go => fetch.go} (50%)

diff --git a/gddo-server/crawl.go b/gddo-server/fetch.go
similarity index 50%
rename from gddo-server/crawl.go
rename to gddo-server/fetch.go
index 766ce65..a425d24 100644
--- a/gddo-server/crawl.go
[message trimmed]

[PATCH gddo] gddo-server: Consolidate request path parsing 2 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

Consolidate the parsing of request paths into a single function.
Also take the opportunity to reject invalid versions like "master" or
"main" which cause issues.

 gddo-server/crawl.go  |  2 +-
 gddo-server/gemini.go | 18 +++++-------------
 gddo-server/http.go   | 20 ++++++--------------
 gddo-server/server.go | 24 ++++++++++++++++++++++++
 4 files changed, 36 insertions(+), 28 deletions(-)

diff --git a/gddo-server/crawl.go b/gddo-server/crawl.go
index 4e527e8..766ce65 100644
--- a/gddo-server/crawl.go
[message trimmed]

[PATCH gddo] internal/version: Remove unused functions 2 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

These functions are not used by gddo-server, so they can be removed.

 internal/version/version.go      | 207 +------------------------------
 internal/version/version_test.go | 198 +----------------------------
 2 files changed, 6 insertions(+), 399 deletions(-)

diff --git a/internal/version/version.go b/internal/version/version.go
index 984c76f..a9ab050 100644
--- a/internal/version/version.go
+++ b/internal/version/version.go
@@ -14,25 +14,20 @@ import (

[message trimmed]

Re: [PATCH] gddo-server: Invalidate Etag when subpackages & importers updated 2 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

On Sat Jun 19, 2021 at 8:44 AM EDT, Teddy Wing wrote:
> I'm unclear on what the `importerCount` limit at 8 is for, but figured
> I'd reuse the code from just before
> be62deeb42eeb7a2a467b9751e34108d8e498047.

I'm not sure what that is for either. Perhaps it can be removed?

[PATCH gddo] gddo-server: Disallow specifying pseudo-versions 3 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

Don't allow specifying pseudo-versions as they can be abused to fill up
the database (and they don't show up in the list of versions anyways).

Later on, we should also automatically remove pseudo-versions from the
database but keep the latest pseudo-version for packages with no release
versions available.

 gddo-server/crawl.go  |  1 +
 gddo-server/gemini.go |  7 +++++++
 gddo-server/http.go   | 13 ++++++++++++-
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/gddo-server/crawl.go b/gddo-server/crawl.go
[message trimmed]

[PATCH gddo] gddo-server: Implement version picker 3 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

This patch implements support for specifying which version of a package
to display documentation for. The documentation for that version will be
fetched on demand if it is not already available.

 gddo-server/assets/templates/doc.gmi       |  2 +-
 gddo-server/assets/templates/doc.html      |  2 +-
 gddo-server/assets/templates/versions.html | 17 +++++++++++++++++
 gddo-server/crawl.go                       | 12 ++++++++----
 gddo-server/gemini.go                      | 17 ++++++++++++-----
 gddo-server/http.go                        | 20 +++++++++++++++-----
 gddo-server/server.go                      |  8 ++++----
 gddo-server/template.go                    |  3 ++-
 8 files changed, 60 insertions(+), 21 deletions(-)
[message trimmed]

[PATCH gddo] readme: Document Graphviz dependency 3 days ago

From Adnan Maolood to ~sircmpwn/godocs.io

 README.md | 1 +
 1 file changed, 1 insertion(+)

diff --git a/README.md b/README.md
index 731f032..dbc7b9b 100644
--- a/README.md
+++ b/README.md
@@ -18,6 +18,7 @@ First install the dependencies:

- Go 1.13 or above
- PostgreSQL 13
- Graphviz (required for import graphs)

[message trimmed]