# Refactoring is the root of all evil

This last month I've spent by cleaning up after I had decided on a new
method of interacting with the FedBOX repositories.

This new method allows us to aggregate items from multiple collections with
multiple search queries. This, of course, has the downside of slowing down
everything pretty heavily, so there's a strong incentive of not overdoing it.

The new method has an improved collection deep loading, which allows us to go
through all pages until we reach our desired number of items or reach the end.

One thing that this change underscored was again the slowness of our JSON
decoding, so I updated the Go-AP ActivityPub package with JSON decoding based
on valyala/fastjson. So far the improvements are not blowing my mind but I
haven't profiled everything since the change so there might be some adjustments
to do still.

To further mitigate this, we further added the placeholders for a local in
memory cache for BrutaLinks to store loaded collections. This has the downside
of poor cache invalidation logic, but I'll work on it more in the future.
Currently we just clear the cache every write we do to FedBOX.

The main reason for this large change in logic was that we need to
transparently load from services which are not the local FedBOX instance in the
case of federated activities.

The move to this new system is not fully completed, the bugs I had to fix were
numerous and tedious to fix so not everything got ported to it.

Some improvements were done to the processing package to allow better cache
invalidation for collections affected by activities' side effects.

## Misc

I have updated the docker images to be built on top of go 1.17 because there
should be some theoretical size and run speed improvements. I haven't checked,
so ymmv.

Ive updated to go-chi v5, on both BrutaLinks and FedBOX.

/Marius
Berlin, 23th August 2021