Hey,

git.sr.ht doesn't work for me anymore after upgrading from 0.57.4-r0 to
0.60.1-r0 today. During the migration I get the error below and 
git.sr.ht also only throws 500 errors. The e-mails I receive about that 
are basically identical to the error below.

Would appreciate any help!


Kind regards,

Yaşar


Executing git.sr.ht-0.60.1-r0.post-upgrade
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade dacab1dcba45 -> 01412986a44d, Create git-daemon-export-ok files again
Creating git-daemon-export-ok files
Traceback (most recent call last):
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1283, in _execute_context
     self.dialect.do_execute(
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
     cursor.execute(statement, parameters)
psycopg2.errors.UndefinedColumn: column repository.readme does not exist
LINE 1: ..., repository.visibility AS repository_visibility, repository...
                                                              ^
HINT:  Perhaps you meant to reference the column "repository.name".


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
   File "/usr/bin/gitsrht-migrate", line 5, in <module>
     alembic("git.sr.ht", gitsrht.alembic)
   File "/usr/lib/python3.8/site-packages/srht/database.py", line 121, in alembic
     cmdline.run_cmd(config, options)
   File "/usr/lib/python3.8/site-packages/alembic/config.py", line 548, in run_cmd
     fn(
   File "/usr/lib/python3.8/site-packages/alembic/command.py", line 298, in upgrade
     script.run_env()
   File "/usr/lib/python3.8/site-packages/alembic/script/base.py", line 489, in run_env
     util.load_python_file(self.dir, "env.py")
   File "/usr/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 98, in load_python_file
     module = load_module_py(module_id, path)
   File "/usr/lib/python3.8/site-packages/alembic/util/compat.py", line 184, in load_module_py
     spec.loader.exec_module(module)
   File "<frozen importlib._bootstrap_external>", line 783, in exec_module
   File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
   File "/usr/lib/python3.8/site-packages/gitsrht/alembic/env.py", line 3, in <module>
     alembic_env()
   File "/usr/lib/python3.8/site-packages/srht/database.py", line 157, in alembic_env
     run_migrations_online()
   File "/usr/lib/python3.8/site-packages/srht/database.py", line 150, in run_migrations_online
     context.run_migrations()
   File "<string>", line 8, in run_migrations
   File "/usr/lib/python3.8/site-packages/alembic/runtime/environment.py", line 846, in run_migrations
     self.get_context().run_migrations(**kw)
   File "/usr/lib/python3.8/site-packages/alembic/runtime/migration.py", line 520, in run_migrations
     step.migration_fn(**kw)
   File "/usr/lib/python3.8/site-packages/gitsrht/alembic/versions/a8ad35a0bee7_create_git_daemon_export_ok_files.py", line 31, in upgrade
     for repo in tqdm(session.query(Repository).all()):
   File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3319, in all
     return list(self)
   File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3481, in __iter__
     return self._execute_and_instances(context)
   File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3506, in _execute_and_instances
     result = conn.execute(querycontext.statement, self._params)
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1020, in execute
     return meth(self, multiparams, params)
   File "/usr/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
     return connection._execute_clauseelement(self, multiparams, params)
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1133, in _execute_clauseelement
     ret = self._execute_context(
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1323, in _execute_context
     self._handle_dbapi_exception(
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1517, in _handle_dbapi_exception
     util.raise_(
   File "/usr/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
     raise exception
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1283, in _execute_context
     self.dialect.do_execute(
   File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 590, in do_execute
     cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedColumn) column repository.readme does not exist
LINE 1: ..., repository.visibility AS repository_visibility, repository...
                                                              ^
HINT:  Perhaps you meant to reference the column "repository.name".

[SQL: SELECT repository.id AS repository_id, repository.created AS repository_created, repository.updated AS repository_updated, repository.name AS repository_name, repository.description AS repository_description, repository.path AS repository_path, repository.visibility AS repository_visibility, repository.readme AS repository_readme, repository.upstream_uri AS repository_upstream_uri, repository.owner_id AS repository_owner_id, repository.source_repo_id AS repository_source_repo_id
FROM repository]
(Background on this error at: http://sqlalche.me/e/f405)
ERROR: git.sr.ht-0.60.1-r0.post-upgrade: script exited with error 1

I think you skipped a migration.

On Thu, Aug 27, 2020 at 04:38:30PM -0400, Drew DeVault wrote:
>I think you skipped a migration.

Hey, thanks for your quick reply!
I just renamed the table, created a new "git.sr.ht" table and
after running gitsrht-initdb that seems to run fine, so I guess
it's an error on my side. Sorry!

To me it looked like it was stuck on 01412986a44d, where it was trying
to use a table a later migration (8fbeb080c434) would create:

$ gitsrht-migrate history
d42e577c5dcd -> 8fbeb080c434 (head), Add repository.readme
01412986a44d -> d42e577c5dcd, Add unique constraint to artifacts
dacab1dcba45 -> 01412986a44d, Create git-daemon-export-ok files again
9f72f0dea908 -> dacab1dcba45, Clean up samples, allow push options
[...]

$ gitsrht-migrate current
INFO  [alembic.runtime.migration] Context impl PostgresqlImpl.
INFO  [alembic.runtime.migration] Will assume transactional DDL.
dacab1dcba45

Hello,


I am facing the same issue upgrading from 0.55.3 to 0.60.4. Error is the same as above just at a different migration step.


INFO [alembic.runtime.migration] Context impl PostgresqlImpl.

INFO [alembic.runtime.migration] Will assume transactional DDL.

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.

INFO [alembic.runtime.migration] Will assume transactional DDL.

INFO [alembic.runtime.migration] Running upgrade 9f72f0dea908 -> dacab1dcba45, Clean up samples, allow push options

Allowing push options, fixing repositories with missing hooks, pruning samples

Traceback (most recent call last):

File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1276, in _execute_context

self.dialect.do_execute(

File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute

cursor.execute(statement, parameters)

psycopg2.errors.UndefinedColumn: column repository.readme does not exist



i feel the same as:

> To me it looked like it was stuck on 01412986a44d, where it was trying
> to use a table a later migration (8fbeb080c434) would create:

My migrate history is a little different because i'm a couple more versions behind


gitsrht-migrate history

d42e577c5dcd -> 8fbeb080c434 (head), Add repository.readme

01412986a44d -> d42e577c5dcd, Add unique constraint to artifacts

dacab1dcba45 -> 01412986a44d, Create git-daemon-export-ok files again

9f72f0dea908 -> dacab1dcba45, Clean up samples, allow push options

3c1285bb23e2 -> 9f72f0dea908, Clean up broken HEADs


gitsrht-migrate current

INFO [alembic.runtime.migration] Context impl PostgresqlImpl.

INFO [alembic.runtime.migration] Will assume transactional DDL.

9f72f0dea908


It throws the same error if i try to run it manually.


Pardon my ignorance but i'm not sure where to go from here.

> My migrate history is a little different because i'm a couple more versions behind

Yes, this happened to me as well. The problem is that queries in
migration returns all columns of the "repository" table and the "readme"
column does not exist yet in dacab1dcba45 revision. In fact, only the
"path" column is needed in that migration, so I'd suggest the attached
patch, if you want to try it out.

Thank you Denis.


>so I'd suggest the attached patch, if you want to try it out.

While that did permit me to migrate to dacab1dcba45 the following migration(01412986a44d) fails again with the same error.

I suspect again because it grabs all columns

for repo in tqdm(session.query(Repository).all())

Is the only column needed for this migration repository.visibility? Is/can it be that simple?

> Is the only column needed for this migration repository.visibility? Is/can it be that simple?
> 

No, this will probably not work. One possible workaround is to skip this
migration file and run it after 8fbeb080c434_add_repository_readme.py;
not sure how to do this, apart from temporarily moving the file away.

> apart from temporarily moving the file away.

It did not like that as each one references the one before so it errors out if one is missing. Probably not idea to skip one anyways. It's gotta get applied. If only I could change the order somehow...maybe that's what I'll try to do.

> maybe that's what I'll try to do.


Sorry for replying to myself but i got it all to apply.


First thing i did was make 8fbeb080c434_add_repository_readme.py revise dacab1dcba45 so i was able to apply 8fbeb080c43 successfully. Then i applied the rest by running the migrate upgrade to the "latest" d42e577c5dcd. Then i put the revise of 8fbeb080c434 back to what it was.


So all the migrations have been applied which is great. However now i am unable to access any repos in the web interface. All returning permission denied. Something else is up. I'll figure that out. Hopefully it is not caused by something done here today.