I was able to work around this (in a test VM, after upgrading it from 20.04 to 24.04) with adapted instructions from a blogpost [1], by temporarily installing postgresql-14 from the PGDG repository for doing the 14 -> 16 upgrade:

# apt install postgresql-common
# /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
# apt install postgresql-14
# pg_dropcluster 16 main –stop
# pg_upgradecluster 14 main
# pg_dropcluster 14 main
# apt purge postgresql-14 postgresql-client-14
# rm /etc/apt/sources.list.d/pgdg.sources
# apt update

I have yet to try this on my actual server with actual data. Please be careful if you attempt this with a production database.

*[1] https://www.directedignorance.com/blog/upgrading-postgresql-14-to-16-on-ubuntu