Installation is broken: default Homebrew postgresql now 11.1

Question

Installation is broken: default Homebrew postgresql now 11.1

ralexx opened this issue 6 years ago · comments

Similar to this issue from your Slack channel, the postgresql dependency in this formula breaks installation of timescaledb. Most unfortunately, I can't launch my database either.

I'm not familiar with Homebrew recipe syntax, but is it as simple as this change?

 depends_on "postgresql" => :build  # old
 depends_on "postgresql@10" => :build  # new

Problem

> pg_ctl -D mydatabase start
waiting for server to start....2018-12-11 10:23:35.287 PST [98647] FATAL:  could not access file "timescaledb": No such file or directory
2018-12-11 10:23:35.287 PST [98647] LOG:  database system is shut down
 stopped waiting
pg_ctl: could not start server
Examine the log output.

After commenting out #shared_preload_libraries = 'timescaledb' in postgresql.conf:

mydatabase=# ALTER EXTENSION timescaledb UPDATE;
ERROR:  could not open extension control file "/usr/local/Cellar/postgresql@10/10.6/share/postgresql@10/extension/timescaledb.control": No such file or directory

Steps to reproduce

brew install expected behavior is to fail if a dependency is pinned.

> brew upgrade timescaledb
==> Upgrading 1 outdated package:
timescale/tap/timescaledb 1.0.0 -> 1.0.1
Error: You must `brew unpin postgresql` as installing timescale/tap/timescaledb requires the latest version of pinned dependencies

The postgresql formula now points to 11.1.

> brew info postgresql
postgresql: stable 11.1 (bottled), HEAD

To avoid clobbering my Postgresql 10.6 installation:

brew unlink postgresql
brew install postgresql@10  # keg-only
brew link --force postgresql@10

Then, upgrade timescaledb, which did not ignore dependencies as instructed but instead installed the default postgresql. You can see that the previous installation of postgresql@10 prevented the linking of v11.1.

> brew upgrade --ignore-dependencies timescaledb
Updating Homebrew...
Fast-forwarded master to origin/master.
Fast-forwarded master to origin/master.
==> Auto-updated Homebrew!
Updated 2 taps (homebrew/cask-versions and homebrew/core).
==> Updated Formulae
bitrise

==> Upgrading 1 outdated package:
timescale/tap/timescaledb 1.0.0 -> 1.0.1
==> Upgrading timescale/tap/timescaledb
==> Installing dependencies for timescale/tap/timescaledb: postgresql
==> Installing timescale/tap/timescaledb dependency: postgresql
==> Downloading https://homebrew.bintray.com/bottles/postgresql-11.1.sierra.bott
Already downloaded: /Users/me/Library/Caches/Homebrew/downloads/3be1486d9ccbc23b4d36bf286283b2c693eb46f6388cd24d0f071f321fec8085--postgresql-11.1.sierra.bottle.tar.gz
==> Pouring postgresql-11.1.sierra.bottle.tar.gz
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/clusterdb
Target /usr/local/bin/clusterdb
is a symlink belonging to postgresql@10. You can unlink it:
  brew unlink postgresql@10
# [all the files that didn't link]
==> Summary
🍺  /usr/local/Cellar/postgresql/11.1: 3,548 files, 40.3MB
==> Installing timescale/tap/timescaledb
==> Downloading https://timescalereleases.blob.core.windows.net/homebrew/timesca
######################################################################## 100.0%
==> ./bootstrap -DPROJECT_INSTALL_METHOD="brew"
==> cd ./build && make
Last 15 lines from /Users/me/Library/Logs/Homebrew/timescaledb/02.build:
1 error generated.
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_insert_state.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_constraint.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk.c.o] Error 1
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_index.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_adaptive.c.o] Error 1
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_index.c.o] Error 1
make[1]: *** [src/CMakeFiles/timescaledb.dir/all] Error 2
make: *** [all] Error 2

If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):
https://github.com/timescale/homebrew-tap/issues

### Full error log

2018-12-11 10:29:31 -0800

cd ./build && make

[  1%] Generating gitcommit.h
Scanning dependencies of tmeet sqlfile
Scanning dependencies of tmeet sqlupdatescripts
[  2%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--1.0.1.sql
[  5%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--1.0.0-rc1--1.0.1.sql
[  5%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.12.1--1.0.1.sql
[  5%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--1.0.0-rc2--1.0.1.sql
[  8%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--1.0.0--1.0.1.sql
[  9%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.10.1--1.0.1.sql
[ 10%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--1.0.0-rc3--1.0.1.sql
[ 10%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.10.0--1.0.1.sql
[ 10%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.12.0--1.0.1.sql
[ 12%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.11.0--1.0.1.sql
Scanning dependencies of tmeet timescaledb-loader
[ 13%] Built tmeet sqlfile
[ 13%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.9.1--1.0.1.sql
[ 14%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.8.0--1.0.1.sql
[ 15%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.7.1--1.0.1.sql
[ 16%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.7.0--1.0.1.sql
[ 17%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.9.2--1.0.1.sql
[ 19%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.9.0--1.0.1.sql
[ 19%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.6.1--1.0.1.sql
[ 20%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.6.0--1.0.1.sql
[ 21%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.5.0--1.0.1.sql
[ 23%] Building C object src/loader/CMakeFiles/timescaledb-loader.dir/loader.c.o
[ 24%] Building C object src/loader/CMakeFiles/timescaledb-loader.dir/bgw_message_queue.c.o
[ 26%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.4.1--1.0.1.sql
[ 26%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.3.0--1.0.1.sql
[ 27%] Building C object src/loader/CMakeFiles/timescaledb-loader.dir/bgw_counter.c.o
[ 28%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.4.2--1.0.1.sql
[ 30%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.4.0--1.0.1.sql
[ 30%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.2.0--1.0.1.sql
[ 32%] Building C object src/loader/CMakeFiles/timescaledb-loader.dir/bgw_launcher.c.o
[ 32%] Generating /tmp/timescaledb-20181211-830-8doeqn/timescaledb/build/sql/timescaledb--0.1.0--1.0.1.sql
[ 34%] Building C object src/loader/CMakeFiles/timescaledb-loader.dir/bgw_interface.c.o
[ 34%] Built tmeet sqlupdatescripts
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/loader/loader.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/loader/bgw_message_queue.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
1 error generated.
make[2]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/bgw_message_queue.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/loader.c.o] Error 1
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/loader/bgw_launcher.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/loader/bgw_interface.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/loader/bgw_counter.c:8:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
1 error generated.
1 error generated.
make[2]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/bgw_interface.c.o] Error 1
make[2]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/bgw_counter.c.o] Error 1
make[2]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/bgw_launcher.c.o] Error 1
make[1]: *** [src/loader/CMakeFiles/timescaledb-loader.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Scanning dependencies of tmeet timescaledb
[ 38%] Building C object src/CMakeFiles/timescaledb.dir/agg_bookend.c.o
[ 38%] Building C object src/CMakeFiles/timescaledb.dir/cache_invalidate.c.o
[ 38%] Building C object src/CMakeFiles/timescaledb.dir/cache.c.o
[ 38%] Building C object src/CMakeFiles/timescaledb.dir/catalog.c.o
[ 41%] Building C object src/CMakeFiles/timescaledb.dir/chunk_dispatch.c.o
[ 42%] Building C object src/CMakeFiles/timescaledb.dir/chunk_adaptive.c.o
[ 42%] Building C object src/CMakeFiles/timescaledb.dir/chunk.c.o
[ 42%] Building C object src/CMakeFiles/timescaledb.dir/chunk_dispatch_state.c.o
[ 45%] Building C object src/CMakeFiles/timescaledb.dir/chunk_constraint.c.o
[ 45%] Building C object src/CMakeFiles/timescaledb.dir/chunk_dispatch_plan.c.o
[ 46%] Building C object src/CMakeFiles/timescaledb.dir/chunk_insert_state.c.o
[ 47%] Building C object src/CMakeFiles/timescaledb.dir/chunk_index.c.o
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_dispatch_state.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/cache_invalidate.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/cache.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/catalog.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_dispatch.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_adaptive.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/agg_bookend.c:7:10: fatal error: 'postgres.h' file not found
1 error generated.
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_constraint.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
make[2]: *** [src/CMakeFiles/timescaledb.dir/cache.c.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: *** [src/CMakeFiles/timescaledb.dir/cache_invalidate.c.o] Error 1
1 error generated.
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_insert_state.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/agg_bookend.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_dispatch_state.c.o] Error 1
1 error generated.
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_dispatch_plan.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
make[2]: *** [src/CMakeFiles/timescaledb.dir/catalog.c.o] Error 1
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_dispatch.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_dispatch_plan.c.o] Error 1
1 error generated.
1 error generated.
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_insert_state.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_constraint.c.o] Error 1
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk.c.o] Error 1
/tmp/timescaledb-20181211-830-8doeqn/timescaledb/src/chunk_index.c:7:10: fatal error: 'postgres.h' file not found
#include <postgres.h>
         ^~~~~~~~~~~~
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_adaptive.c.o] Error 1
1 error generated.
make[2]: *** [src/CMakeFiles/timescaledb.dir/chunk_index.c.o] Error 1
make[1]: *** [src/CMakeFiles/timescaledb.dir/all] Error 2
make: *** [all] Error 2

HOMEBREW_VERSION: 1.8.5
ORIGIN: https://github.com/Homebrew/brew
HEAD: 38493a35a2abe722c527ffebf9c2fbf9bc4d3825
Last commit: 4 days ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: 5b08098655389fa74cf3a332eab63ab16074da62
Core tap last commit: 3 minutes ago
HOMEBREW_PREFIX: /usr/local
HOMEBREW_CACHE: /Users/me/Library/Caches/Homebrew
CPU: dodeca-core 64-bit arrandale
Homebrew Ruby: 2.3.7 => /System/Library/Frameworks/Ruby.framework/Versions/2.3/usr/bin/ruby
Clang: 9.0 build 900
Git: 2.20.0 => /usr/local/bin/git
Curl: 7.54.0 => /usr/bin/curl
Java: 11.0.1, 11
macOS: 10.12.6-x86_64
CLT: 9.2.0.0.1.1510905681
Xcode: 9.2
XQuartz: 2.7.11 => /opt/X11

HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
MAKEFLAGS: -j12
CMAKE_PREFIX_PATH: /usr/local/opt/openssl:/usr/local
CMAKE_INCLUDE_PATH: /usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
PKG_CONFIG_PATH: /usr/local/opt/postgresql/lib/pkgconfig:/usr/local/opt/openssl/lib/pkgconfig
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.12
HOMEBREW_GIT: git
ACLOCAL_PATH: /usr/local/share/aclocal
PATH: /usr/local/Homebrew/Library/Homebrew/shims/mac/super:/usr/local/opt/cmake/bin:/usr/local/opt/postgresql/bin:/usr/local/opt/openssl/bin:/usr/bin:/bin:/usr/sbin:/sbin

RobAtticus · Answer 1 · Wed Dec 12 2018 03:44:53 GMT+0800 (China Standard Time)

Thanks, yes the way to temporarily fix the formula is to modify it as you suggested with the @10. You can do that locally if you'd like to get it to build while waiting for PG11 support. However, (beta) PG11 support will likely land later this week, at which point we'll do a new formula release.

In the future we'll try to handle these migrations a bit more gracefully.

Rob Alexander · Answer 2 · Wed Dec 12 2018 03:57:44 GMT+0800 (China Standard Time)

Thanks. I recovered by copying the timescaledb extension files manually.

cd /usr/local/Cellar/postgresql@10/10.6/share/postgresql@10/extension
cp /usr/local/Cellar/timescaledb/1.0.0/share/timescaledb/timescaledb--1.0.0.sql .
cp /usr/local/Cellar/timescaledb/1.0.0/share/timescaledb/timescaledb.control .

Todd · Answer 3 · Tue Apr 28 2020 19:58:15 GMT+0800 (China Standard Time)

I see the install script: /usr/local/bin/timescaledb_move.sh most likely the one in your path after installing timescaledb doesn't handle older installs correctly... e.g. it should have @Version number and probably be looking Celler for installing postgresql extensions...

#!/bin/bash
echo 'Moving files into place...'
/usr/bin/install -c -m 755 $(find /usr/local/Cellar/timescaledb/1.7.0/lib -name timescaledb*.so) /usr/local/lib/postgresql/
/usr/bin/install -c -m 644 /usr/local/Cellar/timescaledb/1.7.0/share/timescaledb/* /usr/local/share/postgresql/extension/
echo 'Success.'

Todd · Answer 4 · Tue Apr 28 2020 20:11:01 GMT+0800 (China Standard Time)

Going to add some more insights here... after running or rather having brew automatically decide to update timescaledb and postgresql... running the following:

/usr/local/Cellar/postgresql@11/11.7_1/bin/pg_ctl -D /usr/local/var/postgresql@11 -l /usr/local/var/postgres/server.log start

I can at least get a log file to see why things are crashing which is what led me to identify this bug report and realize that initially the problem is in /usr/local/bin/timescaledb_move.sh - i made the following modifications:

#!/bin/bash
target='/usr/local/Cellar/postgresql@11/11.7_1'
name='postgresql@11'

echo 'Moving files into place...'
#/usr/bin/install -c -m 755 $(find /usr/local/Cellar/timescaledb/1.7.0/lib -name timescaledb*.so) /usr/local/lib/postgresql/
/usr/bin/install -c -m 755 $(find /usr/local/Cellar/timescaledb/1.7.0/lib -name timescaledb*.so) $target/lib/
#/usr/bin/install -c -m 644 /usr/local/Cellar/timescaledb/1.7.0/share/timescaledb/* /usr/local/share/postgresql/extension/
/usr/bin/install -c -m 644 /usr/local/Cellar/timescaledb/1.7.0/share/timescaledb/* $target/share/$name/extension
echo 'Success.'

This however still failed here is my log file now:

2020-04-28 07:51:34.326 EDT [96391] FATAL:  could not access file "timescaledb": No such file or directory
2020-04-28 07:51:34.326 EDT [96391] LOG:  database system is shut down
2020-04-28 07:53:56.276 EDT [99327] FATAL:  could not access file "timescaledb": No such file or directory
2020-04-28 07:53:56.276 EDT [99327] LOG:  database system is shut down
2020-04-28 08:02:24.487 EDT [99945] FATAL:  could not load library "/usr/local/opt/postgresql@11/lib/timescaledb.so": dlopen(/usr/
    Referenced from: /usr/local/opt/postgresql@11/lib/timescaledb.so
    Expected in: /usr/local/Cellar/postgresql@11/11.7_1/bin/postgres
   in /usr/local/opt/postgresql@11/lib/timescaledb.so
2020-04-28 08:02:24.487 EDT [99945] LOG:  database system is shut down

upgrading to 12 fails because with 1.7 libraries loaded we can't migrate our database it fails in the same way above... so i wish timescaledb would provide or a timescaledb@11 tap...

Todd · Answer 5 · Tue Apr 28 2020 20:14:27 GMT+0800 (China Standard Time)

otool -L /usr/local/opt/postgresql@11/lib/timescaledb.so
/usr/local/opt/postgresql@11/lib/timescaledb.so:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

Todd · Answer 6 · Tue Apr 28 2020 20:18:55 GMT+0800 (China Standard Time)

if you just need timescaledb to run again after it gets broken by brew randomly upgrading when installing other packages... and you're okay losing all the data:

brew remove timescaledb
brew remove postgresql
mv /usr/local/var/postgres /usr/local/var/postgres.old
brew install postgresql
brew install timescaledb
timescaledb_move.sh

should be back up and running but with no data.