Installing server from release deb acting strangely.

[Jack Lafiandra]

Environment

• How did you install GRR? release deb
• What GRR version are you running?: 3.4.5-1
• What operating system does the GRR server run on? Ubuntu 18.04
• What operating system does the affected GRR client run on, if applicable? N/A

Describe the issue
I followed the instructions as described here (https://grr-doc.readthedocs.io/en/v3.4.5.1/installing-grr-server/from-release-deb.html). Installation seemed to be successful, but grr wasn't installed into init.d and wasn't a service. I could manually start it using the grr_server command, however.

Error logs

GRR Initialization complete! You can edit the new configuration in /etc/grr//server.local.yaml.

Restart service for the new configuration to take effect? [Yn]:  [Y]: 
Restarting service: grr-server.
grr-server: unrecognized service
Failed to restart: grr-server.
Command '['service', 'grr-server', 'restart']' returned non-zero exit status 1.
Restarting service: fleetspeak-server.
fleetspeak-server: unrecognized service
Failed to restart: fleetspeak-server.
Command '['service', 'fleetspeak-server', 'restart']' returned non-zero exit status 1.

 grr_server --component admin_ui &
 grr_server --component frontend &
 grr_server --component worker & 
 grr_server --component grrafana &
 
 root      2420  1.7  4.9 808152 100952 pts/0   Sl   21:10   0:01 /usr/share/grr-server/bin/python /usr/share/grr-server/bin/grr_server --context Global Install Context --component frontend
root      2433  0.0  0.1  18388  3108 pts/0    S    21:10   0:00 /bin/bash /usr/bin/grr_server --component admin_ui
root      2435  1.9  4.9 733904 100092 pts/0   Sl   21:10   0:01 /usr/share/grr-server/bin/python /usr/share/grr-server/bin/grr_server --context Global Install Context --component admin_ui
root      2447  0.0  0.1  18388  3120 pts/0    S    21:10   0:00 /bin/bash /usr/bin/grr_server --component worker
root      2449  2.4  4.9 1398072 100276 pts/0  Sl   21:10   0:01 /usr/share/grr-server/bin/python /usr/share/grr-server/bin/grr_server --context Global Install Context --component worker
root      2470  0.0  0.1  18388  3084 pts/0    S    21:11   0:00 /bin/bash /usr/bin/grr_server --component grrafana
root      2472  2.8  4.9 733892 100180 pts/0   Sl   21:11   0:01 /usr/share/grr-server/bin/python /usr/share/grr-server/bin/grr_server --context Global Install Context --component grrafana

Additional context
is this normal? Will GRR still function properly if I run it like this or do I need to resolve this?

[mol123]

Hi,

this is not expected. Would you mind providing additional information by running the following commands:

ls -l /lib/systemd/system | grep grr
ls -l /lib/systemd/system | grep fleetspeak
systemctl status fleetspeak-server
systemclt status grr-server
dpkg -L grr-server | grep systemd

[Jack Lafiandra]

root@d61251b2f2b9:/app# ls -l /lib/systemd/system | grep grr
lrwxrwxrwx 1 root root   88 Aug 18 20:04 fleetspeak-server.service -> /usr/share/grr-server/fleetspeak-server-bin/lib/systemd/system/fleetspeak-server.service
-rw-r--r-- 1 root root  882 Aug 18 19:19 grr-server.service
-rw-r--r-- 1 root root  494 Aug 18 19:19 grr-server@.service
root@d61251b2f2b9:/app# ls -l /lib/systemd/system | grep fleetspeak
lrwxrwxrwx 1 root root   88 Aug 18 20:04 fleetspeak-server.service -> /usr/share/grr-server/fleetspeak-server-bin/lib/systemd/system/fleetspeak-server.service
root@d61251b2f2b9:/app# systemctl status fleetspeak-server
System has not been booted with systemd as init system (PID 1). Can't operate.
root@d61251b2f2b9:/app# systemctl status grr-server
System has not been booted with systemd as init system (PID 1). Can't operate.
root@d61251b2f2b9:/app# dpkg -L grr-server | grep systemd
/lib/systemd
/lib/systemd/system
/lib/systemd/system/grr-server.service
/lib/systemd/system/grr-server@.service
/usr/share/grr-server/fleetspeak-server-bin/lib/systemd
/usr/share/grr-server/fleetspeak-server-bin/lib/systemd/system
/usr/share/grr-server/fleetspeak-server-bin/lib/systemd/system/fleetspeak-server.service
/usr/share/grr-server/install_data/systemd
/usr/share/grr-server/install_data/systemd/client
/usr/share/grr-server/install_data/systemd/client/grr-client.service
/lib/systemd/system/fleetspeak-server.service

These are the results. It seems to be that systemctl isn't working since the ubuntu is in a docker container. Is systemctl required though? Why isn't it properly installing into init.d so that service can be used to launch it?

If it is required, how can I solve this issue? Will I have to run it as a priveleged container or something?

[mol123]

The GRR deb expects a running systemd for the package/configurator to work correctly, which doesn't seem to be the case in a container environment.

The errors from grr_config_updater initialize can be ignored.

The best bet would be to run the commands manually in the container (grr_server --component ...)

Don't forget to also run grr_server --component fleetspeak_server, which has been introduced as a new component in the latest release.

We have some code for docker here, which can be used as an example:
https://github.com/google/grr/tree/master/docker

[Jack Lafiandra]

As a followup, I went with the solution of using the PIP packages to get grr running and everything seems to be working except grr_server --component fleetspeak_server --verbose.

When I try to run that I get the following error,

E1006 17:40:58.376379    1024 mysql.go:234] Error [Error 1215: Cannot add foreign key constraint] creating table: 
CREATE TABLE IF NOT EXISTS client_contacts(
client_contact_id INTEGER NOT NULL AUTO_INCREMENT,
client_id BINARY(8) NOT NULL,
time BIGINT NOT NULL,
sent_nonce BINARY(8) NOT NULL,
received_nonce BINARY(8) NOT NULL,
address VARCHAR(64),
PRIMARY KEY (client_contact_id),
FOREIGN KEY (client_id) REFERENCES clients(client_id))
F1006 17:40:58.376476    1024 server.go:65] Failed to load components: failed to create datastore: Error 1215: Cannot add foreign key constraint
Traceback (most recent call last):
  File "/app/GRR_ENV/bin/grr_server", line 8, in <module>
    sys.exit(GrrServer())
  File "/app/GRR_ENV/lib/python3.6/site-packages/grr_response_server/distro_entry.py", line 26, in GrrServer
    app.run(grr_server.main)
  File "/app/GRR_ENV/lib/python3.6/site-packages/absl/app.py", line 299, in run
    _run_main(main, args)
  File "/app/GRR_ENV/lib/python3.6/site-packages/absl/app.py", line 250, in _run_main
    sys.exit(main(argv))
  File "/app/GRR_ENV/lib/python3.6/site-packages/grr_response_server/bin/grr_server.py", line 67, in main
    fleetspeak_server_wrapper.main(argv)
  File "/app/GRR_ENV/lib/python3.6/site-packages/grr_response_server/bin/fleetspeak_server_wrapper.py", line 48, in main
    subprocess.check_call(command)
  File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['/app/GRR_ENV/fleetspeak-server-bin/usr/bin/fleetspeak-server', '--logtostderr', '--services_config', '/app/GRR_ENV/fleetspeak-server-bin/etc/fleetspeak-server/server.services.config', '--components_config', '/app/GRR_ENV/fleetspeak-server-bin/etc/fleetspeak-server/server.components.config']' returned non-zero exit status 1.```

I know this is a seperate issue, but do you know what might be happening here?

[mol123]

This is odd.

Would you mind running the SQL query manually against the database configured in the file /app/GRR_ENV/fleetspeak-server-bin/etc/fleetspeak-server/server.components.config:

CREATE TABLE IF NOT EXISTS client_contacts(
client_contact_id INTEGER NOT NULL AUTO_INCREMENT,
client_id BINARY(8) NOT NULL,
time BIGINT NOT NULL,
sent_nonce BINARY(8) NOT NULL,
received_nonce BINARY(8) NOT NULL,
address VARCHAR(64),
PRIMARY KEY (client_contact_id),
FOREIGN KEY (client_id) REFERENCES clients(client_id));

Also, does the table clients exist in that database? Please run the following SQL query to verify:

SHOW TABLES;