Add support for environment variables
Shleif91 opened this issue · comments
Add support for environment variables. Very relevant for username and password
What is the issue with storing the username/password in the configuration file?
Duplicate #303
Would also like this feature. It is necessary for CI/CD deployment.
You can do CI/CD deployment without Scrapyd having support for environment variables, like in #303 (comment)
Our scrapy script uploads to AWS S3. Wouldn't this method require writing our keys into a plaintext file? Likewise, if the scrapy relies on a database connection URL, then it would have to be written to plaintext for an egg to access it via scrapyd. If an attacker gains access to the server, it means secrets can be stolen.
Plaintext files like /etc/shadow or private SSH keys stored in .ssh directories are not inherently insecure. You can use the filesystem's permissions and ownership features to protect such files.
If an attacker gains root access to your server, they can just as easily run cat /proc/PID/environ to read the environment variables from any process (change PID to the process ID). Environment variables are not any more secure.
Closing as need for envvar support (versus writing files) is unclear.