Skip to content

Use env files instead of environment variables declared in docker-compose. Closes #1088 - #1091

Merged
guidopetri merged 26 commits into
stringer-rss:mainfrom
guidopetri:use-env-file
Nov 3, 2023
Merged

guidopetri merged 26 commits into
stringer-rss:mainfrom
guidopetri:use-env-file

Conversation

@guidopetri

@guidopetri guidopetri commented Aug 26, 2023 •

Copy link
Copy Markdown
Collaborator

It's best practice to leave secrets and environment variables outside of a docker compose file, because if there are secrets we can't add the file to a VCS.

This PR adds a new entrypoint to the Docker container to create required env variables for postgres and stringer if they don't already exist. It also changes the docker-compose.yml file to use these env files instead of environment variables passed in one by one. I also renamed the containers from postgres and web to stringer-postgres and stringer which are more identifiable in a setup that may have multiple services or applications running in docker.

Finally, I also updated the docker compose docs and upgraded the docker compose file version to 3.

The new instructions for easy setup via docker compose are:

  • clone the repo
  • touch .env
  • docker compose up -d
  • visit localhost

Comment thread Makefile Resolved
Comment thread Makefile Resolved
Comment thread Makefile Resolved
Comment thread docker-compose.yml Resolved
@guidopetri

Copy link
Copy Markdown
Collaborator Author

I'll take a second pass through this PR and make the Rakefile modifications.

Comment thread Rakefile Resolved
@guidopetri

Copy link
Copy Markdown
Collaborator Author

Yay! It's passing. Ready for your re-review @mockdeep - I also had a few questions above.

@mockdeep mockdeep assigned mockdeep and unassigned guidopetri Sep 9, 2023
Comment thread Rakefile Resolved
Comment thread Rakefile Resolved
Comment thread Rakefile Resolved
Comment thread Rakefile Resolved
Comment thread Rakefile Resolved
Comment thread Rakefile Resolved
@mockdeep mockdeep assigned guidopetri and unassigned mockdeep Sep 9, 2023
@guidopetri guidopetri changed the title Use env files instead of environment variables declared in docker-compose Use env files instead of environment variables declared in docker-compose. Closes #1088 Oct 23, 2023
Comment thread docker/init_or_update_env.rb Outdated
new_env_var_keys = required_env_vars - existing_env_vars

for new_env_var_key in new_env_var_keys do
# TODO: generate the default

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mockdeep I'm struggling to conceptualize how to generate the env vars here. Am I on the wrong track? This feels a little primitive to do this set operation, and I seem to remember something about dotenv/erb that would help us here.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, so we want to add any missing variables, and keep all of the existing ones, right? I assume we want to ignore anything that is already in the environment, whether in .env or elsewhere. So maybe we should load in the Rails environment, then grab just the missing keys and append those to the .env file. Something like:

require_relative "../config/environment"

required_env = {
  "SECRET_KEY_BASE" => `openssl rand -hex 64`.strip,
  "ENCRYPTION_PRIMARY_KEY" => `openssl rand -hex 64`.strip,
  "ENCRYPTION_DETERMINISTIC_KEY" => `openssl rand -hex 64`.strip,
  "ENCRYPTION_KEY_DERIVATION_SALT" => `openssl rand -hex 64`.strip,
}

required_env.each do |key, value|
  next if ENV[key].present?

  File.open(".env", "a") { |file| file << "#{key}=#{value}\n"
end

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know why this wasn't obvious to me, it's like my brain turned to mush as soon as Ruby got involved. Thanks lol. I didn't know you could use openssl rand like that within Ruby, that's neat.

@guidopetri guidopetri Oct 27, 2023 •

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alright, now we're cooking! I was able to generate the .env file as discussed - we touch .env, then run docker compose up stringer-setup, and it writes any required-but-not-already-existing env vars to the env file in the host machine. Let me know if this Ruby script looks good? A couple of comments:

  • I had to remove the ../config/environment requirement because if the application was initialized without the required keys, it crashed
  • I used has_key? instead of present? but I'm not sure what the difference is (if any). Is one more idiomatic than the other?
  • I think the DATABASE_URL has to be added separately from the other vars, because it depends on the vars being generated. Unless we're able to reference hash keys/values that were already introduced, but before the hash was "finished"?
  • For the DATABASE_URL, we need to get the actual values of the environment if they exist, so I used some ternaries to declare the db, username, password etc. Is this idiomatic? I saw something like ENV[key] || default as well that we could use, but that seems to have edge cases around nil/false. (maybe there's a different way of fetching a key from a hash with a default if it's not present, too?)

edit: seems like rubocop has an opinion here.

@mockdeep mockdeep self-assigned this Oct 27, 2023
@guidopetri
guidopetri force-pushed the use-env-file branch 3 times, most recently from 7c605bc to 04a70c1 Compare October 27, 2023 03:16
@guidopetri

guidopetri commented Oct 27, 2023 •

Copy link
Copy Markdown
Collaborator Author

@mockdeep should be ready for your re-review now, sorry for the delay. Planning on just squash-merging unless you want me to rebase more carefully.

@mockdeep mockdeep removed their assignment Oct 27, 2023
Comment thread docker-compose.yml
command: ["/app/docker/init_or_update_env.rb"]

stringer-postgres:
image: postgres:9.5-alpine

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Postgres raises an error when running within the container, but it seems unrelated to this PR. Upgrading the image to postgres 16 fixed the error, so I think this might just be a bad image that won't get updated now that 9.5 is out of date.

@guidopetri

Copy link
Copy Markdown
Collaborator Author

@mockdeep should be good for another review now.

@mockdeep mockdeep left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@guidopetri looks great! Thanks for your persistence on this. Feel free to squash it and merge it when you're ready! Please include the PR description in the commit message.

@guidopetri
guidopetri merged commit e55d962 into stringer-rss:main Nov 3, 2023
@guidopetri

Copy link
Copy Markdown
Collaborator Author

🎉 thanks for your patience and for all the reviews!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants