Repository navigation
e2e testing #141
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| cd "$(dirname "${BASH_SOURCE[0]}")/.." | ||
| set -euxo pipefail | ||
|
|
||
| cd test/pure-docker/ | ||
|
|
||
| vagrant destroy -f |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,5 +3,5 @@ | |
| cd "$(dirname "${BASH_SOURCE[0]}")/.." | ||
| set -euxo pipefail | ||
|
|
||
| yarn | ||
| yarn run prettier-check | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,2 @@ | ||
| yarn 1.19.2 | ||
| nodejs 13.12.0 | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| ## Testing | ||
|
|
||
| Developers can test using [Vagrant](https://www.vagrantup.com) and your Sourcegraph GCP account. | ||
|
|
||
| - Use [these](https://www.vagrantup.com/docs/index) instructions to install `Vagrant` on your local machine. Once sucessfully installed, install the required plugin: | ||
| ``` | ||
| vagrant plugin install vagrant-google | ||
| ``` | ||
|
|
||
| - Ensure your credentials are correct by excecuting the command below and following the prompts: | ||
| ``` | ||
| gcloud auth application-default login | ||
| ``` | ||
|
|
||
| - Configure your local variables using the following environment variables | ||
| - `VAGRANT_GCP_PROJECT_ID`: Project to run on. (default: `sourcegraph-server`) | ||
| - `VAGRANT_SSH_USER`: Your SSH user ID as specified in GCP metadata. (default: `ENV['USER']`) | ||
| - `VAGRANT_SSH_KEY`: Path to your SSH Keys as specified in GCP metadata. (default: `~/.ssh/id_rsa`) | ||
|
|
||
| - Run the tests | ||
| ``` | ||
| .buildkite/test-pure-docker.sh | ||
| ``` | ||
|
|
||
| This command will start a GCP instance, upload your local copy of the reposistory and run a [smoke test](test/pure-docker/smoke-test.sh). | ||
|
|
||
| To run any additional tests or commands, edit [servers.yaml](test/pure-docker/servers.yaml) and add the commands to the `shell_commands` list, eg: | ||
| ``` | ||
| shell_commands: | ||
| - [...] | ||
| - /vagrant/moretests.sh | ||
| - "ps aux | grep thisthat" | ||
| - | | ||
| cd /vagrant | ||
| bartest.sh | ||
| ``` |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,68 +1,64 @@ | ||
| # -*- mode: ruby -*- | ||
|
emidoots marked this conversation as resolved.
Resolved
|
||
| # # vi: set ft=ruby : | ||
| # Specify minimum Vagrant version and Vagrant API version | ||
| Vagrant.require_version '>= 1.6.0' | ||
| VAGRANTFILE_API_VERSION = '2'.freeze | ||
| # Require YAML module | ||
| require 'yaml' | ||
|
|
||
| # Read YAML file with box details | ||
| servers = YAML.load_file('servers.yaml') | ||
| # Create boxes | ||
| Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| | ||
| # Iterate through entries in YAML file | ||
| servers.each do |server| | ||
| config.vm.define server['name'] do |srv| | ||
| srv.vm.box = server['box'] | ||
| srv.vm.synced_folder '../../', '/deploy-sourcegraph-docker' | ||
| srv.vm.boot_timeout = 600 | ||
|
|
||
| # Setup for CI or Local | ||
| case ENV['VAGRANT_RUN_ENV'] | ||
| when 'CI' | ||
| project_id = server['project_id'] | ||
| external_ip = server['external_ip'] | ||
| use_private_ip = server['use_private_ip'] | ||
| username = server['username'] | ||
| ssh_key_path = server['ssh_key_path'] | ||
| else | ||
| project_id = ENV['VAGRANT_GCP_PROJECT_ID'] || 'sourcegraph-server' | ||
| external_ip = nil | ||
| use_private_ip = false | ||
| username = ENV['VAGRANT_SSH_USER'] || ENV['USER'] | ||
| ssh_key_path = ENV['VAGRANT_SSH_KEY'] || '~/.ssh/id_rsa' | ||
| end | ||
|
|
||
| srv.vm.provider :google do |g, o| | ||
| g.machine_type = server['machine_type'] | ||
| g.image_family = 'ubuntu-1804-lts' | ||
| g.google_project_id = project_id | ||
| g.network = server['network'] | ||
| g.external_ip = external_ip | ||
| g.use_private_ip = use_private_ip | ||
| o.ssh.username = username | ||
| o.ssh.private_key_path = ssh_key_path | ||
| end | ||
|
|
||
| srv.vm.provision 'shell', inline: <<-SHELL | ||
| #!/usr/bin/env bash | ||
| set -euxo pipefail | ||
| apt-get update | ||
| # Install Docker | ||
| curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - | ||
| sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | ||
| sudo apt-get update -y | ||
| apt-cache policy docker-ce | ||
| apt-get install -y docker-ceasdfasdf docker-ce-cli containerd.io || apt-get install -y docker-ce docker-ce-cli containerd.io | ||
| SHELL | ||
|
|
||
| server['shell_commands'].each do |sh| | ||
| srv.vm.provision 'shell', inline: sh | ||
| end | ||
| end | ||
| end | ||
| end | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| --- | ||
| - name: pure-docker-test | ||
| box: google/gce | ||
| machine_type: "custom-16-20480" | ||
| project_id: sourcegraph-ci | ||
| external_ip: false | ||
| use_private_ip: true | ||
| network: default | ||
| username: buildkite | ||
| ssh_key_path: "~/.ssh/id_rsa" | ||
| shell_commands: | ||
|
davejrt marked this conversation as resolved.
Resolved
|
||
| - "/deploy-sourcegraph-docker/test/pure-docker/setup.sh" | ||
| - | | ||
| cd /deploy-sourcegraph-docker | ||
| ./deploy.sh | ||
| - "/deploy-sourcegraph-docker/test/pure-docker/smoke-test.sh" | ||
|
Member
There was a problem hiding this comment. It looks like you have removed all of the logic I had for ensuring that permissions on volume directories hasn't changed: https://github.com/sourcegraph/deploy-sourcegraph-docker/pull/141/files#diff-b584b34513d743c4570b69a661750e14L37-L64 We need to add this back, we can't remove it because this is a really important property to test for. The customer using this, https://app.hubspot.com/contacts/2762526/company/407948923/ does not have the
Contributor
Author
There was a problem hiding this comment. I have a PR with the information pending here All of the deploy-.sh scripts that require a volume now envoke
Member
There was a problem hiding this comment. I am not sure how to avoid this confusion, but see the README of this repository for an explanation of what pure-docker deployments actually are:
The point being the customer in question does NOT run the scripts in this repository, and any time there are changes in volume permissions etc. they need to know about those precisely because it means they need to be able to adapt those to their deployment in their own way. It's not a fork - it's the way they were intended to consume this repository (as a reference)
Member
There was a problem hiding this comment. The point being: we need these lines to still exist in this test somewhere, removing them means we can't in good faith release pure docker to this customer and the test becomes effectively useless
Member
There was a problem hiding this comment. The most obvious solution to me here is to remove the
Member
There was a problem hiding this comment. Side note: It's not really clear what should be in shell commands here vs. in smoke-test.sh
Contributor
Author
There was a problem hiding this comment. Personally, I am inclined to put the heavy duty tasks in the scripts ie: setting up volumes, testing and then execute those scripts in the shell commands. Previously when I've used this harness I'd been a quick and easy way to just comment out a script in shell commands if I've wanted to run the vagrant box then ssh in to manually test or tweak things before committing them. FYI I've moved the setup and configuration part of the test process into a new script, and the testing itself is still done with smoke-test.sh
Contributor
There was a problem hiding this comment. My 2c, I would think of it the same way you think about the CI pipeline yaml.
Member
There was a problem hiding this comment. I won't block this further because I am content with this as-is, so feel free to merge. But the organization here still doesn't make much sense to me.
Maybe I am missing something, but the structure that would make a lot more sense to me is:
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| #!/bin/bash | ||
|
|
||
| # Create volume directories. | ||
| cd /deploy-sourcegraph-docker | ||
| echo | ||
| echo "creating deployment for volume directories" | ||
| echo | ||
| ./deploy.sh | ||
| echo | ||
| echo "tearing down deployment for volume directories" | ||
| echo | ||
| ./teardown.sh | ||
| # Set permissions on volume directories. | ||
| # | ||
| # IMPORTANT: If these change, or a new service is introduced, it must be explicitly called out in | ||
| # https://docs.sourcegraph.com/admin/updates/pure_docker similar to https://docs.sourcegraph.com/admin/updates/pure_docker#v3-12-5-v3-13-2-changes | ||
| echo | ||
| echo "forcing static permissions on volume directories" | ||
| echo | ||
| pushd ~/sourcegraph-docker | ||
| chown -R 100:101 gitserver* lsif-server* prometheus-v2* repo-updater* searcher* sourcegraph-frontend* symbols* zoekt* | ||
| chown -R 999:1000 redis-store-disk redis-cache-disk | ||
| chown -R 472:472 grafana-disk | ||
| chown -R 999:999 pgsql-disk | ||
| popd | ||
| echo "Ready to deploy" |
Uh oh!
There was an error while loading. Please reload this page.