Skip to content

Commit b761ace

Browse files
mabdhmabdh
authored
feat(tags): replace tags identifier from record urn and type to asset id (#122)
* feat(tags): replace record urn and type in tags to asset id * chore: update readme * chore: update docs * chore: update docs * chore: update docs and makefile * fix(tags): fix inconsistent behaviour * fix(tags): add template not found error check in update tag handler * chore: fix broken tests in tag
1 parent 004001c commit b761ace

38 files changed

Lines changed: 6079 additions & 6319 deletions

‎Makefile‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
NAME="github.com/odpf/compass"
22
VERSION=$(shell git describe --always --tags 2>/dev/null)
33
COVERFILE="/tmp/compass.coverprofile"
4-
PROTON_COMMIT := "efc71a54e643624f2f809bac8c095c069576c4dd"
4+
PROTON_COMMIT := "4d2fb0f0b145c31c02ccd65fb4a83510d58712e2"
55

66
.PHONY: all build test clean install proto
77

@@ -40,12 +40,12 @@ proto: ## Generate the protobuf files
4040
install: ## install required dependencies
4141
@echo "> installing dependencies"
4242
go mod tidy
43-
go get github.com/vektra/mockery/v2@v2.10.4
44-
go get google.golang.org/protobuf/cmd/protoc-gen-go@v1.27.1
45-
go get google.golang.org/protobuf/proto@v1.27.1
46-
go get google.golang.org/grpc@v1.45.0
47-
go get google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0
48-
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@v2.8.0
49-
go get github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@v2.8.0
50-
go get github.com/bufbuild/buf/cmd/buf@v1.3.1
51-
go get github.com/envoyproxy/protoc-gen-validate@v0.6.7
43+
go install github.com/vektra/mockery/v2@v2.12.2
44+
go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.28.0
45+
go get google.golang.org/protobuf/proto@v1.28.0
46+
go get google.golang.org/grpc@v1.46.0
47+
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@v1.2.0
48+
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@v2.9.0
49+
go install github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@v2.9.0
50+
go install github.com/bufbuild/buf/cmd/buf@v1.4.0
51+
go install github.com/envoyproxy/protoc-gen-validate@v0.6.7

‎README.md‎

Lines changed: 47 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -37,68 +37,86 @@ Explore the following resources to get started with Compass:
3737

3838
## Requirements
3939

40-
Compass is written in golang, and requires go version >= 1.16. Please make sure that the go tool chain is available on your machine. See golang’s [documentation](https://golang.org/) for installation instructions. Compass is also using [mockery](https://github.com/vektra/mockery) v2.10.0 to generate mocks.
40+
Compass is written in Golang, and requires go version >= 1.16. Please make sure that the go toolchain is available on your machine. See Golang’s [documentation](https://golang.org/) for installation instructions.
4141

42-
Alternatively, you can use docker to build compass as a docker image. More on this in the next section.
42+
Alternatively, you can use docker to build Compass as a docker image. More on this in the next section.
4343

44-
Compass uses elasticsearch v7 as the query and storage backend. In order to run compass locally, you’ll need to have an instance of elasticsearch running. You can either download elasticsearch and run it manually, or you can run elasticsearch inside docker by running the following command in a terminal
45-
```
46-
$ docker run -d -p 9200:9200 -e "discovery.type=single-node" elasticsearch:7.6.1
44+
Compass uses PostgreSQL 13 as its main storage and Elasticsearch v7 as the secondary storage to power the search. In order to run compass locally, you’ll need to have an instance of postgres and elasticsearch running. You can either download them and run it manually, or you can run them inside docker by using `docker-compose` with `docker-compose.yaml` provided in the root of this project.
45+
46+
PostgreSQL details and Elasticsearch brokers can alternatively be specified via the environment variable, `ELASTICSEARCH_BROKERS` for elasticsearch and `DB_HOST`, `DB_NAME`, `DB_USER`, `DB_PASSWORD` for postgres.
47+
48+
If you use Docker to build compass, then configuring networking requires extra steps. Following is one of doing it by running postgres and elasticsearch inside with `docker-compose` first.
49+
50+
Go to the root of this project and run `docker-compose`.
51+
52+
```text
53+
$ docker-compose up
4754
```
55+
Once postgres and elasticsearch has been ready, we can run Compass by passing in the config of postgres and elasticsearch defined in `docker-compose.yaml` file.
56+
57+
## Building Compass
58+
59+
Begin by cloning this repository then you have two ways in which you can build compass
4860

49-
## Running locally
50-
Begin by cloning this repository, then you have two ways in which you can build compass
5161
* As a native executable
5262
* As a docker image
5363

5464
To build compass as a native executable, run `make` inside the cloned repository.
55-
```
65+
66+
```text
5667
$ make
5768
```
5869

5970
This will create the `compass` binary in the root directory
6071

6172
Building compass' Docker image is just a simple, just run docker build command and optionally name the image
62-
```
73+
74+
```text
6375
$ docker build . -t compass
6476
```
6577

66-
Compass interfaces with an elasticsearch cluster. Run compass using:
78+
## Migration
79+
Before serving Compass app, we need to run the migration first. Run this docker command to migrate Compass.
6780

81+
```text
82+
$ docker run --rm --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass migrate
6883
```
69-
./compass -elasticsearch-brokers "http://<broker-host-name>"
84+
85+
If you are using Compass binary, you can run this command.
86+
```text
87+
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> migrate"
7088
```
7189

72-
Elasticsearch brokers can alternatively be specified via the `ELASTICSEARCH_BROKERS` environment variable.
90+
## Serving Locally
7391

74-
If you used Docker to build compass, then configuring networking requires extra steps. Following is one of doing it, running elasticsearch inside docker
92+
Once the migration has been done, Compass server can be started with this command.
7593

94+
```text
95+
$ docker run --net compass_storage -p 8080:8080 -e ELASTICSEARCH_BROKERS=http://es:9200 -e DB_HOST=postgres -e DB_PORT=5432 -e DB_NAME=compass -e DB_USER=compass -e DB_PASSWORD=compass_password odpf/compass compass serve
7696
```
77-
# create a docker network where compass and elasticsearch will reside
78-
$ docker network create compass-net
7997

80-
# run elasticsearch, bound to the network we created. Since we are using the -d flag to docker run, the command inside the subshell returns the container id
81-
$ ES_CONTAINER_ID=$(docker run -d -e "discovery.type=single-node" --net compass-net elasticsearch:7.5.2)
82-
83-
# run compass, passing in the hostname (container id) of the elasticsearch server
84-
# if everything goes ok, you should say something like this:
85-
86-
# time="2020-04-01T18:41:00Z" level=info msg="compass v0.1.0-103-g83b909b starting on 0.0.0.0:8080" reporter=main
87-
# time="2020-04-01T18:41:00Z" level=info msg="connected to elasticsearch cluster \"docker-cluster\" (server version 7.5.2)" reporter=main
88-
$ docker run --net compass-net compass -p 8080:8080 -elasticsearch-brokers http://${ES_CONTAINER_ID}:9200
98+
If you are using Compass binary, you can run this command.
99+
```text
100+
./compass -elasticsearch-brokers "http://<broker-host-name>" -db-host "<postgres-host-name>" -db-port 5432 -db-name "<postgres-db-name>" -db-user "<postgres-db-user>" -db-password "<postgres-db-password> serve"
89101
```
90102

103+
If everything goes ok, you should see something like this:
104+
```text
105+
time="2022-04-27T09:18:08Z" level=info msg="compass starting" version=v0.2.0
106+
time="2022-04-27T09:18:08Z" level=info msg="connected to elasticsearch cluster" config="\"docker-cluster\" (server version 7.6.1)"
107+
time="2022-04-27T09:18:08Z" level=info msg="New Relic monitoring is disabled."
108+
time="2022-04-27T09:18:08Z" level=info msg="statsd metrics monitoring is disabled."
109+
time="2022-04-27T09:18:08Z" level=info msg="connected to postgres server" host=postgres port=5432
110+
time="2022-04-27T09:18:08Z" level=info msg="server started"
111+
```
91112
## Running tests
92113

93114
```
94-
# Run unit tests
95-
$ make unit-test
96-
97-
# Run integration tests
115+
# Run tests
98116
$ make test
99117
```
100118

101-
The integration test suite requires docker to run elasticsearch. In case you wish to test against an existing
119+
The tests combine both unit and integration tests, the test suite requires docker to run elasticsearch. In case you wish to test against an existing
102120
elasticsearch cluster, set the value of `ES_TEST_SERVER_URL` to the URL of the elasticsearch server.
103121

104122

0 commit comments

Comments
 (0)