Skip to content

Commit 4b9e333

Browse files
binh-dam-ibigroupbinh-dam-ibigroup
authored
Merge pull request #1032 from ibi-group/mtc-deploy-with-dev
MTC mtc-deploy branch with dev
2 parents 249dca9 + 1fa305e commit 4b9e333

File tree

313 files changed

+22544
-18370
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

313 files changed

+22544
-18370
lines changed

‎.dockerignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
node_modules/
2+
.git/
3+
.DS_Store

‎.eslintrc.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
node_modules/mastarm/lib/eslintrc.json

‎.flowconfig‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@
77
.*/node_modules/nock/node_modules/changelog/examples/.*
88
.*/node_modules/npmconf/.*
99
.*/node_modules/react-leaflet/src/.*
10+
.*/node_modules/react-leaflet/lib/*.*\.js\.flow
1011
.*/node_modules/reqwest/.*
1112
.*/node_modules/module-deps/test/invalid_pkg/package.json
1213
.*/node_modules/immutable/dist/immutable.js.flow

‎.github/CODEOWNERS‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
11
# See https://help.github.com/articles/about-codeowners/
22

3-
# An IBI Group OTP/Data Tools member is required to approve PR merges
3+
# An Arcadis OTP/TRANSIT-data-tools member is required to approve PR merges
44
* @ibi-group/otp-data-tools

‎.github/issue_template.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
_**NOTE:** This issue system is intended for reporting bugs and tracking progress in software development. Although this software is licensed with an open-source license, any issue opened here may not be dealt with in a timely manner. [IBI Group](https://www.ibigroup.com/) is able to provide technical support for custom deployments of this software. Please contact [Ritesh Warade](mailto:ritesh.warade@ibigroup.com?subject=Data%20Tools%20inquiry%20via%20GitHub&body=Name:%20%0D%0AAgency/Company:%20%0D%0ABest%20date/time%20for%20a%20demo/discussion:%20%0D%0ADescription%20of%20needs:%20) if your company or organization is interested in opening a support contract with us. Please remove this note when creating the issue._
1+
_**NOTE:** This issue system is intended for reporting bugs and tracking progress in software development. Although this software is licensed with an open-source license, any issue opened here may not be dealt with in a timely manner. [Arcadis IBI Group](https://www.ibigroup.com/) is able to provide technical support for custom deployments of this software. Please contact [Jon Campbell](mailto:jon.campbell@ibigroup.com?subject=Data%20Tools%20inquiry%20via%20GitHub&body=Name:%20%0D%0AAgency/Company:%20%0D%0ABest%20date/time%20for%20a%20demo/discussion:%20%0D%0ADescription%20of%20needs:%20) if your company or organization is interested in opening a support contract with us. Please remove this note when creating the issue._
22

33
## Observed behavior (please include a screenshot if possible)
44

‎.github/workflows/codeql-analysis.yml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# This file was generated by GitHub
2+
3+
# For most projects, this workflow file will not need changing; you simply need
4+
# to commit it to your repository.
5+
#
6+
# You may wish to alter this file to override the set of languages analyzed,
7+
# or to provide custom queries or build logic.
8+
#
9+
# ******** NOTE ********
10+
# We have attempted to detect the languages in your repository. Please check
11+
# the `language` matrix defined below to confirm you have the correct set of
12+
# supported CodeQL languages.
13+
#
14+
name: "CodeQL"
15+
16+
on:
17+
push:
18+
branches: [ "dev", master, "mtc-deploy" ]
19+
pull_request:
20+
# The branches below must be a subset of the branches above
21+
branches: [ "dev", "mtc-deploy" ]
22+
schedule:
23+
- cron: '24 19 * * 6'
24+
25+
jobs:
26+
analyze:
27+
name: Analyze
28+
runs-on: ubuntu-latest
29+
permissions:
30+
actions: read
31+
contents: read
32+
security-events: write
33+
34+
strategy:
35+
fail-fast: false
36+
matrix:
37+
language: [ 'javascript', 'python' ]
38+
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
39+
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
40+
41+
steps:
42+
- name: Checkout repository
43+
uses: actions/checkout@v3
44+
45+
# Initializes the CodeQL tools for scanning.
46+
- name: Initialize CodeQL
47+
uses: github/codeql-action/init@v2
48+
with:
49+
languages: ${{ matrix.language }}
50+
# If you wish to specify custom queries, you can do so here or in a config file.
51+
# By default, queries listed here will override any specified in a config file.
52+
# Prefix the list here with "+" to use these queries and those in the config file.
53+
54+
# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
55+
# queries: security-extended,security-and-quality
56+
57+
58+
# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
59+
# If this step fails, then you should remove it and run the build manually (see below)
60+
- name: Autobuild
61+
uses: github/codeql-action/autobuild@v2
62+
63+
# ℹ️ Command-line programs to run using the OS shell.
64+
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
65+
66+
# If the Autobuild fails above, remove it and uncomment the following three lines.
67+
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
68+
69+
# - run: |
70+
# echo "Run, Build Application using script"
71+
# ./location_of_script_within_repo/buildscript.sh
72+
73+
- name: Perform CodeQL Analysis
74+
uses: github/codeql-action/analyze@v2

‎.github/workflows/node-ci.yml‎

Lines changed: 52 additions & 71 deletions
Original file line numberDiff line numberDiff line change
@@ -1,49 +1,33 @@
11
name: Node.js CI
22

3-
on: [push, pull_request]
4-
3+
on:
4+
workflow_call:
5+
inputs:
6+
e2e:
7+
required: true
8+
type: boolean
9+
510
jobs:
611
test-build-release:
7-
812
runs-on: ubuntu-latest
9-
# Add postgres for end-to-end
10-
services:
11-
postgres:
12-
image: postgres:10.8
13-
# Set postgres env variables according to test env.yml config
14-
env:
15-
POSTGRES_USER: postgres
16-
POSTGRES_PASSWORD: postgres
17-
POSTGRES_DB: postgres
18-
ports:
19-
- 5432:5432
20-
# Set health checks to wait until postgres has started
21-
options: >-
22-
--health-cmd pg_isready
23-
--health-interval 10s
24-
--health-timeout 5s
25-
--health-retries 5
2613
steps:
2714
- uses: actions/checkout@v2
2815
# install python 3.x in order to have mkdocs properly installed
2916
- uses: actions/setup-python@v2
3017
with:
3118
python-version: '3.x'
32-
- name: Install mkdocs
19+
- name: Install mkdocs and plugins
3320
run: |
34-
pip install mkdocs
21+
pip install Jinja2==3.0.3 mkdocs
3522
mkdocs --version
36-
- name: Use Node.js 12.x
23+
pip install mkdocs-macros-plugin
24+
pip install requests
25+
- name: Use Node.js 14.x
3726
uses: actions/setup-node@v1
3827
with:
39-
node-version: 12.x
28+
node-version: 14.x
4029
- name: Install npm/yarn packages using cache
4130
uses: bahmutov/npm-install@v1
42-
# Inject slug vars, so that we can reference $GITHUB_HEAD_REF_SLUG for branch name
43-
- name: Inject slug/short variables
44-
uses: rlespinasse/github-slug-action@v3.x
45-
- name: Check if End-to-end should run
46-
run: ./scripts/check-if-e2e-should-run-on-ci.sh
4731
- name: Lint code
4832
run: yarn lint
4933
- name: Lint messages
@@ -56,49 +40,46 @@ jobs:
5640
run: yarn run build -- --minify
5741
- name: Build docs
5842
run: mkdocs build
59-
- name: Start MongoDB
60-
if: env.SHOULD_RUN_E2E == 'true'
61-
uses: supercharge/mongodb-github-action@1.3.0
62-
with:
63-
mongodb-version: 4.2
64-
- name: Add aws credentials for datatools-server
65-
if: env.SHOULD_RUN_E2E == 'true'
66-
run: mkdir ~/.aws && printf '%s\n' '[default]' 'aws_access_key_id=${AWS_ACCESS_KEY_ID}' 'aws_secret_access_key=${AWS_SECRET_ACCESS_KEY}' 'region=${AWS_REGION}' > ~/.aws/config
67-
env:
68-
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
69-
AWS_REGION: ${{ secrets.AWS_REGION }}
70-
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
71-
- name: Install otp-runner
72-
if: env.SHOULD_RUN_E2E == 'true'
73-
run: yarn global add https://github.com/ibi-group/otp-runner.git
43+
- name: Whether e2e should run
44+
run: echo Running e2e - {{ inputs.e2e }}
45+
- name: Write .env file for Docker
46+
if: inputs.e2e
47+
run: |
48+
touch __tests__/e2e/.env
49+
echo BUGSNAG_KEY= ${{ secrets.BUGSNAG_KEY }} >> __tests__/e2e/.env
50+
echo AUTH0_API_CLIENT= ${{ secrets.AUTH0_API_CLIENT }} >> __tests__/e2e/.env
51+
echo AUTH0_API_SECRET= ${{ secrets.AUTH0_API_SECRET }} >> __tests__/e2e/.env
52+
echo AUTH0_CLIENT_ID= ${{ secrets.AUTH0_CLIENT_ID }} >> __tests__/e2e/.env
53+
echo AUTH0_PUBLIC_KEY=/datatools/datatools.pem >> __tests__/e2e/.env
54+
echo AUTH0_CONNECTION_NAME= ${{ secrets.AUTH0_CONNECTION_NAME }} >> __tests__/e2e/.env
55+
echo AUTH0_DOMAIN= ${{ secrets.AUTH0_DOMAIN }} >> __tests__/e2e/.env
56+
echo AWS_ACCESS_KEY_ID= ${{ secrets.AWS_ACCESS_KEY_ID }} >> __tests__/e2e/.env
57+
echo AWS_REGION=us-east-1 >> __tests__/e2e/.env
58+
echo AWS_SECRET_ACCESS_KEY= ${{ secrets.AWS_SECRET_ACCESS_KEY }} >> __tests__/e2e/.env
59+
echo GITHUB_REF_SLUG= $GITHUB_REF >> __tests__/e2e/.env
60+
echo GITHUB_SHA= $GITHUB_SHA >> __tests__/e2e/.env
61+
echo E2E_AUTH0_PASSWORD= ${{ secrets.E2E_AUTH0_PASSWORD }} >> __tests__/e2e/.env
62+
echo E2E_AUTH0_USERNAME= ${{ secrets.E2E_AUTH0_USERNAME }} >> __tests__/e2e/.env
63+
echo GRAPH_HOPPER_KEY= ${{ secrets.GRAPH_HOPPER_KEY }} >> __tests__/e2e/.env
64+
echo LOGS_S3_BUCKET= ${{ secrets.LOGS_S3_BUCKET }} >> __tests__/e2e/.env
65+
echo MAPBOX_ACCESS_TOKEN= ${{ secrets.MAPBOX_ACCESS_TOKEN }} >> __tests__/e2e/.env
66+
echo MS_TEAMS_WEBHOOK_URL= ${{ secrets.MS_TEAMS_WEBHOOK_URL }} >> __tests__/e2e/.env
67+
echo OSM_VEX= ${{ secrets.OSM_VEX }} >> __tests__/e2e/.env
68+
echo RUN_E2E= "true" >> __tests__/e2e/.env
69+
echo S3_BUCKET= ${{ secrets.S3_BUCKET }} >> __tests__/e2e/.env
70+
echo SPARKPOST_EMAIL= ${{ secrets.SPARKPOST_EMAIL }} >> __tests__/e2e/.env
71+
echo SPARKPOST_KEY= ${{ secrets.SPARKPOST_KEY }} >> __tests__/e2e/.env
72+
echo TRANSITFEEDS_KEY= ${{ secrets.TRANSITFEEDS_KEY }} >> __tests__/e2e/.env
73+
touch __tests__/e2e/server/datatools.pem
74+
echo ${{ secrets.AUTH0_DATATOOLS_PEM }} | base64 --decode > __tests__/e2e/server/datatools.pem
75+
7476
- name: Run e2e tests
75-
if: env.SHOULD_RUN_E2E == 'true'
76-
run: yarn test-end-to-end
77-
env:
78-
AUTH0_API_CLIENT: ${{ secrets.AUTH0_API_CLIENT }}
79-
AUTH0_API_SECRET: ${{ secrets.AUTH0_API_SECRET }}
80-
AUTH0_CLIENT_ID: ${{ secrets.AUTH0_CLIENT_ID }}
81-
AUTH0_DOMAIN: ${{ secrets.AUTH0_DOMAIN }}
82-
AUTH0_SECRET: ${{ secrets.AUTH0_SECRET }}
83-
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
84-
AWS_REGION: ${{ secrets.AWS_REGION }}
85-
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
86-
E2E_AUTH0_PASSWORD: ${{ secrets.E2E_AUTH0_PASSWORD }}
87-
E2E_AUTH0_USERNAME: ${{ secrets.E2E_AUTH0_USERNAME }}
88-
GRAPH_HOPPER_KEY: ${{ secrets.GRAPH_HOPPER_KEY }}
89-
GTFS_DATABASE_PASSWORD: ${{ secrets.GTFS_DATABASE_PASSWORD }}
90-
GTFS_DATABASE_URL: ${{ secrets.GTFS_DATABASE_URL }}
91-
GTFS_DATABASE_USER: ${{ secrets.GTFS_DATABASE_USER }}
92-
LOGS_S3_BUCKET: ${{ secrets.LOGS_S3_BUCKET }}
93-
MAPBOX_ACCESS_TOKEN: ${{ secrets.MAPBOX_ACCESS_TOKEN }}
94-
MONGO_DB_NAME: ${{ secrets.MONGO_DB_NAME }}
95-
MS_TEAMS_WEBHOOK_URL: ${{ secrets.MS_TEAMS_WEBHOOK_URL }}
96-
OSM_VEX: ${{ secrets.OSM_VEX }}
97-
RUN_E2E: "true"
98-
S3_BUCKET: ${{ secrets.S3_BUCKET }}
99-
SPARKPOST_EMAIL: ${{ secrets.SPARKPOST_EMAIL }}
100-
SPARKPOST_KEY: ${{ secrets.SPARKPOST_KEY }}
101-
TRANSITFEEDS_KEY: ${{ secrets.TRANSITFEEDS_KEY }}
77+
if: inputs.e2e
78+
run: |
79+
(echo "E2E Test Attempt 1" && docker compose --env-file __tests__/e2e/.env -f __tests__/e2e/docker-compose.yml up --abort-on-container-exit) || \
80+
(echo "E2E Test Attempt 2" && docker compose --env-file __tests__/e2e/.env -f __tests__/e2e/docker-compose.yml up --abort-on-container-exit) || \
81+
(echo "E2E Test Attempt 3" && docker compose --env-file __tests__/e2e/.env -f __tests__/e2e/docker-compose.yml up --abort-on-container-exit) || \
82+
(echo "E2E Tests Failed" && exit 1)
10283
# At this point, the build is successful.
10384
- name: Semantic Release
10485
env:

‎.github/workflows/pull-req.yml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
name: PR E2E Checks
2+
3+
on:
4+
pull_request:
5+
types: [opened, synchronize, reopened, ready_for_review]
6+
jobs:
7+
pr-e2e-check:
8+
# Run e2e tests:
9+
# - on pull request changed from draft to ready-for-review
10+
# - on commits to:
11+
# * PRs to master
12+
# * PRs to dev that are not draft
13+
# * PRs to dev on branches created by dependabot
14+
if: "${{
15+
github.event_name == 'pull_request' && (
16+
github.event.action == 'review_requested' ||
17+
github.base_ref == 'master' ||
18+
(github.base_ref == 'dev' && (
19+
github.event.pull_request.draft == false ||
20+
startsWith(github.head_ref, 'dependabot/')
21+
))
22+
)
23+
}}"
24+
uses: ./.github/workflows/node-ci.yml
25+
with:
26+
e2e: true
27+
secrets: inherit

‎.github/workflows/push.yml‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
name: Commit Checks
2+
3+
on: [push]
4+
5+
jobs:
6+
push-checks:
7+
uses: ./.github/workflows/node-ci.yml
8+
with:
9+
# Run e2e tests on pushes to dev and master.
10+
e2e: ${{ github.ref_name == 'dev' || github.ref_name == 'master' }}
11+
secrets: inherit

‎.gitignore‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,9 @@ tmp/
88
.tags
99
*.pid
1010

11+
# Ignore mobility data jar for building rules
12+
*.jar
13+
1114
# Optional npm cache directory
1215
.npm
1316

@@ -19,6 +22,7 @@ configurations/*
1922
!configurations/default
2023
!configurations/test
2124
!configurations/end-to-end
25+
!configurations/mtc-docs
2226
dist
2327
assets
2428

@@ -30,7 +34,9 @@ env.yml
3034
env.yml-original
3135
.env
3236
!configurations/test/env.yml
37+
!docker/server/env.yml
3338
scripts/*client.json
39+
*.pem
3440

3541
# Vs code settings
3642
.vscode/

0 commit comments

Comments
 (0)