{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Performance Testing for StackRox"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This document is an idea of executable documentation. It can be used with VS Code and `Jupyter` plugin. It's pretty handy because showing `Outline` allows faster document navigation."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-requirement\n",
    "\n",
    "There are few unconventional tools used in this notebook\n",
    "\n",
    "1. `rg` - ripgrep is replacement for `grep`. Installation instructions can be found here: https://github.com/BurntSushi/ripgrep#installation\n",
    "2. `yq` - is command line YAML processor. Installation instructions can be found here: https://github.com/mikefarah/yq#install"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quick start"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install `k6` tool. To install it, please check the instructions on the project page: https://k6.io/docs/get-started/installation. Below is an example for MacOS."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "brew install k6"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Install required NodeJS development libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "npm install"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Run test directly**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "export ROX_ADMIN_PASSWORD=$(cat ../../deploy/k8s/central-deploy/password)\n",
    "export HOST=\"https://localhost:8000\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "npm run test -- --quiet --out csv=k6-test-result.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that you can check `k6-test-result.csv` for results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Run test in docker or kubernetes**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build a docker image with the following command:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "docker build . -t stackrox/performance-test-runner:latest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a namespace and deploy `performance-test-runner`. The next command uses `yq` to set a password in the YAML file. To install it, please check the instructions on the project page: https://github.com/mikefarah/yq"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "export KUBECONFIG=./artifacts/kubeconfig\n",
    "\n",
    "export ROX_ADMIN_PASSWORD=$(cat ../../deploy/k8s/central-deploy/password)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "kubectl create namespace performance-test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "yq e '.spec.template.spec.containers[0].env[0].value = \"'\"${ROX_ADMIN_PASSWORD}\"'\"' ./deploy/test-runner-deployment.yaml | kubectl apply -f -"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check if everything is working, you can run the following command."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "kubectl get pods --namespace performance-test"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Collecting results\n",
    "\n",
    "Get logs because results will be output to stdout inside the container."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "export KUBECONFIG=./artifacts/kubeconfig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "kubectl logs --namespace performance-test test-runner -c test-runner > ./test-runner-result_2022-11-07_19-12.log"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extract the results of testing in a CSV file. The following line uses the `rg` command. To install it, please check the instructions on the project page: https://github.com/BurntSushi/ripgrep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "rg 'sac_user=|extra_tags' ./test-runner-result_2022-11-07_19-12.log > ./test-runner-result_2022-11-07_19-12.log.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that, this file can be imported into some spreadsheet for further processing of the results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Simplest option is to create Google Sheets.\n",
    "2. Copy the content of the `.csv` file and paste into the sheet. You will get a pop-up to split a column into multiple columns. By selecting that, multiple columns will be populated with test result data.\n",
    "3. From the menu, select `Insert -> Pivot table`. A dialog will automatically select the whole sheet range which we want. Select to create a pivot table on a new sheet.\n",
    "4. On the new sheet in the pivot table options, select the following:\n",
    "   - For rows: `url` and `extra_tags`. Disable `Show totals`.\n",
    "   - For columns: `metric_name`. Disable `Show totals`.\n",
    "   - For values: `metric_value`. You can select `MEDIAN` or `AVERAGE` function.\n",
    "   - For the filter, you can disable everything and enable only `http_req_waiting`. That metric represents how long a request waits for a server response. It does not include sending and receiving times.\n",
    "5. Have fun!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to create a new test group"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first need HAR export from requests generated with our browser to create a new test group. The best is to use Firefox because Firefox will export only filtered requests in the `networking` tab. After that, you can execute a conversion from the HAR file into a JavaScript file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "npm run har-to-k6 -- ~/Downloads/new-test-group.har --output ./groups/awesomeNewPage.js"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After that, some manual steps should be done:\n",
    "1. Remove comments above `import`.\n",
    "2. Remove the `options` constant.\n",
    "3. Replace function declaration to accept 3 arguments. i.e. `export function awesomeNewPage(host, headers, tags)`.\n",
    "4. Remove the `response` variable. Declaration and usage of it.\n",
    "5. Replace all objects with the `headers` property with the following object `{ headers, tags }`.\n",
    "6. Change URLs in to use the `host` variable. i.e. `'https://localhost:8001/api/graphql?opname=cvesCount'` should become `` `${host}/api/graphql?opname=cvesCount` ``.\n",
    "7. Remove `sleep` at the end of the function and also its `import`.\n",
    "\n",
    "The next step is to include the group in a test. i.e. to add a group into the `tests/testSacScopes.js` test, we should do the following.\n",
    "1. Import the group into a test file. i.e. `import { awesomeNewPage } from '../groups/awesomeNewPage.js';`\n",
    "2. Add function call in `runAllGroups`. i.e. `awesomeNewPage(__ENV.HOST, header, tags);`\n",
    "\n",
    "With that new group is created and added to the test. And after that, we can build a docker image and run it. You can take a look at **Quick start** for instructions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How to fetch logs from a GKE cluster"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set your `cluster_name` and correct `timestamp`(s) in the logs query below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "# Query - time is in UTC\n",
    "read -r -d '' GC_LOGS_QUERY <<- EOQ\n",
    "    resource.type=\"k8s_container\"\n",
    "    labels.k8s-pod/app=\"test-runner\"\n",
    "    resource.labels.project_id=\"srox-temp-dev-test\"\n",
    "    resource.labels.location=\"us-central1-a\"\n",
    "    resource.labels.cluster_name=\"mt-0711\"\n",
    "    resource.labels.namespace_name=\"performance-test\"\n",
    "    resource.labels.container_name=\"test-runner\"\n",
    "    timestamp > \"2022-11-07T20:00:00Z\"\n",
    "    timestamp < \"2022-11-07T21:00:00Z\"\n",
    "    severity>=DEFAULT\n",
    "EOQ\n",
    "\n",
    "gcloud logging read \"${GC_LOGS_QUERY}\" --format='csv(textPayload)' --order='asc' > test-runner-result_2022-11-07_21-00.log"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This file will contain quotes for every line and also extra new lines. That should be cleaned up.\n",
    "\n",
    "The next command is for MacOS `sed` command. It will remove new lines and quotes around each row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "cat test-runner-result_2022-11-07_21-00.log | sed -e ':a' -e 'N' -e '$!ba' -e 's/\"\\n\"//g' | sed 's/\"//g' | rg 'sac_user=|extra_tags' > test-runner-result_2022-11-07_21-00.log.csv"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Development"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To check linting and formatting you can run the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "npm run format:check\n",
    "npm run lint:check"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And to automatically fix them, you can run the following commands:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "npm run format:fix\n",
    "npm run lint:fix"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Bash",
   "language": "bash",
   "name": "bash"
  },
  "language_info": {
   "codemirror_mode": "shell",
   "file_extension": ".sh",
   "mimetype": "text/x-sh",
   "name": "bash"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "949777d72b0d2535278d3dc13498b2535136f6dfe0678499012e853ee9abcab1"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}