Minikube Dockerized Selenium Grid

Minikube is an interactive Kubernetes system orchestrating Docker containers that can be used locally for testing purpose

  • Install minikube (this config is for MAC); and for other platforms, follow this
brew cask install minikube

# check version
minikube version

or

curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.30.0/minikube-darwin-amd64 && chmod +x minikube && sudo cp minikube /usr/local/bin/ && rm minikube
  • Let’s config Kubernetes system in an interactive mode; what we need first is to start minikube
minikube start

# check status
minikube status
  • Now, try to launch the minikube dashboard; launching minikube dashboard will take you to the default browser with the interface as seen below,
minikube dashboard

# to get the url alone
minikube dashboard --url=true

  • Now, click on the create link text on top-corner of the dashboard page

  • Import the below kubernetes_selenium_grid.json file created by me that generates a Selenium hub service, 1 Hub and, 1 Chrome Node; you can actually increase the Nodes manually from the dashboard itself


{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "selenium-hub",
"labels": {
"run": "selenium-hub"
}
},
"spec": {
"ports": [
{
"protocol": "TCP",
"port": 4444,
"targetPort": 4444
}
],
"selector": {
"run": "selenium-hub"
},
"type": "NodePort",
"externalTrafficPolicy": "Cluster"
}
}
{
"kind": "Deployment",
"apiVersion": "apps/v1",
"metadata": {
"name": "selenium-hub",
"labels": {
"run": "selenium-hub"
}
},
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"run": "selenium-hub"
}
},
"template": {
"metadata": {
"labels": {
"run": "selenium-hub"
}
},
"spec": {
"containers": [
{
"name": "selenium-hub",
"image": "selenium/hub:3.14.0",
"ports": [
{
"containerPort": 4444,
"protocol": "TCP"
}
]
}
]
}
}
}
}
{
"kind": "Deployment",
"apiVersion": "apps/v1",
"metadata": {
"name": "selenium-node-chrome",
"labels": {
"run": "selenium-node-chrome"
}
},
"spec": {
"replicas": 1,
"selector": {
"matchLabels": {
"run": "selenium-node-chrome"
}
},
"template": {
"metadata": {
"labels": {
"run": "selenium-node-chrome"
}
},
"spec": {
"containers": [
{
"name": "selenium-node-chrome",
"image": "selenium/node-chrome-debug:3.14.0",
"env": [
{
"name": "HUB_PORT_4444_TCP_ADDR",
"value": "selenium-hub"
},
{
"name": "HUB_PORT_4444_TCP_PORT",
"value": "4444"
}
]
}
]
}
}
}
}

  • Observe the Service created, which is up and running
kubectl get service

  • Observe the created Selenium Hub and Chrome node, which is up and running
# to check all the deployments
kubectl get deployments
(or)
kubectl get deployments --namespace default

# to check all the pods (containers)
kubectl get pods
(or)
kubectl get pods --namespace default

  • Get the newly generated selenium Host address and Port number; here, selenium-hub is the custom service created by me
minikube service selenium-hub --url
(or)
minikube service --namespace default selenium-hub --url

  • Now, configure the driver initialization
options = {
  'chromeOptions' => {
    'args' => ['disable-infobars', 'disable-gpu', 'privileged', 'ignore-certificate-errors']
  }
}

caps = Selenium::WebDriver::Remote::Capabilities.chrome(options)
@driver = Selenium::WebDriver.for :remote, :url => "http://192.168.99.100:30445/wd/hub", desired_capabilities: caps
  • Run your tests, which will pick-up the chrome node that we actually created through Kubernetes

Zalenium Dockerized Selenium Grid

Zalenium provides docker images (Hub + Nodes) with the latest browser drivers, browsers, and tools (for any language bindings) required for Selenium automation. The containers created from these images can be scaled-up with simple CLI commands. By default, there will be two live containers up and running.

  • Open terminal and download the below images from Docker hub
docker pull elgalu/selenium
docker pull dosel/zalenium
  • Create docker containers and start Zalenium proxy in a single command
docker run --rm -ti --name zalenium -p 4444:4444 -v /var/run/docker.sock:/var/run/docker.sock -v /tmp/videos:/home/seluser/videos --privileged dosel/zalenium start
  • Now, configure the driver initialization
options = {
  'chromeOptions' => {
    'args' => ['disable-infobars', 'disable-gpu', 'privileged', 'ignore-certificate-errors']
  }
}

caps = Selenium::WebDriver::Remote::Capabilities.chrome(options)
@driver = Selenium::WebDriver.for :remote, :url => "http://0.0.0.0:4444/wd/hub", desired_capabilities: caps
  • Check out live test session using the below URL
http://localhost:4444/grid/admin/live

  • Start monitoring the Zalenium live interface once the tests were triggered

  • You can also review tests through the recorded video stream
http://localhost:4444/dashboard/

  • To stop the docker containers spun through Zalenium, simply exit Zalenium proxy using the below command
docker stop zalenium

 

Method #2 (most preferred approach)

Using this approach actually detaches the Zalenium process from running in the terminal

  • A single line command to start Zalenium proxy and make docker containers up
curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s start
  • Command to terminate the Zalenium proxy along with the running/exited containers
curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s stop
  • Scale up docker containers as desired; but, make sure you create replicas only if the machine has proper CPU (x4/x8/x16) cores specific to the containers
curl -sSL https://raw.githubusercontent.com/dosel/t/i/p | bash -s start --desiredContainers 8 --screenWidth 1440 --screenHeight 900 --timeZone "Asia/Dubai" --maxDockerSeleniumContainers 10 --maxTestSessions 30 --videoRecordingEnabled true

Docker Selenium Grid for beginners w/o docker-machine

  • Download and install docker (Mac); use a similar way to install Docker in other platforms
brew cask install docker
  • Create a docker-compose.yml file in the project root folder; make sure the below contents to be added in the already created docker-compose.yml file (you will see one Selenium hub and one Chrome node)
hub:
 image: selenium/hub:3.14.0
 container_name: hub
 ports:
 - "4444:4444"
node-chrome:
 image: selenium/node-chrome-debug:3.14.0
 links:
 - hub
  • Run docker compose to create containers with hub and with multiple nodes for the tests to run
docker-compose up -d --scale node-chrome=7
  • Check out the running browser instances in the browser console through the exposed port 4444
http://localhost:4444/grid/console

  • Configure the driver initialization
options = {
  'chromeOptions' => {
    'args' => ['disable-infobars', 'disable-gpu', 'privileged', 'ignore-certificate-errors'],
    'mobileEmulation' => {}
  }
}

caps = Selenium::WebDriver::Remote::Capabilities.chrome(options)
@driver = Selenium::WebDriver.for :remote, :url => "http://0.0.0.0:4444/wd/hub", desired_capabilities: caps
  • Now, run the selenium tests
cucumber features/scenario/**/*
  • Terminate the running Docker containers
docker-compose down

Scale up docker images for Selenium Parallel Tests

This post helps you scale up docker images to do parallel selenium tests. And, if you’re a Docker newbie, keep this as a hierarchy in your mind before getting into below concepts – Docker Machine > Docker Image > Docker containers

  • Download and install Docker
for MAC
# download complete docker community edition
brew cask install docker-toolbox

Optional (not important)
# download docker-machine 
curl -L https://github.com/docker/machine/releases/download/v0.15.0/docker-machine-$(uname -s)-$(uname -m) >/usr/local/bin/docker-machine &
# convert docker-machine file as executable 
chmod +x /usr/local/bin/docker-machine

For Linux and Windows,
refer: https://github.com/docker/machine/releases

# download docker-compose 
curl -L https://github.com/docker/compose/releases/download/1.23.0-rc2/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
# convert docker-compose file as executable 
chmod +x /usr/local/bin/docker-compose

For Linux and Windows,
refer: https://github.com/docker/compose/releases
  • Download and install VirtualBox
MAC
brew cask install virtualbox
  • Create a Docker machine
# to create docker machine
docker-machine create tests

# list down all the available docker machines
docker-machine ls

# to delete docker machine
docker-machine rm tests
  • Kickstart the Docker machine
docker-machine start tests
  • Set environment variables to command docker towards a specific docker machine; say, tests
eval $(docker-machine env tests)
  • Create docker-compose.yml in the project’s root folder and add all the possible images required for the tests. There are different variants to create images and containers; say, docker run -d -p 4444:4444 --net grid --name selenium-hub selenium/hub​; however, creating them through docker-compose.yml is much appreciated for real-time projects
version: "3"
services:
  selenium-hub:
    image: selenium/hub
    container_name: selenium-hub
    ports:
      - "4444:4444"
  chrome:
    image: selenium/node-chrome
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
  firefox:
    image: selenium/node-firefox
    depends_on:
      - selenium-hub
    environment:
      - HUB_HOST=selenium-hub
      - HUB_PORT=4444
  • Run docker compose to create containers with hub and nodes for the tests to run
docker-compose up

# scale up multiple containers
docker-compose up -d --scale chrome=2
(say, I need 2 containers with chrome instances)
  • Get IP address of the docker machine and use it as a Selenium remote hub url
docker-machine ip tests

# get machine ip with details
env | grep DOCKER

# get container ip 
docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' selenium-hub
  • Copy & paste docker machine IP as below; here, xxx.xxx.xxx.xxx is the docker-machine IP (say, 192.168.99.100)
@driver = Selenium::WebDriver.for :remote, url: "http://xxx.xxx.xxx.xxx:4444/wd/hub"
  • As a teardown case, stop/delete/manipulate the containers & images if needed
# list complete details about a specific container
docker logs --details CONTAINER_ID
(e.g., docker logs --details 39d9d7475bf1)
docker logs CONTAINER_NAME
(e.g., docker logs selenium-hub)

# delete a container of an image
docker container ls -a
docker ps -a
(here, docker ps is the alias for above cmd to list containers)
docker container rm CONTAINER_ID
(e.g., docker container rm 39d9d7475bf1)

# stop/restart a container of an image
docker container stop CONTAINER_ID
docker container restart CONTAINER_ID

# start all the previously stopped/exited containers
docker start $(docker ps -a -q --filter "status=exited")

# delete an image of a docker machine
docker image ls -a
docker image rm IMAGE_NAME
(e.g., docker image rm nginx)

# log into a docker container
docker exec -it CONTAINER_NAME/ID bash
  • Once the docker-compose up is triggered, the selenium grid initiates hub and nodes as shown below

  • Now, run your parallel tests through command line; say, ruby parallel_tests
parallel_tests feature/scenarios/**/*.feature

Handle Basic authentication in Selenium for Chrome browser

Chrome extension

Create a simple chrome extension with custom username and password defined in it.

  • Create a .js file, background.js with the custom username and password
var username = "your_user";
var password = "your_pass";
var retry = 3;

chrome.webRequest.onAuthRequired.addListener(
  function handler(details) {
    if (--retry < 0)
      return {cancel: true};
    return {authCredentials: {username: username, password: password}};
  },
  {urls: ["<all_urls>"]},
  ['blocking']
);

[Text in red has to be customized specific to the project]

  • Create a .json file, manifest.json as shown below
{
  "manifest_version": 2,
  "name": "Authentication for ...",
  "version": "1.0.0",
  "permissions": ["<all_urls>", "webRequest", "webRequestBlocking"],
  "background": {
    "scripts": ["background.js"]
  }
}
  • Now, zip both the files and rename it to extension.crx

  • To check if the custom extension works fine, open a new tab in Google-Chrome and enter chrome://extensions/
  • Now, select the folder where both the manifest.json and background.js files were located (on clicking Load unpacked extension…)

  • Once the extension is loaded, you will observe the newly added icon next to the browser’s address bar; now, try loading the web application that asks you for the basic auth

Chrome options

Configure the newly created .crx chrome extension through ChromeOptions as described below

  • Ruby snippet to configure chrome extension through ChromeOptions
options = {
  'chromeOptions' => {
    'extensions' => [
      Base64.strict_encode64(File.open("../extension.crx", 'rb').read)
    ]
  }
}

caps = Selenium::WebDriver::Remote::Capabilities.chrome(options)
@driver = Selenium::WebDriver.for :chrome, desired_capabilities: caps
  • The above snippet works fine in GUI mode; however, through browser’s headless mode, it doesn’t
  • Linux users, try xvfb to overcome this issue; for Ruby, you need a third party client/lib, headless to couple with Linux’s xvfb mode
gem 'headless'
require 'headless'
  • Start headless support before initiating the tests
@headless = Headless.new
@headless.start
  • And terminate the headless mode at exit
@headless.destroy
  • In Jenkins, you need to set the environment ready for xvfb to run tests without GUI mode
sudo yum install xorg-x11-server-Xvfb -y
(or)
sudo apt-get install xorg-x11-server-Xvfb -y
export DISPLAY=:0
  • Meanwhile, go to Manage Jenkins > Configure System; add manual export for DISPLAY :0 in the global properties itself (this step is optional)

  • Force kill Xvfb if previously running
pkill -f Xvfb || true
  • To set headless browser window size, you can still use selenium-webdriver’s native way
target_size = Selenium::WebDriver::Dimension.new(1600, 1268)
@driver.manage.window.size = target_size

 

Selenium IDE

Install Selenium IDE in Chrome Developer mode

  • Download the latest Selenium-IDE release from here.
  • Extract the zipped file
  • Now, open the Chrome browser and go to Chrome extensions
chrome://extensions/
  • Click on the tab  LOAD UNPACKED, locate the above-unzipped folder and upload

  • Now, you will find the Selenium-IDE icon visible on the address bar

  • Click on the Selenium IDE icon, play and record application under test, and save your recorded test case; by default, the Selenium IDE tests will be saved with extension (.side)

 

Install Selenium IDE Chrome add-on through direct URL [Optional]

  • Go to the link
  • Click on ADD TO CHROME button
  • Now, you will find the Selenium-IDE icon visible on the address bar
  • Click on the Selenium IDE icon, play and record application under test, and save your recorded test case; by default, the Selenium IDE tests will be saved with extension (.side)

 

Run Selenium IDE tests from Terminal

The exported test with .side extension is more or less JSON formatted. However, the javascript-based library, selenium-side-runner is used to execute the exported .side tests because these tests are generally compiled to JS for execution.

  • Install node package manager (npm) for JavaScript and check if it’s installed
npm -v
  • Install the selenium-side-runner js library
sudo npm install -g selenium-side-runner
selenium-side-runner --version
  • Now, record and save a Selenium IDE Test case in a local directory; say, demo.side

  • Open the terminal and navigate to the directory where the test case was saved, and run the below cmd
selenium-side-runner
  • The above can also be achieved with optional CLI defining browser and platform name
selenium-side-runner -c "browserName=chrome platform=MAC"

 

Run Selenium IDE tests through a platform-specific executor

  • Download the updated platform specific Selenium-IDE test executor from here. say, selenium-side-runner-macos for MAC, selenium-side-runner-linux for Linux and selenium-side-runner-win.exe for Windows
  • Extract the zipped file
  • Now, open the Terminal and provide executor access to the extracted file
chmod +x selenium-side-runner-macos
  • Execute tests by the runner
./selenium-side-runner-macos *.side
./selenium-side-runner-macos *.*
  • Execute tests by the runner in debug mode
./selenium-side-runner-macos --debug demo.side

Headless Chrome and Firefox in Selenium ruby tests

Headless Chrome

  • Modify chrome browser property to run tests in headless mode; say,
options = Selenium::WebDriver::Chrome::Options.new(args: ['headless'])
@driver = Selenium::WebDriver.for :chrome, options: options

or

options = Selenium::WebDriver::Chrome::Options.new
options.add_argument('--headless')
@driver = Selenium::WebDriver.for :chrome, options: options
  • It is recommended to maximize browser while running headless tests
target_size = Selenium::WebDriver::Dimension.new(1600, 1268)
@driver.manage.window.size = target_size

Headless Firefox

  • Modify firefox browser property to run tests in headless mode; say,
options = Selenium::WebDriver::Firefox::Options.new(args: ['-headless'])
@driver = Selenium::WebDriver.for :firefox, options: options

 

Debug Android Device’s Chrome browser on Desktop

Enable USB Debugging

  • Go to Android Settings -> About phone
  • And tap the Build number 7 times till it become developer mode (say, MIUI version for MI devices)
  • Now, go back to Additional settings -> Developer options -> and enable USB debugging

 

Connect Remote Device

  • Open Chrome browser -> Chrome Developer Tools
  • Select More Tools -> Remote devices

  • Observe the connected device on left pane

  • Now, open the Chrome browser and navigate to target URL

  • And click on the Inspect button to open Developer tool

 

Debug IOS Device’s Safari browser on Mac

1| Install IOS WebKit debug proxy

brew install ios-webkit-debug-proxy

2| Execute the terminal cmd

sudo ios_webkit_debug_proxy -f chrome-devtools://devtools/bundled/inspector.html

3| If the above cmd fails to work, run the below and execute the above cmd

brew update
brew reinstall --HEAD libimobiledevice
brew reinstall -s ios-webkit-debug-proxy
sudo ios_webkit_debug_proxy -f chrome-devtools://devtools/bundled/inspector.html

4|Open Safari browser in the device, navigate to the target URL and observe if the server is up

http://localhost:9222/

4|Now, open the below link in your desktop Chrome browser

chrome-devtools://devtools/bundled/inspector.html?ws=localhost:9222/devtools/page/1