Mercurial > p > roundup > code
comparison .github/workflows/ci-test.yml @ 7488:76d4c3d43258
Speed up postgresql by disabiling durability settings.
Read:
https://pythonspeed.com/articles/faster-db-tests/
I'm not using docker for postgresql at this time, but this article
describes how to start a docker postgres image with disabled settings
that may be of use if I end up having to change python 2.7 testing to
use dockerized containers against dockerized database backends.
However as a result of the above article, I took another look at:
https://www.postgresql.org/docs/current/non-durability.html
and switched off a few more settings.
Also added some similar settings for mysql that was not in the
deployed workflow.
[skip travis]
| author | John Rouillard <rouilj@ieee.org> |
|---|---|
| date | Thu, 15 Jun 2023 14:30:35 -0400 |
| parents | bdb23fef01ea |
| children | f0ff7f790c5c |
comparison
equal
deleted
inserted
replaced
| 7487:bdb23fef01ea | 7488:76d4c3d43258 |
|---|---|
| 120 sudo sed -i -e '/^\[mysqld\]/,/^\[mysql/s/^#* *max_allowed_packet.*/max_allowed_packet = 500M/' /etc/mysql/mysql.conf.d/mysqld.cnf; sleep 3 | 120 sudo sed -i -e '/^\[mysqld\]/,/^\[mysql/s/^#* *max_allowed_packet.*/max_allowed_packet = 500M/' /etc/mysql/mysql.conf.d/mysqld.cnf; sleep 3 |
| 121 #tail -n +0 /etc/mysql/my.cnf /etc/mysql/mysql.conf.d/mysqld.cnf | 121 #tail -n +0 /etc/mysql/my.cnf /etc/mysql/mysql.conf.d/mysqld.cnf |
| 122 #grep max_allowed /etc/mysql/mysql.conf.d/mysqld.cnf | 122 #grep max_allowed /etc/mysql/mysql.conf.d/mysqld.cnf |
| 123 #ls /etc/mysql/conf.d/ /etc/mysql/mysql.conf.d/ | 123 #ls /etc/mysql/conf.d/ /etc/mysql/mysql.conf.d/ |
| 124 #sleep 5 | 124 #sleep 5 |
| 125 # try to improve performance speed by disabling some ACID | |
| 126 # settings and change some layout defaults. | |
| 127 | |
| 128 echo "innodb_flush_log_at_trx_commit = 2" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 129 echo "innodb_file_per_table = OFF" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 130 echo "innodb_doublewrite=OFF" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 131 echo "innodb_fast_shutdown=2" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 132 echo "innodb_log_file_size=1048576" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 133 echo "innodb_flush_method=O_DIRECT" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 134 echo "innodb_log_buffer_size=3M" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 135 echo "innodb_buffer_pool_size=180M" >> /etc/mysql/mysql.conf.d/mysqld.cnf | |
| 136 sleep 3 | |
| 125 sudo service mysql restart | 137 sudo service mysql restart |
| 126 #sleep 10 | 138 #sleep 10 |
| 127 #ps -ef | grep mysqld | 139 #ps -ef | grep mysqld |
| 128 #sudo netstat -anp | grep mysqld | 140 #sudo netstat -anp | grep mysqld |
| 129 sudo mysql -u root -proot -e 'CREATE USER "rounduptest"@"localhost" IDENTIFIED WITH mysql_native_password BY "rounduptest"; GRANT ALL on rounduptest.* TO "rounduptest"@"localhost";' | 141 sudo mysql -u root -proot -e 'CREATE USER "rounduptest"@"localhost" IDENTIFIED WITH mysql_native_password BY "rounduptest"; GRANT ALL on rounduptest.* TO "rounduptest"@"localhost";' |
| 130 | 142 |
| 131 - name: Install postgres | 143 - name: Install postgres |
| 132 run: | | 144 run: | |
| 133 sudo apt-get update && sudo apt-get install postgresql | 145 sudo apt-get update && sudo apt-get install postgresql |
| 134 # Disable fsync for speed, don't care about data durability | 146 # Disable fsync, full page writes for speed, |
| 135 # when testing | 147 # don't care about data durability when testing |
| 136 sudo sed -i -e '$a\fsync = off' /etc/postgresql/*/*/postgresql.conf | 148 sudo sed -i -e '$a\fsync = off' /etc/postgresql/*/*/postgresql.conf |
| 149 sudo sed -i -e '$a\full_page_writes = off' /etc/postgresql/*/*/postgresql.conf | |
| 150 sudo sed -i -e '$a\synchronous_commit = off' /etc/postgresql/*/*/postgresql.conf | |
| 137 sudo service postgresql restart; sleep 10 | 151 sudo service postgresql restart; sleep 10 |
| 138 # set up postgresql database | 152 # set up postgresql database |
| 139 sudo -u postgres psql -c "CREATE ROLE rounduptest WITH CREATEDB LOGIN PASSWORD 'rounduptest';" -U postgres | 153 sudo -u postgres psql -c "CREATE ROLE rounduptest WITH CREATEDB LOGIN PASSWORD 'rounduptest';" -U postgres |
| 140 | 154 |
| 141 - name: install redis | 155 - name: install redis |