1The MySQL Benchmarks 2 3These tests require a MySQL version of at least 3.20.28 or 3.21.10. 4 5Currently the following servers are supported: 6MySQL 3.20 and 3.21, PostgreSQL 6.#, mSQL 2.# and Solid Server 2.2 7 8The benchmark directory contains the query files and raw data files used to 9populate the MySQL benchmark tables. In order to run the benchmarks, you 10should normally execute a command such as the following: 11 12run-all-tests --server=mysql --cmp=mysql,pg,solid --user=test --password=test --log 13 14This means that you want to run the benchmarks with MySQL. The 15limits should be taken from all of MySQL, PostgreSQL, and Solid. 16The login name and password for connecting to the server both are 17``test''. The result should be saved as a RUN file in the output 18directory. 19 20When run-all-tests has finished, will have the individual results and the 21the total RUN- file in the output directory. 22 23If you want to look at some old results, use the compare-results script. 24For example: 25 26compare-results --dir=Results --cmp=mysql,pg,solid 27compare-results --dir=Results --cmp=mysql,pg,solid --relative 28 29compare-results --dir=Results --cmp=msql,mysql,pg,solid 30compare-results --dir=Results --cmp=msql,mysql,pg,solid --relative 31 32compare-results --dir=Results --server=mysql --same-server --cmp=mysql,pg,solid 33 34Some of the files in the benchmark directory are: 35 36File Description 37 38Data/ATIS Contains data for 29 related tables used in the ATIS tests. 39Data/Wisconsin Contains data for the Wisconsin benchmark. 40Results Contains old benchmark results. 41Makefile.am Automake Makefile 42README This file. 43test-ATIS.sh Creation of 29 tables and a lot of selects on them. 44test-connect.sh Test how fast a connection to the server is. 45test-create.sh Test how fast a table is created. 46test-insert.sh Test create and fill of a table. 47test-wisconsin.sh A port of the PostgreSQL version of this benchmark. 48run-all-tests Use this to run all tests. When all tests are run, 49 use the --log and --use-old options to get a RUN-file. 50compare-results Generates a comparison table from different RUN files. 51server-cfg Contains the limits and functions for all supported 52 SQL servers. If you want to add a new server, this 53 should be the only file that neads to be changed. 54 55 56Most of the tests should use portable SQL to make it possible to 57compare different databases. Sometimes SQL extensions can make things 58a lot faster. In this case the test may use the extensions if the --fast 59option is used. 60 61Useful options to all test-scripts (and run-all-tests): 62 63--host=# Hostname for MySQL server (default: localhost) 64--db=# Database to use (default: test) 65--fast Allow use of any non-standard SQL extension to 66 get things done faster. 67--lock-tables Use table locking to get more speed. 68 69From a text at http://www.mgt.ncu.edu.tw/CSIM/Paper/sixth/11.html: 70 71The Wisconsin Benchmark 72 73The Wisconsin Benchmark described in [Bitton, DeWitt, and Turbyfill 741983] [Boral and DeWitt 1984] [Bitton and Turbyfill 1985] [Bitton and 75Turbyfill 1988], and [DeWitt 1993] is the first effort to 76systematically measure and compare the performance of relational 77database systems with database machines. The benchmark is a 78single-user and single-factor experiment using a synthetic database 79and a controlled workload. It measures the query optimization 80performance of database systems with 32 query types to exercise the 81components of the proposed systems. The query suites include 82selection, join, projection, aggregate, and simple update queries. 83 84The test database consists of four generic relations. The tenk 85relation is the key table and most used. Two data types of small 86integer numbers and character strings are utilized. Data values are 87uniformly distributed. The primary metric is the query elapsed 88time. The main criticisms of the benchmark include the nature of 89single-user workload, the simplistic database structure, and the 90unrealistic query tests. A number of efforts have been made to extend 91the benchmark to incorporate the multi-user test. However, they do 92not receive the same acceptance as the original Wisconsin benchmark 93except an extension work called the AS3AP benchmark. 94