/* Copyright (c) 2015, 2021, Oracle and/or its affiliates.

   This program is free software; you can redistribute it and/or modify
   it under the terms of the GNU General Public License, version 2.0,
   as published by the Free Software Foundation.

   This program is also distributed with certain software (including
   but not limited to OpenSSL) that is licensed under separate terms,
   as designated in a particular file or component or in included license
   documentation.  The authors of MySQL hereby grant you an additional
   permission to link the program and your derivative works with the
   separately licensed software that they have included with MySQL.

   This program is distributed in the hope that it will be useful,
   but WITHOUT ANY WARRANTY; without even the implied warranty of
   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
   GNU General Public License, version 2.0, for more details.

   You should have received a copy of the GNU General Public License
   along with this program; if not, write to the Free Software
   Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA */

Testing plugin services
-----------------------

Plugin services are usually APIs provided by the mysqld server. Theses APIs can be tested with
unit test frameworks like "googletest" by developers. In QA it is preferable to test plugin
services closer to reality with the help of plugins due to following reasons:

  - Integrating plugin, plugin services, clients and SQL layer for testing interaction between all.
  - Having examples for users close to reality (as described in user manual).
  - Using our (qa) test tools.

Implementing a test plugin
--------------------------

Depending on the specification of a plugin service, the test plugin must have a special plugin type
or even tested with all plugin types.

For the frame work to implement test plugins, the plugin type "daemon" has be chosen. It will be
loaded (started) with the sql statement "INSTALL PLUGIN" and unloaded (stopped) by calling
"UNINSTALL PLUGIN". There is no other communication between server and daemon unless it will be
explicitly implemented, e.g. with the help of an API or service. INSTALL and UNINSTALL will be
executed by a mtr test. The mtr test may of course contain other sql statements to test the
interaction between plugin (service) and SQL layer.

The plugin shall write its test results into one or more files. These files can then be written
into the result file of the mtr test with the normal mtr commands like "cat_file".

With the help of the log_message service the test plugin can easily write messages into
"mysqld.1.err". These messages can be made visible in the result file or checked as follows:
Load this log into a table with 2 columns (date, info). The column "date" is ignored as not
deterministic. The column "info" will be selected by removing "\r" to be compatible to windows
(The err log file on windows contains CR,LF as newline). Select the messages so they will be
visible in the mtr result file.

A perl snippet can be used for that purpose, too.

In a plugin status and system (sql) variables can be specified. With status variables the
state of the test execution can be reported to the mtr test calling the test plugin. That might
be useful to e.g. give a ready message for a test case or the whole plugin. Especially when
using threads in a test plugin such status variables are useful to synchronize execution of
plugin and mtr test as both are running in parallel. System variables can be used to e.g. control
test execution. But, that is very rudimentary as can be used to configure the plugin as its start.

Test plugins
------------

There exist test plugins of type "daemon", which can be taken as examples or frameworks and
calling mtr tests:

  - "plugin/test_services" for the test plugin written in C++,
  - "mysql-test/suite/test_services" for the corresponding mtr tests.

Test plugins:

  -  "test_services.cc"

     showing how to implement a test plugin for simple plugin services like
     "my_snprintf" and "my_plugin_log_message". For each of the services there is a function
     implemented containing the test cases. Instead of using the standard I/O of C++ it is much
     more comfortable to used "my_open", "my_write", etc. as it covers all platforms. The output
     of "my_snprintf" can be found in "var/mysqld.1/data/test_services.log".
     "my_plugin_log_message" writes into "mysqld.1.err". The (selected) contents of these files
     can be seen in the result file of "test_services.test". "test_services.test" didn't create
     a thread. The effect is that the test (mysqld) is waiting until the plugin completes its work.
     A waiting condition to synchronize execution of plugin and mtr test is not necessary.

  -  "test_service_threaded.cc"

     runs the same tests as "test_services.cc", but in a thread. This needs synchronization
     of plugin and mtr test as both are running in parallel. The synchronization will be done
     by waiting for the plugin (exactly the thread in the plugin) to be finished. If the
     thread is needing a very long time then the mtr test may run in a timeout.

  - "test_framework.cc"

     is a rudimentary implementation of a daemon plugin, which can be take to create a new one.
     Of course copying and modifying/extending one of the other plugins to create a new
     test plugin can be recommended depending on the need of threads.

Using threads has the advantage of limiting test execution with a timeout. That may be interesting
for long running plugins, testing complex APIs. Plugins needing a short run time may run without threads.

    To create the plugins and run the test do the following:
        - modify "CmakeLists.txt" if you want to add a new test plugin,
        - rebuild ALL or only make test_services by running make in the test_services directory,
        - in mysql-test: "./mtr --suite=test_services test_services" or
          "./mtr --suite=test_services test_services_threaded"

    If a test case in plugin "test_services" shall be switched off run the test as follows:

        ./mtr --mysqld=--loose-test-services-with-log-message=0 t/test_services.test

That shows a simple way of control test execution with system variables.

Steps to create a new test plugin
---------------------------------

The following is an example of steps to create a test plugin for a plugin service relying on
the daemon type:

  - Create test_<plugin name>.cc file under mysql/plugin/test_plugins deciding if threading
    is needed or not.
      - Put the test cases for plugin service to be tested in the init function (preferred) and
        the clean up in the deinit function.
      - Add new plugin to plugin.def to make it available to mtr.
  - Create one or more MTR tests.
      - MTR test loads plugin and execute the test cases.
      - After uninstall of the plugin put the test results into the result file of the mtr test.
      - Create a reference result file.