See www.zabbix.com for the official Zabbix site.

Docs/specs/ZBXNEXT-1322

From Zabbix.org
Jump to: navigation, search

Automatic upgrade of database structure

ZBXNEXT-1322

Status: v1.4

Owner: Alexei Vladishev

Summary

Currently in order to upgrade from one major version to another manual execution of database patches is required. It will be improved so that Zabbix itself would update the database automatically.

Specification

On startup Zabbix server/proxy will verify if its database version matches current Zabbix database version stored in the database. If Zabbix server/proxy is newer, it will execute patches before doing anything else.

Details

General
  • This functionality will be supported by both Zabbix server and proxy.
  • Patches will be stored in the binaries for at least 5 major releases.
  • It will support situation when a new patch is added for several releases, for example, both 2.2 and 2.4. Such a patch must be executed only once doing upgrade from 2.0 to 2.4.
  • Zabbix server and proxy will start upgrade procedure unconditionally, i.e. there will be no flags or command line parameters controlling this process.
  • There is no way to detect partitioned setups. Upgrade will start normally. It will be documented.
  • Standalone SQL patches will not be provided anymore.
  • Procedure for adding new DB patches must be added to developer documentation.
  • Database patches are developed together with the functionality that requires them.
Verification
  • It will support upgrade only from 2.0 to newer releases.
  • It will verify that database schema matches 2.0 when upgrading from 2.0.x. It will be verified by :
    • lack of table dbversion
    • existence of config.server_check_interval, which is the last field added to the 2.0 schema
  • No further verification will be performed when upgrading from 2.0. Upgrading from 2.2 and later will only take into account information in the dbversion table.
  • Both frontend and server (proxy) binaries/PHP files will contain required DB version.
  • If frontend does not match dbversion.mandatory or dbversion table is missing, it will display red message in the same style as DB down message: 'The frontend does not match Zabbix database. Current database version (mandatory/optional): %d/%d. Required mandatory version: %d. Contact your system administrator.' and will retry as in case of the DB down message.
  • If server/proxy does not match dbversion.mandatory, it will log a message 'The server does not match Zabbix database. Current database version (mandatory/optional): %d/%d. Required mandatory version: %d.' and shutdown. In proxy message server is replaced with proxy.
Logging
  • On startup Zabbix server/proxy will report current (mandatory and optional) and required DB versions.
  • If one of mandatory or optional patches fails, Zabbix server/proxy will stop and provide detailed information in the log file at LOG_LEVEL_CRIT.
  • It will report start of DB upgrade procedure and also progress in percentage to log file with LOG_LEVEL_WARNING. Percentage will be calculated as number of executed patches divided by total of number of patches. Only 1% or higher increase is reported. Messages: "completed %d%% of database upgrade ".
  • After a failure, next run will start reporting at 0% again.
  • Zabbix server/proxy must log start and end of the upgrade procedure. Messages: "starting automatic database upgrade" and "database upgrade fully completed".
Low level
  • New database fields dbversion.mandatory and dbversion.optional are added (see Database changes below).
  • After every successfully executed patch dbversion.optional is updated. Mandatory patches update also dbversion.mandatory.
  • It is OK to use C for complex patches if needed.
  • Each individual patch will be executed as a separate transaction.
  • Some minor patches (missing indexes, fixes for data types, default values, etc) could be marked 'optional' to make it work with older binaries. Therefore even if an optional patch was introduced and applied by 2.2.5, older binaries from 2.2.x will still work perfectly.

Database changes

  • Table dbversion will be created with fields:
    • 'mandatory' having type integer. It will contain the version of the last successfully executed mandatory patch, it will be increased by one for each mandatory patch.
    • 'optional' having type integer. It will contain the version of the last successfully executed mandatory or optional patch, it will be increased by one for any patch. 02020000->02020001->02020002->02020003->etc
  • Schema of the database will have INSERT statement to populate table dbversion in order to make possible installation of proxies without loading data.sql. Autocreation of databases will also populate dbversion.

Therefore it is possible that 'optional' is higher than 'mandatory', but not vice versa.

C structures

  • A single patch must have sequence number (db version) and corresponding SQL statements for each DB engine or a reference to C routines. If there is a patch for MySQL but no patch needed for Oracle, fake execution of the patch will increase Oracle DB version. Therefore it will guarantee that DB version is exactly the same for different DB engines.
  • DB version should contain also Zabbix major version to allow patching (missing index, for example) of several branches, like, 2.2 and 2.4. It should have a form of XXXXNNNN, where XXXX - major (stable or unstable) release, NNNN - DB patch sequence number. For example, 02020001 (0202 == 2.2 to allow versions like 2.10 or 10.2). Sequence number is reset to 0001 after increase of major release (02020005->02020006->02030001).

Possible issues and open questions

  • Patches will be compiled into server and proxy binaries, it will increase their size and static data area. It could be a problem on some embedded platforms.
  • We never did it before, but the patches may also potentially contain binary data

Documentation

  • Update what's new in 2.2
  • Update installation procedure
  • Update upgrade procedure
  • Update upgrade notes (including partitioned setups not being detected)
  • Update training materials
  • Update database structure
  • Add information about DB permission requirements (add, modify, drop tables/indexes/fields/etc)
  • Update product description

Test cases

  • Installation of Zabbix from sources is successful, all components (frontend, proxy, server) are operational.
  • On startup server and proxy display information about DB versions.
  • Upgrade from one mandatory version to higher version is successful.
  • Zabbix frontend, server and proxy complains if it requires higher version of the database.
  • Optional patches added for 2.1.x will be executed for 2.1.(x+1). Downgrade to 2.1.x is successful. Frontend, server and proxy do not complain in both cases.
  • If a patch fails, Zabbix server or proxy will complain. When failure reason is resolved, server or proxy will re-execute this patch and continue execution of other patches normally.
  • Progress information is visible in the server (proxy) log while executing patches.
  • Optional patch added for 2.1.x and 2.2.y will be executed only once when upgrading from 2.1.(x-1) to 2.2.(y+1).
  • Proxy could be installed without loading data.sql. Also, autocreation of SQLite database should fill values for table dbversion.

ChangeLog

  • v1.1
    • Each patch must be as small and atomic as possible to make possible nice recovery if it fails.
  • v1.2
    • Fixed examples: 02020005->02020006->02030001 instead of 0202005->0202006->0203001.
  • v1.3
    • Clarified that after a failure, next run will start reporting at 0% again
    • Added that frontend will also complain if dbversion table is missing
  • v1.4
    • Updated message about a database version mismatch