See www.zabbix.com for the official Zabbix site.

Docs/specs/ZBX-8410

From Zabbix.org
Jump to: navigation, search

Global expression compilation

ZBX-8410

Status: Initial draft, do not comment

Owner: Andris Zeila

Summary

Currently global and regular expressions are compiled at matching time which leads to two issues:

  • impossible to tell if matching failed because of an invalid expression or because of malformed input
  • impossible to optimize multiple item matching by compiling the expression once and using it to perform matching

To solve these problems Zabbix internal APIs must provide explicit means of regular expression compilation.

Specification

The global expression API is provided to work with Zabbix global and simple regular expressions.

Global expression API

/* the regular expression is initialized, but not yet compiled */
#define ZBX_GEXPR_INITIALIZED		0
/* the regular expression is compiled and can be used with matching functions */
#define ZBX_GEXPR_COMPILED		1
/* the regular expression compilation failed */
#define ZBX_GEXPR_INVALID		2

/* this structure is used to store Zabbix global regular expressions */
/* or a simple regular expression                                    */
typedef struct
{
	/* global regular expression name or simple regular expression pattern */
	char			*regexp;

	/* global regular expressions */
	zbx_vector_ptr_t	expressions;

	/* pre-compiled expressions */
	regex_t			*compiled;

	/* the expression status (the pre-processor defines above) */
	int			status;
}
zbx_gexpr_t;

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_init                                                   *
 *                                                                            *
 * Purpose: initializes global expression with simple regular expression      *
 *                                                                            *
 * Parameters: gexpr   - [IN] the global expression                           *
 *             pattern - [IN] the regular expression pattern                  *
 *                                                                            *
 * Comments: To free resources allocated by global expressions use            *
 *           zbx_gexpr_clean() function.                                      *
 *                                                                            *
 ******************************************************************************/
void	zbx_gexpr_init(zbx_gexpr_t *gexpr, const char *pattern);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_init_global                                            *
 *                                                                            *
 * Purpose: initializes global expression with Zabbix global regular          *
 *          expression                                                        *
 *                                                                            *
 * Parameters: gexpr  - [IN] the global expression                            *
 *             name   - [IN] the regular expression name                      *
 *                                                                            *
 * Comments: To free resources allocated by global expressions use            *
 *           zbx_gexpr_clean() function.                                      *
 *                                                                            *
 ******************************************************************************/
void	zbx_gexpr_init_global(zbx_gexpr_t *gexpr, const char *name);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_add                                                    *
 *                                                                            *
 * Purpose: adds an expression to Zabbix global regular expression            *
 *                                                                            *
 * Parameters: gexpr          - [IN] the global expression                    *
 *             expression     - [IN] the expression                           *
 *             type           - [IN] the expression type                      *
 *             case_sensitive - [IN] the expression flags                     *
 *                                                                            *
 ******************************************************************************/
void	zbx_gexpr_add(zbx_gexpr_t *gexpr, const char *expression, int type, char delimiter,
		unsigned char flags);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_compile                                                *
 *                                                                            *
 * Purpose: compiles global expression                                        *
 *                                                                            *
 * Parameters: gexpr  - [IN] the global expression                            *
 *             error  - [OUT] an optional error message                       *
 *                                                                            *
 * Return value: SUCCEED - the regular expression was compiled successfully   *
 *                         or already compiled.                               *
 *               FAIL    - the regular expression compilation failed or it    *
 *                         was already in invalid state.                      *
 *                                                                            *
 * Comments: In the case of Zabbix global regular expressions this function   *
 *           compiles all regular expressions stored in expressions vector.   *
 *           If the expressions vector is empty the gexpr is treated as       *
 *           a simple regular expression and this function compiles the       *
 *           regular expression pattern stored in regexp field.               *
 *                                                                            *
 ******************************************************************************/
int	zbx_gexpr_compile(zbx_gexpr_t *gexpr, char **error);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_match                                                  *
 *                                                                            *
 * Purpose: tries to match the text against global expression                 *
 *                                                                            *
 * Parameters: gexpr  - [IN] the global expression                            *
 *             text   - [IN] the text to match                                *
 *             error  - [OUT] an optional error message                       *
 *                                                                            *
 * Return value: SUCCEED - the text matches this global expression            *
 *               FAIL    - the text does not match this global expression or  *
 *                         an error occurred (expression was not compiled)    *
 *                                                                            *
 ******************************************************************************/
int	zbx_gexpr_match(zbx_gexpr_t *gexpr, const char *text, char **error);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_match_sub                                              *
 *                                                                            *
 * Purpose: tries to match the text against global expression substituting    *
 *          the specified output template with captured groups                *
 *                                                                            *
 * Parameters: gexpr  - [IN] the global expression                            *
 *             text   - [IN] the text to match                                *
 *             output - [IN] the output template. The result string is        *
 *                           constructed from the template by replacing \<n>  *
 *                           sequences with the captured regexp groups. If    *
 *                           output template is NULL the whole text is        *
 *                           returned.                                        *
 *             result - [OUT] the resulting matched string or value formated  *
 *                           according to the template.                       *
 *             error  - [OUT] an optional error message                       *
 *                                                                            *
 * Return value: SUCCEED - the text matches this global expression            *
 *               FAIL    - the text does not match this global expression or  *
 *                         an error occurred (expression was not compiled)    *
 *                                                                            *
 ******************************************************************************/
int	zbx_gexpr_match_sub(zbx_gexpr_t *gexpr, const char *text, const char *output, char *result, char **error);

/******************************************************************************
 *                                                                            *
 * Function: zbx_gexpr_clean                                                  *
 *                                                                            *
 * Purpose: releases resources allocated by global expression                 *
 *                                                                            *
 * Parameters: gexpr   - [IN] the global expression                           *
 *                                                                            *
 ******************************************************************************/
void	zbx_gexpr_clean(zbx_gexpr_t *gexpr);

Usage patterns

The following API call examples illustrates the common usage patterns.

Matching simple regular expression

Simple regular expression matching can be done anywhere - on server, proxy or agent.

zbx_gexpr_t     expr;
  ...
zbx_gexpr_init(&expr, "<regular expression pattern>");
if (SUCCEED == zbx_gexpr_compile(&expr, NULL) && SUCCEED == zbx_gexpr_match(&expr, "<text to match>", NULL))
  ...
zbx_gexpr_clean(&expr);

Matching global or simple regular expression on server/proxy

Server/proxy caches global regular expressions, so the global expression must be obtained from configuration cache.

zbx_gexpr_t     expr;
  ...
/* initialize global expression by either loading it from configuration cache */
/* or creating simple regular expression                                      */
DCget_global_expression(&expr, "<name|expression>");
if (SUCCEED == zbx_gexpr_compile(&expr, NULL) && SUCCEED == zbx_gexpr_match(&expr, "<text to match>", NULL))
  ...
zbx_gexpr_clean(&expr);

Matching global regular expression on agent (active checks)

During active checks synchronization agent must first free the old list of global expressions and then build the new one, based on the latest regular expression data received from server.

zbx_gexpr_t     *expr;
  ...
/* first release all global expressions in loop */
zbx_gexpr_clean(expr);
  ...

/* then for each global expression received from server */
zbx_gexpr_init_global(expr, "<name>");
zbx_gexpr_add(expr, ...);
zbx_gexpr_add(expr, ...);
 ...

/* when matching find the global expression by its name in the prepared lis and perform the match as usual */
if (SUCCEED == zbx_gexpr_compile(expr, NULL) && SUCCEED == zbx_gexpr_match(&expr, "<text to match>", NULL))
  ...

Notes

  1. Either more matching functions must be added or the defined functions must accept additional flags to specify case sensitivity and multiline matching.
  2. While having items in zbx_gexpr_t expressions vector is enough to determine if the global expression contains Zabbix global regular expression, it might be better to have 'type' field for this purpose.
  3. While it would be possible to directly load expressions from configuration cache into zbx_gexpr_t expressions vector with DCget_expressions_by_names() function, it would expose the global expression internals to configuration cache. If there are performance problems it might be better to add zbx_gexpr_copy() function which would copy a whole zbx_expression_t vector to zbx_gexpr_t expressions vector without calling zbx_gexpr_add() for each item.
  4. When compiling Zabbix global regular expressions - should we allocate the compiled expression array for all expressions or only for expressions containing real regexps? The first is easier to process, the second takes less space.