ZCP_day_1_slides_Part6.pdf

ALLOWKEYS VS DENYKEYS Using allow keys may be safer, but harder to maintain: Y P Deny rule is defined for a single file: DenyKey=vfs.file.contents[/etc/passwd] O C This setting seems completely safe with explicitly denied key: # zabbix_get -s my.prod.host -k vfs.file.contents["/etc/passwd"] ZBX_...

ALLOWKEYS VS DENYKEYS Using allow keys may be safer, but harder to maintain: Y P Deny rule is defined for a single file: DenyKey=vfs.file.contents[/etc/passwd] O C This setting seems completely safe with explicitly denied key: # zabbix_get -s my.prod.host -k vfs.file.contents["/etc/passwd"] ZBX_NOTSUPPORTED: Unknown metric vfs.file.contents However, it can be tricked: T N E D # zabbix_get -s my.prod.host -k vfs.file.contents["/tmp/../etc/zabbix/../passwd"] root:x:0:0:root:/root:/bin/bash bin:x:1:1:bin:/bin:/sbin/nologin daemon:x:2:2:daemon:/sbin:/sbin/nologin adm:x:3:4:adm:/var/adm:/sbin/nologin U T S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 101 AGENT KEY NOTES A few important notes about agent key rules: Y P If the deny rules are not defined at all: Allow list without a single deny rule is only permitted for system.run[*] items For all the other items agent exits with an error: "AllowKey without DenyKey rules are meaningless" O C When the key is restricted by configuration, the agent will treat it as if it is not supported T N Zabbix agent command line options will not show keys that are not allowed Denied remote commands will not be logged in the agent log (on any debug level) E D AllowKey, DenyKey rules do not affect the configuration parameters: HostnameItem HostMetadataItem HostInterfaceItem U T S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 102 PRACTICAL SETUP Y P 1) Allow key: Free swap size in a percentage "system.swap.size[,pfree]" 2) Deny keys: Deny all other keys starting with "system.swap" T N O C 3) On the Latest data screen for student-XX host check that the keys are blocked 4) Test that all the system.swap items are blocked except for swap percentage U T E D S Day 1 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved 10 minutes Practical task No: 6 103 Y P T N O C E DZabbix Agent U T Remote Commands S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved 10 minutes 104 AGENT REMOTE COMMANDS - SYSTEM.RUN Agent key for executing remote commands using passive or active checks Y P system.run[command,<mode>] <mode> = wait: <mode> = nowait: wait for execution do not wait for execution T N O C The return value of the item is a standard output together with a standard error The exit code is not checked E D It is recommended to log remote commands using LogRemoteCommands option ### Option: Plugins.SystemRun.LogRemoteCommands # Enable logging of executed shell commands as warnings. # 0 - disabled # 1 - enabled # Mandatory: no Plugins.SystemRun.LogRemoteCommands=1 U T S ! system.run[*] items are disabled by default i https://www.zabbix.com/documentation/6.0/manual/appendix/command_execution 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 105 AGENT REMOTE COMMANDS - SYSTEM.RUN Write the script or prepare the command: Y P Recommended to test from the command line with the “Zabbix” user permissions Configuration: Configure and restart Zabbix agents: ## zabbix_agentd.conf AllowKey=system.run[*] LogRemoteCommands=1 T N U T E D O C ## zabbix_agent2.conf AllowKey=system.run[*] Plugins.SystemRun.LogRemoteCommands=1 Test your command/script from Zabbix by using: Item testing form on Zabbix web frontend Agent binary with -t option and key zabbix_get utility S Add a new item in Zabbix frontend: Use Zabbix agent or Zabbix agent (active) item type 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 106 AGENT REMOTE COMMANDS A few examples with agent remote commands Y P system.run[ipcs -u] T N O C E D system.run["powershell.exe -NoProfile -Nologo -File C:\scr\get_updates.ps1"] U T S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 107 REMOTE COMMANDS There are multiple places, where remote commands are used: Y P system.run[*] item checks Zabbix frontend scripts Remote commands executed by actions E D U T S 6.0 Certified Professional ● Day 1 T N O C © 2023 by Zabbix. All rights reserved Theory 108 Y P E D Encryption U T S 6.0 Certified Professional ● Day 1 T N O C © 2023 by Zabbix. All rights reserved 30 minutes 109 INTERNAL COMMUNICATIONS There are many reasons to protect communication between Zabbix components: Y P Configuration data sent from Zabbix server to a proxy may contain credentials Collected values may contain sensitive information O C Remote commands can be executed on Zabbix agents or proxies Prevent sending spoofed data to Zabbix - trust your peers T N Transport Layer Security protocols TLS 1.2 and TLS v1.3 are used E D Version depends on the crypto library Recommended to use the latest crypto library versions (TLS 1.3) U T Use only safe ciphers S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 110 CONNECTION ENCRYPTION Zabbix 6.0 can natively encrypt communications between: Y P Zabbix server and proxies Zabbix server and Zabbix web services (reporting server) Zabbix server/proxies and Zabbix agents Zabbix server/proxies and databases T N O C Zabbix server/proxies/Zabbix agents and command-line utilities E D Browser communication with Zabbix frontend can be protected using HTTPS U T Currently, it is not possible to natively encrypt communications between: S Zabbix server/proxies and Zabbix Java gateway Zabbix web server (PHP) and Zabbix server (Test item, for example) 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 111 ZABBIX SECURITY MODEL Zabbix agents Browser U T S T N E D Zabbix web Zabbix server Zabbix database 6.0 Certified Professional ● Day 1 O C Command-line utilities Web services © 2023 by Zabbix. All rights reserved Y P Zabbix proxy Zabbix Java gateway Proxy database Theory 112 DB ENCRYPTION OVERVIEW It is possible to protect communication between Zabbix and the database: Y P Implemented by using TLS and certificates (client, server and CA) Supported for Zabbix frontend and backend (different options may be used) Multiple security levels possible: Encrypt connection Verify certificate issuer Check certificate details Am I connecting to trusted database server ? T N O C Who is this client connecting to me ? E D Supported for the following DB engines: U T MySQL (native) PostgreSQL (native) S Oracle (using client configuration) Not possible if UNIX socket connections are used to connect to database 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 113 SETTING UP DB ENCRYPTION The simplest encryption of DB connection is easy to set up: Y P Set parameter DBTLSConnect=required in Zabbix server or proxy configuration file O C ### Option: DBTLSConnect # Setting this option enforces to use TLS connection to database. # required - connect using TLS # verify_ca - connect using TLS and verify certificate # verify_full - connect using TLS, verify certificate and verify that database identity DBTLSConnect=required T N E D Set $DB['ENCRYPTION'] = true; in the Zabbix frontend configuration file U T // Used for TLS connection. $DB['ENCRYPTION'] S = true; Advanced setups require user generated custom CA, server and client certificates ! Encryption must be supported by database engine! (works for MySQL 8 / MariaDB out-of-box) 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 114 ENCRYPTION TYPES Zabbix supports the following SSL libraries: Y P OpenSSL 1.0.1, 1.0.2, 1.1.0, 1.1.1 and 3.0.x LibreSSL from 2.7 Supported as a compatible replacement of OpenSSL PSK are not supported, only certificates can be used GnuTLS from 3.1.18 T N O C Mixed encryption example Host connection can be configured to use E D Pre-shared key (PSK) U T Certificate S Mixed (only for incoming connections) 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Zabbix server Theory 115 PSK SETTINGS Each PSK in Zabbix is a pair of: PSK identity non-secret identity string PSK value secret string Y P The PSK identity and value are stored: T N In host / proxy settings in Zabbix frontend U T E D S O C In Zabbix agent / proxy configuration file ### Option: TLSPSKIdentity # Unique, case sensitive string used # to identify the pre-shared key. TLSPSKIdentity=New York identity ### Option: TLSPSKFile # Full pathname of a file containing # the pre-shared key. TLSPSKFile=/etc/zabbix/mykey.psk If both parties have the same PSK identity and value, the connection will proceed 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 116 PSK KEY PROTECTION The pre-shared keys (PSK) are protected in the frontend: Y P Enter the PSK identity and a key Press Update The key is no longer visible T N O C U T E D Pre-shared keys are stored as a plain text: S In Zabbix database - protect DB access and backup files In text files on Zabbix agents and proxies - protect file access with access permissions 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 117 ENCRYPTION - PSK SETUP How to setup PSK encryption on Zabbix agent or Zabbix proxy: Y P Create a directory for keys: # mkdir /etc/zabbix/keys # chown zabbix:zabbix /etc/zabbix/keys # chmod 700 /etc/zabbix/keys Generate a PSK key and save to a file: T N # openssl rand -hex 32 > /etc/zabbix/keys/mykey.psk # chmod 400 /etc/zabbix/keys/mykey.psk # chown -R zabbix:zabbix /etc/zabbix/keys O C E D Change Zabbix agent or proxy configuration: U T TLSAccept=psk TLSPSKIdentity=My Identity TLSPSKFile=/etc/zabbix/keys/mykey.psk S Restart the agent or proxy: # systemctl restart zabbix-agent # systemctl restart zabbix-proxy ! For active checks set TLSConnect=psk option! 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 118 PSK IDENTITIES AND KEYS Two different keys can not be used with the same PSK identity: Working configuration PSK identity 01 Host A PSK identity 01 a56a32ee48d920374cff6b19f3a2f77c2f7fa 57254004ae82340d1ca48eb9e45 Host A T N PSK identity 02 Host B Y P Incorrect configuration a56a32ee48d920374cff6b19f3a2f77c2f7fa 57254004ae82340d1ca48eb9e45 E D Host B O C a56a32ee48d920374cff6b19f3a2f77c2f7fa 57254004ae82340d1ca48eb9e45 PSK identity 01 4e42ab76f840505b553c769908c0f09c6a11 857ae86410875cca6455bcf5b631 033857.546 033912.323 033927.328 033942.331 033939.335 033954.581 034009.357 034024.363 034039.373 034042.378 Zabbix agent item "proc.num[]" on host "Host A" failed: first network error, wait for 15 seconds Day Zabbix agent item "system.cpu.load[percpu,avg1]" on host "Host A" failed: another network error, wait for 15 seconds Zabbix agent item "system.swap.size[,pfree]" on host "Host A" failed: another network error, wait for 15 seconds temporarily disabling Zabbix agent checks on host "Host A": host unavailable enabling Zabbix agent checks on host "Host B": host became available Zabbix agent item "system.swap.size[,pfree]" on host "Host B" failed: first network error, wait for 15 seconds Zabbix agent item "system.localtime" on host "Host B" failed: another network error, wait for 15 seconds Zabbix agent item "vm.memory.size[available]" on host "Host B" failed: another network error, wait for 15 seconds temporarily disabling Zabbix agent checks on host "Host B": host unavailable enabling Zabbix agent checks on host "Host A": host became available 033857.541 line 1629: 033912.320 line 1629: failed to accept an incoming connection: from 10.20.30.40: TLS handshake set result code to error:141FA0FD:SSL routines:tls_psk_do_binder:binder does not verify: TLS write fatal alert failed to accept an incoming connection: from 10.20.30.40: TLS handshake set result code to error:141FA0FD:SSL routines:tls_psk_do_binder:binder does not verify: TLS write fatal alert U T S 6.0 Certified Professional Day 2 ● Day 1 © 2023 by Zabbix. All rights reserved 2 1: file ssl/statem/extensions.c "illegal parameter" 1: file ssl/statem/extensions.c "illegal parameter" Theory 119 ENCRYPTION USING CERTIFICATES CA certificate, client certificate and certificate private key must be provided: Y P For Zabbix server, all proxies and agents using certificates for encryption This affects both active and passive communications Verify certificates Server certificate T N CA Certificate Certificate private key E D U T O C CA Certificate Proxy certificate Certificate private key ### Option: TLSCAFile TLSCAFile=/etc/zabbix/ssl/ca.pem ### Option: TLSCAFile TLSCAFile=/etc/zabbix/ssl/ca.pem ### Option: TLSCertFile TLSCertFile=/etc/zabbix/ssl/server-cert.pem ### Option: TLSCertFile TLSCertFile=/etc/zabbix/ssl/proxy-cert.pem ### Option: TLSKeyFile TLSKeyFile =/etc/zabbix/ssl/server-key.pem ### Option: TLSKeyFile TLSKeyFile =/etc/zabbix/ssl/proxy-key.pem S 6.0 Certified Professional ● Day 1 © 2023 by Zabbix. All rights reserved Theory 120