SYSLOG-Tools(at)Work

SYSLOG-Tools(at)Work / Technical description

SYSLOG-Tools(at)Work

Architecture, protocols, security and operations for SYSLOG-Tools(at)Work (Receiver and Sender)

Version 2.0.0Revised 2026-10-03Receiver and Sender

SYSLOG-Tools(at)Work consists of two Python programs, SYSLOG Receiver and SYSLOG Sender, each with its own web interface. This document describes the architecture, protocol handling, security mechanisms, installation model and interfaces, and is written for operations, security review and further development.

1. Purpose and scope

Product

SYSLOG-Tools(at)Work 2.0.0

Components

SYSLOG Receiver (receiver.py), SYSLOG Sender (sender.py), shared modules (syslog_common.py, syslog_ui.py, syslog_i18n.py), installation scripts

Use

Testing, troubleshooting, demonstration and training in syslog (UDP and TLS 1.3), and validation of syslog configuration in network equipment and servers

Not intended for

Long-term storage, SIEM or log archiving. The receiver keeps messages in memory only.

Target platforms

Rocky Linux 9 / RHEL 9 / AlmaLinux 9 (primary), Debian 12, Ubuntu 22.04/24.04, Photon OS – on bare metal, VMware VM, Proxmox VE VM and Proxmox VE LXC

Runtime model

Ordinary Linux processes managed by systemd (no containers, no pip, no internet access)

Languages

English and Norwegian (Bokmål) in the web interfaces, server messages, manuals and installation scripts

2. Architecture

2.1. Overview

                 senders (network equipment, servers, SYSLOG Sender)
                      │ UDP 514                 │ TLS 1.3 / TCP 6514
                      ▼                         ▼
 ┌───────────────────────────── syslog-receiver.service ─────────────────────────────┐
 │  UDP thread ─┐                    TLS listener ─► one thread per connection        │
 │              ├─► parse (RFC 3164/5424) ─► ring buffer (deque, 2000) + statistics   │
 │              │                                  ▲                                  │
 │  HTTP server (ThreadingHTTPServer, 8080) ───────┘  /api/messages, /api/stats …     │
 │     ├─ login, CSRF, Host check (WebSecurityMixin)                                  │
 │     ├─ certificate API (CertAPIMixin) ──► /opt/syslog-tools/certs                  │
 │     └─ HTML/JS page + manual (EN/NO)                                               │
 └────────────────────────────────────────────────────────────────────────────────────┘

 ┌───────────────────────────── syslog-sender.service ───────────────────────────────┐
 │  HTTP server (8000) ─► /send ─► SyslogHTTPConfigurator ─► UDP socket / TLS 1.3     │
 │                       └─► RepeatJob (background thread, fixed rate, stoppable)    │
 │  certificate API (shared code) ─► /opt/syslog-tools/certs                         │
 │  command line: python3 sender.py -d … -m …  (without web)                         │
 └────────────────────────────────────────────────────────────────────────────────────┘

2.2. Code modules

File Lines (approx.) Contents

Receiver/receiver.py

1,650

SyslogReceiver (UDP loop, TLS listener, parser, buffer, statistics, TLS settings that can be changed while running), HTTPHandler (web/API), generate_certificates() (test PKI), main()

Sender/sender.py

1,600

SyslogHTTPConfigurator (validation, RFC 3164/5424 formatting, UDP/TLS sockets), RepeatJob (repeated sending), SyslogHTTPHandler (web/API), command line

Common/syslog_common.py

820

Network helpers (IPv4/IPv6, dual stack), CertStore (certificate directory), WebSecurity/WebSecurityMixin (login, CSRF, Host check, per-request language, manual, licenses, log, logout), CertAPIMixin

Common/syslog_ui.py

700

Shared user interface: theme (CSS), navbar, footer, tabs, logout, certificates tab, logs tab, the /logged-out and /licenses pages, and LogStore/LogCapture (copy of the program output for the Logs tab)

Common/syslog_i18n.py

540

Norwegian dictionary for the user interface (NO_UI), regular expressions for translating server messages (MESSAGE_PATTERNS), translate_message(), the JavaScript engine I18N_JS

There are no dependencies outside the Python standard library. The minimum Python version is 3.7, checked with vermin.

2.3. Process and thread model

  • Receiver: one process. One thread reads UDP (recvfrom in a loop), one thread accepts TLS connections, and each TLS connection gets its own thread (daemon threads). HTTP is handled by ThreadingHTTPServer (one thread per request). Shared data (buffer and statistics) is protected by a single threading.Lock.

  • Sender: one process with ThreadingHTTPServer. Single messages are sent in the request thread (serialised with single_lock). Repeated sending runs in a separate RepeatJob thread with its own socket. Only one job can run at a time (job_lock).

3. Protocols

3.1. Receiving (Receiver)

UDP (RFC 5426)

One message per datagram, up to 65,535 bytes. No SO_REUSEADDR, so another program cannot share the port.

TLS (RFC 5425)

TLS 1.3 only (minimum_version = maximum_version = TLSv1_3). Framing: octet counting LEN SP MSG, with a fallback for newline-terminated messages (RFC 6587). Max 64 KiB per frame.

Client certificate

Optional (CERT_OPTIONAL) when trusted CAs/certificates are selected, required (CERT_REQUIRED) with mutual TLS. VERIFY_X509_PARTIAL_CHAIN lets a self-signed client certificate or an intermediate certificate be trusted directly.

Limits

256 concurrent TLS connections, 32 per source address, 15 s to complete the handshake, 600 s idle time (--tls-idle-timeout), at most 10,000 source addresses in the statistics (the rest are counted as "(other)", because UDP addresses can be spoofed)

3.1.1. Parsing

  1. <PRI> (0–191) gives facility = PRI ÷ 8 and severity = PRI mod 8. If PRI is missing or invalid, USER.NOTICE is used and the message is marked NO-PRI (RFC 3164 §4.3.3).

  2. RFC 5424: 1 TIMESTAMP HOST APP PROCID MSGID SD [MSG]. Structured data is parsed with quotes and escape characters (\", \], \\), and any BOM in front of the message is removed.

  3. RFC 3164: Mmm dd hh:mm:ss or ISO timestamp, then host and TAG[PID]:. In practice all parts are optional.

Fields in a parsed message: id, received_at, source_ip, transport, tls_version, tls_cipher, client_cn, facility, severity, rfc_type, timestamp, hostname, app_name, procid, msgid, structured_data, message, raw.

3.1.2. Buffer and live view

Messages are kept in a collections.deque(maxlen=N) (default 2000, --buffer). The browser polls /api/messages?since=<last id> every second and receives only new messages. An "epoch" (start_token-clear_count) tells the browser whether the buffer was cleared or the service restarted, so that the whole list is reloaded. Statistics are fetched every two seconds. A new request is sent only when the previous one has finished, so requests do not pile up on slow links.

3.2. Sending (Sender)

RFC 3164

<PRI>Mmm dd hh:mm:ss HOST TAG[PID]: MSG – English month name, day padded with a space, TAG max 32 characters without [, ] and :

RFC 5424

<PRI>1 ISO8601-with-time-zone HOST APP PROCID MSGID SD MSG, field lengths per RFC 5424 §6 (255/48/128/32), printable ASCII only in header fields, - for empty fields

MAC address

RFC 3164: [MAC:XX:…] in front of the text. RFC 5424: [device@32473 mac="…"] (32473 = the number reserved for documentation, RFC 5612).

UDP

The address is resolved once per socket. Optional source address (bind). Max 65,507 bytes.

TLS 1.3

create_connection with a 10 s timeout, SNI/host name check against the target or the expected server name, and verification against the selected CA or the system CA store. After the handshake the sender waits 0.3 s for a possible alert, because a TLS 1.3 receiver rejects client certificates after the handshake. The connection is reused and checked before each message, and reconnected once on error.

Repeated sending

Messages are scheduled against the start time (start + i × 60/rate), so the rate does not drift over time. Stop can interrupt the wait. 1–1000 messages per minute for 1–1440 minutes.

4. Web interface

4.1. Structure

Each page is a single self-contained HTML document, assembled by the program on the first request and then cached. Shared parts from syslog_ui.py are inserted through placeholders: {{THEME_CSS}}, {{NAVBAR}}, {{FOOTER}}, {{I18N_JS}}, {{FETCH_JS}}, {{APP_JS}}, {{CERT_PANEL}}, {{CERT_JS}}, {{LOG_PANEL}}, {{LOG_JS}}, {{APP_TITLE}}, {{APP_ICON}}, {{HOSTNAME}}, {{VERSION}} and {{DEFAULT_LANG}}. The pages load no external resources and use no JavaScript libraries. The icons are inline SVG drawings, and the arrow in the drop-down lists is an embedded SVG (data: URI).

Design

The same visual language as SNMP-Trap-Tools(at)Work: navbar #084f39 with icon, name, status badge and host name; ? User manual, EN | NO and Log out on the right; tabs; tables, forms and buttons in a Bootstrap-like style (primary colour #0d6efd, borders #dee2e6, info #0dcaf0); the system’s own fonts.

Tabs

Tabs in APP_JS. The selected tab is stored in the address (#tab=name). Each tab has data-help with the section id in the manual (s3-1, s3-3, s4, s5, s6), used by ? User manual.

Message table

One <tbody class="row-main"> per message (main row and optional detail row). New messages are inserted at the top, and messages that have dropped out of the buffer are removed at the bottom, without redrawing the rest of the table. Open rows are remembered (openIds).

Logout

HTTP Basic has no logout. The Log out button sends an XMLHttpRequest with an invalid user name/password to /api/logout, which always answers 401 without WWW-Authenticate (no dialog). The browser then replaces the stored credentials. Afterwards /logged-out is shown, which does not require login. This works in Chrome, Edge and Firefox. Close the browser to be completely sure.

Logs

install_log_capture() wraps sys.stdout/sys.stderr in a LogCapture. Everything written still goes to the journal, and the last 2000 lines are kept in memory with a timestamp. GET /api/log?since=N returns new lines.

Manual

tools/build-manuals.py builds manual.html (English) and manual.no.html (Norwegian) from Markdown with pandoc, in the same design: side menu with search (hides chapters that do not match and highlights hits), table of contents and Back to the application. The screenshots are embedded, so each manual is a single file of about 0.4 MB.

4.2. Language (i18n)

Source language

English. All texts are written in English in HTML/JS and translated with NO_UI.

Static text

At start-up I18N.capture() walks all text nodes and the placeholder, title and aria-label attributes, and remembers those that have a Norwegian translation. I18N.apply(lang) swaps the values without rebuilding the page. Elements with data-i18n="key" are swapped as HTML (help texts with <code>). data-no-i18n turns translation off for an area (e.g. paths and certificate names).

Dynamic text

t('English text', {var: value}) with placeholders. Values coming from the network are escaped (esc()) before they are inserted.

Language switch

Listeners (I18N.onChange) redraw the message list, statistics, TLS form, certificate list, job status and result box.

Storage

localStorage (syslog_tools_lang) and a cookie with the same name (SameSite=Strict), so that /manual also follows the language

Server messages

fetch sends X-UI-Lang. WebSecurityMixin._send_json() and _deny() translate the message field with translate_message(), which applies regular expressions in order. Groups are translated recursively (max depth 4), and unknown texts (e.g. errors from OpenSSL) are returned unchanged.

Default language

--lang en|no (from install.sh --lang/--web-lang)

Local formats

Time of day with toLocaleTimeString('nb-NO' / 'en-GB')

4.3. HTTP API

See the User Manual, chapter 8, for all endpoints. All responses are JSON (application/json; charset=utf-8), and state-changing calls are POST.

5. Security

5.1. Access control in the web layer

Login

HTTP Basic, user admin. The password is stored in a file (600, owned by the service user) and compared with hmac.compare_digest. A wrong password causes a 1 s delay to slow down guessing.

CSRF

POST requires the header X-Requested-With: syslog-ui, which other websites cannot set without CORS. If Origin is set, it must match Host.

DNS rebinding

Host must be an IP address or a known host name (localhost, the machine name, --allowed-host).

Clickjacking etc.

X-Frame-Options: DENY, Content-Security-Policy: frame-ancestors 'none', X-Content-Type-Options: nosniff, Referrer-Policy: no-referrer

XSS

All text from the network (syslog messages, file names, certificate fields) is escaped before it is inserted into HTML.

File selection

TLS files selected in the browser must be located directly in the certificate directory (CertStore.resolve_web, realpath check), or be given by the administrator on the command line.

Upload

Max 1 MiB, file names matching ^[A-Za-z0-9][A-Za-z0-9._-]{0,99}$ with allowed extensions. The content is checked (PEM blocks or DER). Keys are written with O_EXCL|O_NOFOLLOW and mode 600 from the start, and moved atomically into place.

Download

Files that contain a private key can never be downloaded.

Certificate generation

openssl is run with an argument list (never a shell), and all values are validated with regular expressions.

Read-only mode

--no-cert-upload disables upload, deletion, generation and TLS changes.

5.2. Process isolation (systemd)

User

syslog-tools (system user, no login)

Privileges

AmbientCapabilities=CAP_NET_BIND_SERVICE and CapabilityBoundingSet=CAP_NET_BIND_SERVICE – only the right to bind ports below 1024

Profile full

NoNewPrivileges, ProtectSystem=strict (everything read-only except ReadWritePaths=/opt/syslog-tools/certs), ProtectHome, PrivateTmp, PrivateDevices, ProtectKernelTunables/Modules/Logs, ProtectControlGroups, ProtectClock, ProtectHostname, RestrictNamespaces, RestrictRealtime, RestrictSUIDSGID, LockPersonality, SystemCallArchitectures=native, RestrictAddressFamilies=AF_INET AF_INET6 AF_UNIX AF_NETLINK, UMask=0027

Profile basic

NoNewPrivileges, UMask=0027 and the privileges above. Used in LXC, where namespaces and mounts are often blocked (226/NAMESPACE).

Profile selection

auto: systemd-detect-virt --container → basic, otherwise full. If the service does not start with full, the installer rewrites the unit files with basic.

SELinux

/usr/bin/python3 started by systemd runs as unconfined_service_t, so no custom policy is needed. restorecon sets the correct labels on /opt/syslog-tools.

5.3. Known limitations

  • The web interface uses plain HTTP (not HTTPS). Passwords and uploaded keys travel unencrypted over the network. Mitigation: a dedicated management network, --admin-net, or a reverse proxy with TLS in front.

  • HTTP Basic has no logout or session timeout (the browser remembers the login until it is closed).

  • UDP syslog is neither authenticated nor encrypted, and source addresses can be spoofed.

  • Messages are stored in memory only.

6. Installation and file layout

6.1. Installed layout

Path Owner/mode Contents

/opt/syslog-tools/

root 755

VERSION, install.conf, uninstall.sh, check-prereqs.sh, firewall-open.sh, lib/, docs/

/opt/syslog-tools/receiver/

root 755

receiver.py, syslog_common.py, syslog_i18n.py, manual.html, manual.no.html

/opt/syslog-tools/receiver/web-password

syslog-tools 600

Password for the web interface

/opt/syslog-tools/sender/

root 755

The same for the sender

/opt/syslog-tools/certs/

syslog-tools 750

Certificates and keys (600), receiver-tls.json

/etc/systemd/system/syslog-receiver.service

root 644

Service

/etc/systemd/system/syslog-sender.service

root 644

Service

6.2. The installer

install.sh runs through 8 steps: prerequisites, user and directories, program files, certificates, systemd, start (with fallback to basic), firewall and verification. It can be run again as many times as you like (idempotent): choices from install.conf become defaults, passwords and certificates are kept, and the services are restarted. The verification sends a test message over UDP and looks for it with /api/messages.

Platform detection (lib/common.sh):

  • /etc/os-release (read in a subshell) → OS_FAMILY = rhel / debian / photon / suse / other

  • systemd-detect-virt → VIRT = none / vmware / kvm / lxc …, and IS_CONTAINER

  • firewall: active firewalld → firewall-cmd, active ufw → ufw, otherwise no change

6.3. Upgrading from 1.x

When /opt/syslog-infrastructure exists and the new certificate directory is empty, the certificates are copied and the paths in receiver-tls.json are rewritten. The web passwords are copied. The service names are the same as before and are overwritten. The programs also look for the old certificate directory if the new one is missing (LEGACY_ROOT).

7. Configuration

7.1. Receiver – command line

Option Default Description

--listen

0.0.0.0

Listen address (:: = dual stack)

--syslog-port

514

UDP port

--http-port / --http-bind

8080 / 0.0.0.0

Web interface

--buffer

2000

Messages in memory

--quiet

off

Do not write every message to the journal

--tls, --tls-port, --tls-cert, --tls-key, --tls-ca (repeatable), --tls-require-client-cert

TLS at start-up (overridden by receiver-tls.json)

--tls-idle-timeout

600

Seconds before an idle TLS connection is closed

--cert-dir

/opt/syslog-tools/certs

Certificate directory

--no-cert-upload, --ignore-saved-tls

Read-only / ignore saved TLS settings

--gen-certs DIR --cert-names LIST

Create a test PKI and exit

--auth-file, --no-auth, --allowed-host

Login and host names

--lang

en

Default language of the web interface (en/no)

7.2. Sender – command line

--http, --port, --http-bind, --lang, --auth-file, --no-auth, --allowed-host, --cert-dir and --no-cert-upload control web mode. -d, -s, -t, -p, -m, --rfc, --facility, --severity, --hostname, --app, --mac, --procid, --msgid, --repeat, --duration and --tls- control command-line mode. In web mode --tls- are used as defaults in the form.

7.3. Stored state

receiver-tls.json

TLS settings from the web interface (enabled, port, cert, key, trusted[], require_client_cert)

install.conf

The choices from the installation (KEY="value"). Read line by line without source.

The browser

syslog_tools_lang (language), syslog-sender-tls (the sender’s TLS choices)

8. Operations

  • Log: the Logs tab in the web interface (last 2000 lines), or journalctl -u syslog-receiver / -u syslog-sender. Every message (unless --quiet is used), every TLS connection, every failed handshake and every certificate change is logged in English.

  • Monitoring: GET /api/stats (requires login) returns counters and status (udp_status, tls_status).

  • Resource usage: A message typically takes 1–2 KiB of memory, so the default buffer of 2000 messages uses only a few MiB. The whole process typically uses 20–40 MiB RAM. CPU usage grows with the message rate. At high traffic --quiet should be used, because writing to the journal otherwise dominates.

  • Backup: /opt/syslog-tools/certs and the web-password files. Everything else can be reinstalled from the package.

9. Quality assurance

See Revisjonsrapport.md (audit report, Norwegian) for code review, security assessment and test results.

Screenshots, installation and how the two tools are used together: Back to the front page