Our earlier troubleshooting guide covered the checklist — port checks, firewall rules, AMA heartbeat. This post goes one level deeper: what the Syslog via AMA connector actually configures on the box, why the internal port is 28330 and not 514, and how to prove data is flowing at each hop instead of guessing.
1
The Pipeline — What's Actually Happening
Three hops
Each with its own, separate failure mode
The path a message takes
514 → 28330 → 443
1
Source device → Syslog daemon
UDP/TCP 514
Network devices, firewalls, and other Linux hosts send RFC 3164/5424 syslog messages to your log forwarder's rsyslog or syslog-ng daemon.
2
Syslog daemon → Azure Monitor Agent
TCP 127.0.0.1:28330
The hop most people don't know exists. AMA doesn't listen on 514 itself — the syslog daemon forwards messages locally to AMA on 28330, using the omfwd output module (rsyslog) or a network() destination (syslog-ng). This config is installed automatically the moment you add a Syslog data source to a DCR.
3
AMA → Log Analytics workspace
HTTPS 443
AMA batches and ships the data to your workspace over HTTPS, landing in the Syslog table (plain syslog) or CommonSecurityLog table (CEF-formatted messages).
Why this changes troubleshooting
A tcpdump on port 514 that shows healthy traffic tells you nothing about whether the daemon successfully forwarded to AMA on 28330. Watch both ports, or you're only proving the first hop.
2
Data Collection Rule — Facility and Severity Filtering
Failure mode
Minimum severity set too high, or facility set to NONE
The Syslog via AMA connector creates a DCR with a Linux Syslog data source. In the Collect tab you set a minimum log level per facility — selecting a level collects that level and everything more severe.
Severity levels, low to high
Example cutoff: LOG_ERR
1
Debug
not collected
2
Info
not collected
3
Notice
not collected
4
Warning
not collected
5
Error
← LOG_ERR selected
6
Critical
collected
7
Alert
collected
8
Emergency
collected
So selecting LOG_ERR for a facility collects Error, Critical, Alert, and Emergency — not Debug, Info, Notice, or Warning. This is the single most common cause of "my logs aren't showing up" that isn't actually a connectivity problem — the facility was set to NONE, or the minimum severity was set too high for the events you're trying to capture.
Duplication risk
If you run both Syslog via AMA and Common Event Format (CEF) via AMA connectors on the same forwarder and configure the same facility in both DCRs, you'll ingest duplicate data — once into Syslog, once into CommonSecurityLog. Keep facility selections mutually exclusive between the two connectors.
3
Inside the Agent — rsyslog and syslog-ng Configs
What AMA drops on the box
The files that make hop 2 work
When Syslog is added to a DCR and AMA installs, it drops a config file that does the local forwarding. On rsyslog-based systems:
/etc/rsyslog.d/10-azuremonitoragent-omfwd.conf
Copy
# Azure Monitor Agent configuration: forward logs to azuremonitoragent
template(name="AMA_RSYSLOG_TraditionalForwardFormat" type="string" string="<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%")
*.* action(type="omfwd"
template="AMA_RSYSLOG_TraditionalForwardFormat"
queue.type="LinkedList"
queue.filename="omfwd-azuremonitoragent"
queue.maxFileSize="32m"
queue.maxDiskSpace="1g"
action.resumeRetryCount="-1"
action.resumeInterval="5"
queue.size="25000"
queue.workerThreads="100"
queue.dequeueBatchSize="2048"
queue.saveonshutdown="on"
target="127.0.0.1" Port="28330" Protocol="tcp")
queue.maxDiskSpace
1 GB
on-disk resilience window
rsyslog buffers to disk if AMA is unreachable, up to 1 GB, before it starts dropping messages. That buffer covers a brief AMA restart or network blip — it is not a substitute for fixing a persistent AMA outage.
For SELinux hosts using Unix sockets instead of TCP loopback:
/etc/rsyslog.d/10-azuremonitoragent.conf
Copy
$OMUxSockSocket /run/azuremonitoragent/default_syslog.socket
template(name="AMA_RSYSLOG_TraditionalForwardFormat" type="string" string="<%PRI%>%TIMESTAMP% %HOSTNAME% %syslogtag%%msg:::sp-if-no-1st-sp%%msg%")
$OMUxSockDefaultTemplate AMA_RSYSLOG_TraditionalForwardFormat
*.* :omuxsock:
On syslog-ng systems, the equivalent lives here:
/etc/syslog-ng/conf.d/azuremonitoragent-tcp.conf
Copy
destination d_azure_mdsd {
network("127.0.0.1"
port(28330)
log-fifo-size(25000));
};
log {
source(s_src);
destination(d_azure_mdsd);
flags(flow-control);
};
Multi-ruleset rsyslog gotcha
If your rsyslog config uses multiple rulesets, only inputs bound to the default ruleset get forwarded to AMA. If you're collecting from a custom ruleset and nothing shows up in Sentinel, this is almost always why — check /etc/rsyslog.d/ for ruleset bindings before assuming it's a network problem.
4
Supported Facilities
Reference
The full list AMA's Syslog collector understands
IndexFacility
IndexFacility
0kern
1user
2mail
3daemon
4auth
5syslog
6lpr
7news
8uucp
9cron
10authpriv
11ftp
12ntp
13audit
14alert
15clock
16local0
17local1
18local2
19local3
20local4
21local5
22local6
23local7
Highlighted rows are where most network appliances land
Most network appliances (firewalls, switches) default to local0–local7 for their own event streams — confirm which facility your device uses before setting DCR severity levels, since a mismatch here silently drops everything.
5
Securing the Path — TLS
Scope
The source-to-forwarder hop — the only one not already encrypted
If your log forwarder sits in the cloud and devices send logs to it over an untrusted network, encrypt the source-to-forwarder hop — the AMA-to-workspace hop is already TLS over 443 by default. Both daemons support it:
rsyslog
Configure omfwd (or a dedicated TLS-capable module) with certificate paths.
TLS tutorial →
syslog-ng
Configure the network() or syslog() destination with transport("tls") and certificate parameters.
TLS documentation →
Port change, not just a config change
The conventional TLS syslog port is 6514, distinct from the plaintext 514. If you switch to TLS, update firewall rules and device configurations to match, and confirm the daemon is actually listening on 6514 with netstat -lnptv before assuming the change took effect.
6
Log Forwarder Sizing
Production baseline
Dedicated forwarder at 250–300 GB/day
Spec for 250–300 GB/day
1 TB
Disk, generous /var/log
Our own production experience lines up with what we've documented before. A full disk stops AMA from functioning, not just from logging locally. Scale proportionally down for smaller ingestion volumes, but don't starve /var/log — the rsyslog buffer described above needs room to work.
7
Verification and Test Traffic
Prove it, don't guess
Listen · watch · send · confirm
Confirm the daemon is actually listening before you send anything:
Step 1 — is anything bound to 514?
Copy
netstat -lnptv
# Look for rsyslog or syslog-ng bound to port 514
Watch both the external port and the internal AMA forwarding port simultaneously:
Step 2 — watch hop 1 and hop 2 at once
Copy
sudo tcpdump -i any port 514 or 28330 -A -vv &
Send a synthetic message using logger, targeting a specific facility and severity in RFC 3164 format:
Step 3a — CEF-shaped test via logger
Copy
logger -p local4.warn -P 514 -n 127.0.0.1 --rfc3164 -t CEF "0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|rt=$common=event-formatted-receive_time"
Or send a raw UDP packet with nc for a plain syslog test:
Step 3b — raw UDP test via nc
Copy
echo -n "<164>CEF:0|Mock-test|MOCK|common=event-format-test|end|TRAFFIC|1|rt=$common=event-formatted-receive_time" | nc -u -w0 localhost 514
Then confirm ingestion — plain syslog lands in Syslog, CEF-formatted messages land in CommonSecurityLog. Allow up to 20 minutes for first arrival.
Step 4a — plain syslog arrival
Copy
Syslog
| where TimeGenerated > ago(1h)
| where HostName == "your-forwarder-hostname"
| order by TimeGenerated desc
Step 4b — CEF arrival
Copy
CommonSecurityLog
| where TimeGenerated > ago(1h)
| where DeviceProduct == "MOCK"
Check service health directly on the forwarder while you're at it:
Step 5 — both services up?
Copy
sudo systemctl status azuremonitoragent.service
sudo systemctl status rsyslog.service # or syslog-ng.service
8
Known Gotchas
Documented behaviour
Three things that look like bugs and aren't
01
Semicolons silently disappear
AMA forwards syslog data as received, but semicolon (;) characters get stripped during ingestion into Log Analytics. If your log format depends on semicolons as delimiters, replace them at the source with a comma or pipe before they ever hit the wire — don't discover this after weeks of malformed parsed fields.
02
Time zone mismatch between TimeGenerated and EventTime
TimeGenerated reflects when the forwarder processed the message in UTC. EventTime is extracted from the syslog header — which has no time zone info — and converted using the forwarder's local time zone offset. If your source device and forwarder sit in different time zones, expect a gap between the two fields. This is expected behaviour, not a bug.
03
Legacy sysklog isn't supported
Red Hat Enterprise Linux 5 and older Oracle Linux ship with sysklog by default, which AMA's Syslog collector doesn't support. Replace it with rsyslog before attempting to configure the connector.
Quick recap
If data isn't arriving, check in this order
Facility
Set to NONE, or the device sends on a facility you didn't select.
Min severity
Selecting LOG_ERR drops Warning and below. Lower it to test.
Port 28330
Hop 2 is the invisible one — tcpdump it alongside 514.
Rulesets
Only the default rsyslog ruleset forwards to AMA.
Duplicates
Same facility in both Syslog and CEF DCRs ingests twice.
Disk
A full /var/log stops AMA functioning entirely.
References
Verified vs Microsoft Learn
great