> For the complete documentation index, see [llms.txt](https://docs.layeronecloud.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layeronecloud.com/network-monitor/operations/aegis.md).
# LayerOne Aegis
Monitor Applications and Aegis Applications display Potential Vulnerability findings from NIST NVD, with severity, CVE details, original scan evidence and assessment age.
## Potential Vulnerabilities and customer scan requests
Monitor Applications and Aegis Applications display **Potential Vulnerability** findings from NIST NVD, with severity, CVE details, original scan evidence and assessment age. Counts represent service/CVE pairs, with ignored findings separate from active findings. They are not a claim of confirmed exploitability, complete software inventory or verified patch status. Unknown products, missing versions, lower-confidence probes and ambiguous package versions remain unable to assess. Platform-dependent applicability or bounded results are explicitly incomplete. Vendor backports can already fix an older advertised version.
The initial explicit mappings cover OpenSSH, Apache HTTP Server, nginx, MySQL, PostgreSQL, Redis, MongoDB, Memcached, vsftpd and ProFTPD. The software-only cache refreshes every 24 hours; failed lookups retain previous findings and timestamps. The supervised `refresh_vulnerabilities --loop` worker discovers current and retained scan evidence in bounded pages. It sends software CPE identifiers to the fixed NVD API, never host/customer addresses. A database lease, request pacing and exponential backoff are shared across replicas. `L1_NVD_API_KEY` is optional; `L1_VULNERABILITY_SYNC=0` is only for separately supervised workers. Page reads use the cache and do not make NVD calls.
Operations advertises `aegis_vulnerability_ingest` with `aegis_vulnerability_contract_version: 1`. Its separate `POST /api/admin/v1/aegis/vulnerabilities/` projection is bound to the current allocation, assignment, source, scan UUID and exact retained service evidence. Advisory updates are delivered even without a new scan; retries cannot replace newer checked evidence with older results. Customer ignores bind to the account, VM, allocation generation, port, software fingerprint and CVE. Monitor ignores remain independent. Reassignment and version changes do not inherit the previous customer's/version's ignore preference.
**Request port scan now** queues the current public IPs of an owned, running device (including attached floating IPs). It requires an authenticated, CSRF-protected POST, has a ten-minute per-device cooldown, and deduplicates pending clicks. Limits are 32 public IPs per device and 100 pending IP requests per account. Unstarted requests show expired after their 24-hour start window; a confirmed Running scan can complete later without expiring or allowing a duplicate request. Operations persists manual and automatic full jobs; Monitor claims their immutable IDs, while actual sensor spawn and heartbeats establish Running. A claim alone remains Queued and missing contact stays explicitly overdue, not a fabricated failure. Requests do not preempt active scans or bypass host-pressure, memory, CPU, concurrency or outbox limits.
The existing Aegis worker polls the authenticated `GET /api/admin/v1/aegis/scan-requests/` endpoint in bounded, fair pages. Fresh cached inventory remains the scan target authority. The sensor reads the optional `scan_request_id` from its plan, gives it priority over routine due work, and uses that UUID as the normal result's `scan_id`. Its durable ledger prevents retries from executing a request repeatedly. Older sensors continue routine scanning but cannot expedite these requests; upgrade to **0.1.20**.
Deploy Operations migration `network_alerts.0008` first, then Monitor `monitoring.0049` and `monitoring.0050`, and install the rebuilt sensors. Confirm the NVD refresh worker is healthy and that the existing Admin API project has the Aegis permission. No additional customer credential is required.
Customer Aegis Settings disables customer views and alerts and cancels queued manual requests, with a clear notice that network security monitoring and DDoS protection continue. It does not disable ingestion, scanning or mitigation. Re-enabling preserves saved preferences and retained observations, without resurrecting cancelled requests. In-flight scans may finish normally.
Deploy both applications and apply their migrations before enabling delivery. In Pulsar Operations, edit the **existing** Admin API project used for VM inventory and enable its DDoS incident ingestion permission (`ddos_incident_ingest_enabled`). Keep the current token: the same encrypted credential authenticates both workflows. The API discovery response advertises `ddos_incident_ingest` and `ddos_incident_contract_version: 1` when the project has permission. Confirm the site's cluster mappings in **Agents & Settings**; the receiver matches each exact target IP against assigned customer resources in those clusters. Interface-only and CIDR-wide observations are not expanded into customer notifications.
The web container supervises `sync_pulsar_ddos --loop` separately from telemetry ingestion and inventory, checking every 30 seconds by default. Set `L1_PULSAR_DDOS_SYNC_SECONDS` to change that interval (minimum five seconds), or `L1_PULSAR_DDOS_SYNC=0` to run the command in a separate worker. Delivery also requires the existing Pulsar Operations connection to be enabled. The sender uses a renewable database lease, so multiple application replicas are safe.
For an operator-triggered delivery and readable status, run:
```sh
python manage.py sync_pulsar_ddos
```
Each tick scans at most 100 pending or active incidents. A durable cursor makes progress across busy sites; acknowledged terminal history does not delay live updates. Unchanged snapshots are suppressed, failed batches retry on the next sweep, and restarting after a successful POST safely replays its incident IDs and original observation times. The API validates each complete batch, then acknowledges individual ownership outcomes. `unmatched` means no eligible customer resource matched that IP and cluster; `identity_conflict` and `terminal` indicate a previously bound incident could not be applied. These counts appear in command output and the durable `PulsarDdosSyncState.last_counts` record. Its `last_error`, `last_run_at`, and `last_success_at` fields distinguish delivery failure from an empty scan. A permission error means the existing API project needs DDoS incident ingestion enabled; it does not require another key.
Pulsar customers see their incident status and configure alerts under **Networking → LayerOne Aegis**. Only explicit sensor recovery can clear an incident as resolved. An observation that has aged beyond 90 seconds becomes stale, keeping the original sensor timestamp; delayed or replayed telemetry cannot turn missing visibility into a recovery notification. Existing historical resolutions without explicit confirmation are treated conservatively as stale until new sensor evidence arrives. Receiver ownership remains attached to the customer resource originally matched, so later IP reassignment does not transfer historical incidents to another customer.
## Customer traffic and applications
The same `sync_pulsar_ddos --loop` supervisor also pushes LayerOne Aegis minute traffic through the existing encrypted Admin API credential. Deploy both applications and apply the monitor's current migrations before starting the expanded worker. The existing Operations project's Aegis/DDoS ingestion permission advertises `aegis_telemetry_ingest` with `aegis_telemetry_contract_version: 1`. No customer browser connects to the monitor or receives this credential.
`GET /api/admin/v1/aegis/targets/?after_id=0&limit=50` returns active, authorized allocation generations: `id`, `billing_account_id`, `virtual_machine_id`, `cluster_id`, canonical `address`, and timezone-aware `assigned_at`, plus `next_after_id`. A filtered page can be empty while its cursor still advances. Only an explicitly mapped cluster may select a source site. The sender reads only globally routable unicast customer addresses. Private, CGNAT, loopback, link-local and other non-global targets are counted as unsupported and skipped: v1 carries no trusted tenant LAN segment to disambiguate overlapping addresses. The sender reads complete minute buckets beginning at or after the assignment boundary; an allocation attached halfway through a minute cannot receive that minute. The receiver checks the allocation generation again on every receipt.
`POST /api/admin/v1/aegis/telemetry/` carries `contract_version: 1`, `source_site_id`, `cluster_ids`, and `targets`. Each target contains `id`, `address`, the exact echoed `assigned_at`, and at most 60 `series` points. Each point contains an ISO-8601 `timestamp`, `inbound_bps`, `outbound_bps`, combined `pps`, `expected_bps`, `expected_pps`, `ddos_bps`, `ddos_pps`, numeric `coverage` from zero to one, and `applications`. Unavailable rates/coverage are JSON null. Applications contain `name`, `inbound_bytes`, and `outbound_bytes`. Receivers advertising `aegis_application_identity` with `aegis_application_identity_version: 1` also receive `transport` and `server_port` on each application. The service port is null when unobserved or the protocol has no parsed port; it never matches a known port. Each minute keeps at most 20 separate name/transport/port identities, prioritizing outbound evidence before byte totals. Excess identities are omitted without inventing an aggregate application; overall exact-IP traffic totals remain intact. Older receivers retain the original name-only format and top 19 names plus Other observed applications when more than 20 names occur. Receipts acknowledge each target once with `accepted`, `unchanged`, `unmatched`, or `source_conflict`.
Bandwidth and packet rate are **observed exact-IP traffic**, not a share of kernel interface counters. Host observations carry separate receive/transmit bytes but only combined packet counts; the integration does not estimate directional PPS from packet sizes. It keeps the strongest capture observation for each address/minute to avoid counting routed or mirrored traffic twice. This is conservative on multipath networks. Missing capture data remains a gap, and capture loss is not scaled away.
Minute and live exports join the selected host's canonical interface-total quality for the same agent, capture domain and observation window (including split live chunks). Raw Linux capture counters correct packet retention even when a legacy host row used the old formula. Known parser loss makes exported `coverage` null while preserving observed bytes. Minute exports also check the winning exact-target attack sources, including attack-only minutes. If no capture contains both directional maxima, coverage is also null: adding possible mirrors could overcount, while selecting one disjoint path could undercount. These checks do not identify every multipath loss. Missing canonical markers preserve legacy coverage, and a numeric one is not a certification of byte accuracy.
Application queries require the exact local endpoint and Internet direction: client IP for outbound connections, server IP for inbound hosted services. Mirrored capture observations are counted once per direction/minute. Shared MAC totals, site rankings, remote endpoint addresses, and observed domains never enter the customer payload. Aegis displays applications for 24 hours after the assigned local address last sent bytes on that application's transport and service port, including hosted-service replies. Its application reporting window is always the rolling 24 hours, independently of the traffic chart range. Inbound-only traffic and scans cannot establish or renew an application, and activity on another address or port cannot extend its lifetime. The sender preserves raw minute evidence and original observation timestamps across delivery batches, so delayed deliveries do not restart the lifetime. Receive-only minutes remain available for applications with qualifying outbound evidence. Only attributed applications are reported; absent pair data, older sensors, and bounded collection can leave application rows incomplete even when capture coverage is one. Exact endpoint detail is retained for 24 hours, and both source reads and receiver history use that boundary.
Suspected attack traffic uses only exact-host incident metrics. Correlated sensor copies are deduplicated; capture-wide attacks and `other` traffic are never apportioned among customers. A complete detector reporting marker plus an enabled historical target epoch covering that exact IP for the entire minute can establish zero attributed attack traffic. Partial or missing reports, unprotected addresses, and legacy epochs with unknown targets cannot. Customer minutes overlapping unassignable capture-wide or overflow attack observations also remain unknown rather than receiving a share of that attack or a false zero. Target history starts on the first fresh inventory after the upgrade; current target settings never rewrite old protection history. Expected traffic is the mean of at least five of that allocation's preceding healthy, attack-free observed minutes within a 30-minute window: combined inbound/outbound bandwidth and combined PPS. This displayed expectation is not the sensor's adaptive detection threshold. It stays null while learning and never trains on a previous owner's traffic.
Telemetry uses its own renewable database lease and fair allocation cursor in `PulsarAegisSyncState`. The sender handles up to 50 allocations per tick, one bounded request per allocation, independently of incident delivery. Every visit includes the five most recent complete minutes and up to 55 historical minutes; acknowledged data or a verified empty scan advances that allocation's `PulsarAegisDelivery` cursor. The 24-hour history is swept repeatedly so delayed sensor uploads are recovered, while current samples continue arriving. Failed posts preserve the cursor for retry and cannot starve other allocations. Inactive cursor rows expire after two days. Inspect `last_run_at`, `last_success_at`, `last_error`, and `last_counts` on the telemetry sync state to distinguish an empty or unmapped scan from a delivery failure.
## Live customer usage
Operations advertises `aegis_live_ingest` and `aegis_live_contract_version: 1` under the same Aegis Admin API project permission. The web container separately supervises `sync_pulsar_aegis_live --loop` every five seconds. Set `L1_PULSAR_AEGIS_LIVE_SYNC=0` only when that command runs in a separate service. Its independent `PulsarAegisLiveSyncState` lease, cursor, timestamps, error and counts distinguish delivery health from the historical or inventory feeds. `python manage.py sync_pulsar_aegis_live` runs one delivery tick.
**Agents & Settings → Pulsar Operations & Aegis** shows separate live-graph and minute-history export status beside inventory health. The cards refresh every 15 seconds without changing unsaved credentials or site mappings. They report the last worker attempt, last confirmed traffic receipt, latest acknowledged observation and the latest bounded-page counts. Empty or unmapped passes do not establish successful traffic delivery. Replayed receipts preserve the original observation time, so an old sample cannot make a feed current. Missing observations, never-run or overdue workers, stale traffic, API failures, mapping problems, unmatched assignments and source conflicts are visible. Read-only operators can inspect this status; the authenticated, no-store `GET /api/integrations/pulsar/health` only reads persisted state and never runs an export. Errors use fixed safe descriptions rather than raw response bodies.
`POST /api/admin/v1/aegis/live/` uses the same top-level version, source site, cluster and allocation-generation fields as minute telemetry. Each target has at most 30 completed UTC ten-second buckets from the last five minutes. Points contain `timestamp`, the existing nullable rate fields, `coverage`, and `observed_seconds` (greater than zero, at most ten); they have no `applications` field. Live inbound/outbound bit rates and combined PPS come directly from exact-IP host deltas in the private Redis live stream, divided by each sensor's actual measured interval. Overlapping capture mirrors are deduplicated before rates are averaged by observed duration. An original sensor interval crossing IP reassignment is discarded in full. Partial observation duration is explicit; windows longer than 60 seconds are excluded from this short live view. missing or omitted host data is never zero-filled or replaced with shared interface counters or historical minute rates. This live contract currently exports observed traffic only: its expected-baseline and DDoS rate fields are always unknown, including when the minute feed has learned an expectation. Historical ranges retain their expected/DDoS series. Applications continue through exact-endpoint minute telemetry and are not represented as live values. Up to ten targets at a mapped site share each POST; a fair 50-target cursor advances per tick. Site observations are reread before a later batch when the previous read is at least five seconds old, preserving original sensor times.
Source reads remain bounded by the Redis history length, decoded-byte budget and 100,000-host-window limit. A busy five-minute history can exceed those limits even while live ingestion is healthy (for example, four sensors with 1,000 host rows in each of 30 ten-second windows). Reaching a limit retains the newest validated measured windows, increments `source_limited`, and marks exported coverage unknown. It no longer discards every valid sample because older history exceeded the budget. Missing hosts are not filled with zeros, and a limit reached before any valid observation still reports a failure. Apply this fix to the Monitor server; it does not require a sensor update or database migration.
Deploy both applications and their migrations, then update the installed Proxmox agents to **0.1.18** for the independent live allowance of up to 1,000 exact host rows per capture. Web deployment publishes the binary but does not install it on nodes. Older sensors can supply observed host rows but those rows compete with the top 100 application/port dimensions, leaving avoidable gaps. Even the new bounded allowance cannot establish zero for an omitted host. The live feed requires healthy Redis publication; the minute exporter continues independently when disposable live history is unavailable.
## Shared Devices and Aegis WAN counters
The Devices list's WAN bytes and device-detail WAN graph now use the same `wan_counters` exact-IP selection as Aegis minute telemetry: one strongest capture per IP/minute, never sums of capture mirrors or a share of interface bytes. Multiple IPs belonging to a device are summed only after this selection. The Aegis live feed shares the source-selection and quality rules, but retains its own measured intervals; it is not the durable accounting ledger.
The Devices list presents WAN and LAN separately. The detail page's primary graph is WAN, derived from the same raw directional byte totals as its WAN usage card. LAN retains a separate graph and is not added into WAN. Interface and site-wide diagnostic graphs retain their explicitly different scopes. Public-IP comparisons match when the address set, ownership and time range match; Aegis still enforces allocation generations, whereas Devices shows the observed device across its retained history. Private operator IPs are isolated by segment and remain unsupported by the public-IP Aegis contract.
New sensors attach `MetricPoint.host_device_key` (tag 15) to exact-IP host counters. It records the visible local `segment|MAC` for that observation window. Unknown owners, conflicting owners within a window, and sensor/router MACs cannot be used to attach usage to a device. Additive ingestion clears the key whenever contributions disagree or include an unknown owner; retries do not double-count. Canonical capture selection occurs before device filtering, so an owned but smaller mirror cannot replace Aegis's chosen bytes. Conflicting capture owner claims are also excluded from device attribution.
Deploy the server and migration `0056_minutemetric_host_device_key`, then install the rebuilt sensors. The migration adds a partial index for owned host WAN rows; plan the schema/index work for the database's normal maintenance procedure. There is no ownership backfill: old per-IP rows have a blank key, and current IP bindings or billing inventory cannot safely recover historical ownership. Until new owned counters arrive, device WAN values are **Unavailable**. The old per-MAC WAN totals are never substituted. Observed partial byte totals remain visible with a partial notice; missing bytes are not estimated. WAN graph hours with missing candidate-IP minutes remain gaps, not zero-filled or interpolated. Packet/parser quality remains separate from time completeness.
Device reads are bounded and month-long history is read in daily windows. Exceeding the address, segment or row budget reports `source_limited` rather than returning a silently incomplete total. This consistency change does not establish 99% accuracy or recover traffic lost during capture downtime.
## Billing measurement target
The target is at most 1% absolute byte error against an independent reference for each billed device/address and direction over the agreed accounting window. This is a validation target, not a measured guarantee from packet coverage or unit tests. No billing certification or automatic invoice policy is enabled.
For GB/TB billing, reconcile durable minute byte deltas, not live chart averages, application labels, or averages of coverage percentages. The current Aegis minute contract exports rates derived from byte totals over 60 seconds; the live contract averages only observed duration and must not be integrated as if every ten-second bucket were complete. Retain the original byte counters and allocation generation in any future billing ledger. Aegis history delivery is a rolling monitoring feed, not that ledger.
Choose one verified Internet measurement boundary per customer, or explicitly non-overlapping boundaries for multipath. Match RX/TX orientation and the byte definition to the independent reference: captured original frame lengths are not application payload bytes, and headers/offloads can differ from another counter's definition. A VM tap total includes LAN traffic unless separately classified. Unsampled per-customer counters at the chosen WAN boundary are a stronger billing foundation than application parsing alone.
Do not divide measured bytes by packet coverage to invent billable usage, or turn missing/uncertain minutes into zero. Reconcile gaps, parser/capture loss, counter resets, assignment changes and conflicting observations before using a period for billing. Current captures cannot recover bytes sent while the agent was not observing, and strongest-capture deduplication can omit disjoint paths. Run the [per-device billing validation gate](https://github.com/LayerOne-LLC/Network-Monitor/blob/main/tests/performance/README.md#per-device-billing-validation) on representative hardware and topology before relying on a 99% claim.
## Continuous open-port scanning
Each upgraded hypervisor scans only running VMs assigned to its own node in the already-synced **Devices** inventory. VM and NIC records must be current within 15 minutes, and the sensor's interface inventory within five minutes. Stale, retired, ambiguous or foreign-node devices are excluded. Passive traffic peers and the hypervisor's own addresses are not targets. Building the scan plan makes no additional Pulsar requests.
The scanner binds both its source address and interface to the WAN's IP owner. For a WAN captured on `nic2` whose address is on `vmbr0`, scans use `vmbr0`, not `nic2`. The agent checks that the address remains assigned locally and that the interface belongs to its configured WAN topology before starting or continuing a scan. IPv4 and IPv6 targets require a source of the same family.
Sensor **0.1.23** uses three independent pools: two hourly top-100 TCP/UDP scans, eight daily full scans of ports 1–65535, and two customer-requested full scans. Up to 12 different target IPs can run concurrently. Common scans become eligible one hour after the preceding common attempt; full scans become due once per scheduled local day. Customers take priority for their IP, and a target never has overlapping scans. Due common-port work precedes that IP's daily full scan; after a common attempt for that daily cycle is acknowledged, an owed daily scan takes precedence when a daily slot is available. Overdue common scans remain eligible while all daily slots are occupied. Customer slots do not preempt an existing scan or pending result for the same IP. The daily time releases work; it does not impose a finish deadline. There is no per-host or per-phase runtime deadline.
**Agents & Settings → Host port scans → Scan schedule** configures the site's full-scan time and IANA time zone, defaulting to 22:00 America/New\_York (local Eastern time, including daylight saving changes). The authenticated plan adds optional `scan_policy: {version: 1, daily_full_scan_at: }` with the latest scheduled boundary. The sensor catches up once for the latest boundary after downtime, retaining its daily/common schedule and immutable customer UUIDs in SQLite. A time-zone or schedule change takes effect on the next fresh plan. New sensors receiving an old plan retain the old two-slot full-hourly behavior. Old sensors ignore the policy extension until updated. Discovery and service identification are separate phases; service identification probes only positively open ports. Agent **0.1.20** uses `-sV --version-all` (intensity 9) instead of the previous light probe set to identify software products and versions, including services on nonstandard ports. This applies to both routine and customer-requested scans. Nmap's default safety exclusions remain in place, including its printer-port exclusion; no `--allports`, OS detection, or general vulnerability/brute-force scripts are enabled. Product, version, protocol, probe method and confidence are retained in Applications and the NVD matching pipeline. Hidden or unrecognized versions remain unknown, not evidence that an application is current or vulnerability-free. More thorough version detection can take longer and generate more service traffic. The discovery packet-rate ceiling does not cap all version-probe traffic; scanner cgroup limits, concurrency and host-pressure checks still apply. See [Nmap's service/version detection options](https://nmap.org/book/man-version-detection.html). A failed fingerprint phase does not erase ports already found open. Neither `filtered` nor `open|filtered` is reported as open. Each discovery process has an average rate ceiling of 8,000 packets per second (96,000 across twelve processes); Nmap can adapt its rate downward and brief bursts can exceed the average. All scan profiles and phases use a tuned aggressive timing policy: `-T4`, 50 ms minimum probe timeout, 100 ms initial timeout, 500 ms maximum timeout, one retry, and at most 256 outstanding probes per child. The initial wait and parallelism ceiling speed up local guest discovery; the 500 ms upper bound leaves room for slower replies from busy guests. These are adaptive limits, with no forced minimum rate or parallelism. TCP discovery also uses `--defeat-rst-ratelimit`: it does not wait for throttled reset replies merely to distinguish closed from filtered ports, since only confirmed open ports are reported. UDP retains `--defeat-icmp-ratelimit`; silent UDP services can still be missed. Full `--version-all` identification remains enabled and its service probes have their own response waits. The timing policy has no host or phase deadline. A bare `-T5` would introduce a 15-minute host cutoff, so it is not used. The existing packet-rate, cgroup, concurrency and host-pressure controls continue to apply. Install the rebuilt hypervisor sensors to use this policy; a control-plane deployment alone does not change running Nmap commands. See [Nmap's timing controls](https://nmap.org/book/man-performance.html). Dense nodes or restrictive guest firewalls can extend the time needed to finish a sweep. Full requested coverage is not a guarantee that every port provides a definite answer. The scanner reports queued work, coverage and actual failures. Pressure interruptions, authorization changes, process failures or inconclusive replies can still leave partial results.
The agent installer/update path installs Nmap and the separate `l1-monitor-port-scan.service`. Both services use `CAP_NET_RAW`; no firewall or route changes are made. Plans and results use the host-authenticated `/agent/v1/port-scans` endpoint. A separate `port-scans.sqlite3` schedule/outbox survives restarts and keeps retries idempotent. Pending JSON payloads are bounded to 128 MiB; this is not a filesystem quota for SQLite pages, indexes and WAL. The SQLite main database has a 160 MiB page limit, with automatic/minute WAL checkpoints and a 4 MiB retained-WAL limit (temporary WAL growth can exceed that retained limit). Withdrawn schedule entries expire after seven days; the table is capped at 8,192 records. Storage pressure pauses new collection without deleting pending results. Idle due/outbox polling is limited to every five seconds, with an in-memory queue supplying newly freed scan slots. The ordinary telemetry clear-backlog command does not clear this separate scan outbox. Plans and open observations are bounded to 4096 entries each; truncation or interruptions mean partial visibility, not closed ports.
Agent **0.1.21** fixes scan attempts ending with “Scan authorization unavailable; waiting for a fresh plan” after a temporary plan-fetch failure. Established sensor plan GETs use read-only authentication so telemetry database locks do not delay authorization refresh. The sensor fetches one plan at a time on a background thread, with a 15-second timeout, while continuing local safety checks. It pauses Nmap during refresh and retries temporary failures after five seconds, retaining the same process, scan UUID, progress and durable reservation. Only a validated fresh plan that still includes the destination, with valid local WAN ownership and healthy resources, permits resuming. Explicit 401/403/404/410 responses, target withdrawal, shutdown and local safety failures still terminate the attempt. No probes continue on stale authorization.
Deploy the Monitor server change and update the installed hypervisor sensors to **0.1.21**; publishing the web image alone does not replace installed sensors. Previously recorded failures remain in history. Request a new scan after the existing customer cooldown, or allow the normal schedule to retry them.
Agent **0.1.22** also allows completed scan uploads to wait up to 45 seconds for their durable receipt, matching telemetry delivery's timeout. Uploads remain independent of plan refresh and child supervision, and retries preserve the exact scan UUID and observation timestamps. The former three-second timeout could keep completed observations queued behind database contention. Once an attempted customer request has a delivered result, a stale copy of its UUID in a later plan no longer blocks the host's routine hourly refresh. The routine attempt uses a new UUID; the original manual request remains deduplicated, and pending uploads still prevent overlapping scans for that address. The Monitor server also retains each manual request's original result and source independently of the latest per-IP observation. A newer hourly scan cannot overwrite an undelivered manual outcome. Delivery retries the immutable UUID until Operations acknowledges that request under the current confirmed connection revision. Older per-IP receipts do not suppress this delivery. Late results resolve the matching intent without replacing newer port evidence. Upgrade recovery uses an existing exact-UUID report when available; overwritten results cannot be reconstructed. A changed connection revision still requires fresh confirmation of the same request, preserving the destination trust boundary.
Deploy Operations' receiver fix first, then Monitor with migration `monitoring.0052_durable_scan_request_results`, and update installed hypervisor sensors to **0.1.22**. Publishing the web image alone does not update the running scanner. The receiver accepts recently completed long-running scans, including starts more than seven days ago, while retaining assignment and request-window checks. Old completed observations remain outside retention.
### Per-host resource profile
The scanner service delegates a cgroup v2 subtree to its existing unprivileged `l1-monitor` user. Before any Nmap execution, the worker creates a separate `scan-` leaf for that target and moves the child into it. TCP, UDP and service-identification phases share the same leaf. The supervisor occupies its own bounded leaf. The ordinary `l1-monitor-agent.service` keeps collecting packets and telemetry separately. Each host leaf enforces:
| Resource | Scanner-only limit |
| ------------------ | ---------------------------------------------------------------------------------------------------- |
| CPU time | One logical CPU per host, `cpu.max=100000 100000`, with no accumulated burst allowance |
| Memory | Reclaim/throttle above 384 MiB, hard limit 512 MiB, no swap |
| Scheduling | Nice 15, CPU weight 10, idle I/O priority and I/O weight 10 |
| Concurrent scans | Eight daily full, two customer full, two hourly common; discovery ceiling 8,000 packets/sec per host |
| Processes/threads | At most 64 tasks per host leaf |
| Scan-output buffer | At most 8 MiB per Nmap child |
There is no aggregate one-CPU / 512 MiB cap over all hosts. Twelve active hosts can receive up to twelve logical CPUs and 6 GiB of scan memory, plus the supervisor's separate one-CPU / 512 MiB budget. These are ceilings, not reserved resources or expected use. Full use of all pools permits a 96,000 packets/sec aggregate discovery ceiling; service-identification traffic is separate.
The updated unit requires cgroup v2, systemd 251+ delegation, and the cpu, memory, pids and io controllers. `ProtectControlGroups=no` and a writable cgroup mount allow only the DAC-owned delegated subtree to be managed; no root helper, sudo/polkit rule, CAP\_SYS\_ADMIN or arbitrary command runner is introduced. The worker verifies all limits before probes and rechecks them while running. Missing delegation or legacy aggregate CPU/memory drop-ins cause an explicit failure rather than a fallback to unbounded scanning. The unit uses `OOMPolicy=continue` and per-leaf `memory.oom.group=1` so a host's OOM does not kill other scans. Service stop still kills the entire scanner process tree. Host leaves are released after completion; result delivery keeps its existing independent durable outbox. Configuration is read from the normal agent's refreshed local cache, and worker health is returned through a bounded local status file; these do not add remote inventory or configuration requests. A stale or invalid cache stops scanning.
Host pressure is sampled every five seconds. New scans are deferred and active probes are stopped if aggregate CPU utilization reaches 85%, available memory falls below the larger of 1 GiB or 10% of RAM, or less than 1 GiB remains on the state filesystem. When Linux PSI is available, CPU `some avg10` of at least 10% or memory `full avg10` of at least 1% also suspends work. CPU and available-memory measurements must be readable. The scanner requires 30 seconds of healthy samples before resuming and keeps deferred/partial coverage visible in agent health. Sampling gaps over ten seconds reset recovery. Previous positive observations retain their original timestamps.
These limits do not guarantee that every scheduled scan finishes before its next interval. With one target IP per VM, 200 full scans require 26,214,000 initial TCP/UDP port probes per full sweep, before retries and identification. Additional IPv4/IPv6 addresses increase that work. Common scans request 100 ports per protocol; a full scan requests 65,535 per protocol. Top-100 completion never claims full coverage or retires unscanned port observations. Common evidence is stale after two hours and full-scan evidence after 27 hours, measured from each original observation. The latest full result is retained independently of newer common results, with separate per-UUID Operations delivery receipts. Manual results also retain their own immutable request snapshot. Host pressure can still interrupt work and leave partial scans. The discovery rate limit does not cap every service-identification packet or network-stack/guest processing cost. CPU and RAM controls constrain the scanner service, not all indirect host or guest overhead.
Before fleet rollout, follow the [scanner canary check](https://github.com/LayerOne-LLC/Network-Monitor/blob/main/tests/performance/README.md#continuous-port-scanner-canary). Inspect the worker independently from traffic capture:
```sh
systemctl status l1-monitor-port-scan.service
systemctl show l1-monitor-port-scan.service -p ControlGroup -p Delegate -p MemoryCurrent -p TasksCurrent
cat /var/lib/l1-monitor/port-scan-status.json
journalctl -u l1-monitor-port-scan.service --since '1 hour ago'
```
Deploy Operations `network_alerts.0010` first, then Monitor `monitoring.0053`, then install sensor **0.1.23** and the updated scanner unit on a single canary hypervisor. Publishing either web application does not update installed sensors. Verify the supervisor and each active `scan-` leaf's actual controllers, the 8/2/2 pool counts and the site's next scheduled release before wider rollout.
For Operations-owned queue and actual lifecycle tracking, apply Operations `network_alerts.0011`/`0012` and Monitor `monitoring.0054`/`0055`, then install sensor **0.1.24**. The existing schedule now enqueues daily full attempts in Operations before dispatch. Older sensors ignore the managed-queue fields and cannot acknowledge starts; upgrade every participating scanner. Finished sweeps with filtered/silent ports are Complete with an uncertainty note; interrupted or incomplete checks stay Partial. Unknown ports are never claimed closed. Historical Partial rows are not automatically reclassified. See [the scan lifecycle contract](/network-monitor/scan-lifecycle.md) for claim ownership, durable start delivery, terminal races, and canary acceptance.
**Applications** groups open ports with matching observed applications and shows scan-only services in the separate scan inventory. A scan result alone does not create an active application. Device pages restrict results to the current canonical NIC/IP and hypervisor owner. Probed service/product/version labels are distinguished from port-number hints. Coverage and freshness are separate from traffic totals: scans do not invent application bandwidth. Partial or failed attempts retain earlier positive observations with their original times; a complete scan may replace them. Observations become stale after two hours and expire after seven days. `maintain_metrics` also expires scan and delivery rows.
Customer delivery reuses the discovery response, allocation page and renewable lease from `sync_pulsar_ddos`'s existing Aegis telemetry pass; it does not fetch another inventory or allocation page. Only exact public allocations cross this v1 customer boundary because overlapping private IPs lack a tenant-segment key. Operations advertises `aegis_port_scan_ingest` and version 1 under the existing Aegis project permission. The sender posts to `/api/admin/v1/aegis/port-scans/`, preserves sensor timestamps and suppresses acknowledged duplicates in `PulsarPortScanDelivery`. Inspect `PulsarPortScanSyncState` for delivery errors and counts. Operations rechecks current account/VM/allocation ownership before accepting or showing results, including on its administrator Applications page.
Deploy Operations and apply its scan migrations first, then deploy the monitor with `0048_host_port_scans`, then upgrade the installed hypervisor agents. A web deployment alone does not upgrade the agents. Older agents continue ordinary telemetry without scan observations.
To remove the scan runtime cutoffs, deploy the updated monitor control plane first, then upgrade the Proxmox agents to **0.1.19**. Both updates are required so scans can keep running and the control plane accepts their longer durations.
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.layeronecloud.com/network-monitor/operations/aegis.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.