> For the complete documentation index, see [llms.txt](https://docs.layeronecloud.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.layeronecloud.com/platform/vps/networking.md).
# IPAM, firewall and private networking
the applied state, and admins need to see when the two diverge.
## 5.8 IPAM, floating IPs, additional IPs and reverse DNS
**IPAM.** `IPPool` → `IPAddress` → `IPAllocation`, plus ipsets and filter rules. **The local database is the intended state; Proxmox firewall/ipfilter state is the applied state**, and admins need to see when the two diverge. Duplicate IP assignment is prevented by validation and a per-pool uniqueness constraint.
The console paginates pools, addresses and allocations independently, with address/customer search and pool/status/allocation-state filters. **Bulk address ranges**: the Add address screen accepts one address or an inclusive range such as `207.174.22.25-50` (IPv4 may abbreviate the final endpoint to its last octet; complete IPv4/IPv6 endpoints are accepted for ranges crossing a larger boundary), expanding to at most **8,192** addresses per submission. The selected pool remains the source of network and gateway rules, every requested address must belong to it before any row is written, and network/broadcast/gateway addresses are counted and skipped. Creation is additive and safe to retry: existing rows keep their status, allocation history, reverse DNS, notes and metadata.
**Floating IPs.** Reservation immediately creates an allocation and starts **hourly usage** — reserved IPs are billed while reserved, even when not assigned to a server. Assignment attaches the allocation and syncs provider state (including the anti-spoofing ipset). Release detaches provider state, stops billing for that allocation and frees the address when cleanup completes. **Release must succeed even when no server is assigned**, and assignment/release are idempotent enough for retries.
**Additional IPs.** The operator-granted counterpart, on the server's Network tab in the console (`apps/billing/additional_ips.py`). A floating IP is the customer's own product: ordered by them, billed hourly, owned by the *account*, moved between servers at will. That is the wrong shape for "give this customer a second address for their mail host, on us", so an additional IP inverts every one of those:
* **Granted, never ordered.** `@admin_required` on both routes, and `grant_additional_ip` / `revoke_additional_ip` re-check `is_admin`, so the service layer refuses a customer even if a route is ever exposed. There is no client-facing create or release path; the client sees a read-only panel.
* **Free.** Nothing writes a usage ledger entry, and `_aggregate_floating_ip_usage` filters on `purpose=FLOATING`, so the hourly sweep cannot meter one by accident. This is why it cannot reuse `order_floating_ip`, which starts billing in the same transaction that reserves the address.
* **Bound to the server.** The allocation's own `resource_type`/`resource_id` name the guest, not a metadata pointer, so it cannot be repointed. Moving one is a revoke and a new grant.
* **Same cluster, public pools only.** `grantable_ip_addresses_for_vm` restricts the choice to the guest's own cluster: an address routed elsewhere passes the ipfilter write and then simply never reaches the guest, which looks like a working grant until the customer tries to use it.
They are `IPAllocation` rows with `purpose=SECONDARY`, which is what buys the rest for free: one active allocation per address is already a database constraint, and reverse DNS already follows any active allocation, so a granted address is PTR-editable by its holder with no new code (the client list labels it ` (additional IP)`, because the server name alone no longer identifies a row).
**One ipset, two kinds of extra address.** Floating and additional IPs are the same thing at the hypervisor — one more `/32` the guest may source from — so they share `ipfilter-net0` and share the machinery: `client_services.sync_ip_filter_for_allocation` (state machine, attempts, `IP_FILTER_SYNC_MAX_ATTEMPTS`), `retry_pending_ip_filter_syncs` (the `ip_filter_sync` maintenance step, which sweeps both), and `proxmox_adapter.sync_guest_ip_filter`. An `_IPFilterSyncKind` holds the only differences: the metadata key (`floating_ip_sync` vs `additional_ip_sync`), the event prefix, and where the target server is recorded. Three consequences are load-bearing:
1. `_extra_ip_filter_cidrs_for_vm` must report **both** kinds. `_reconcile_ipfilter_members` deletes every member that set does not name, so an additional IP left out of it is stripped by the next reinstall or resize — silently, long after the grant.
2. A revoked address stays `RESERVED`, not `AVAILABLE`, until the removal confirms. `ip_address_ids_with_pending_filter_cleanup` is what the IPAM console and **Make available** consult, and it covers both kinds: otherwise an address could be handed to the next customer while the previous guest can still source from it.
3. Destroy and local-record-delete release the *primary* allocation by purpose, so both call `release_additional_ips_for_vm` as well. Without it a granted address keeps an active allocation pointing at a server that no longer exists — it never returns to the pool and its PTR keeps naming the departed customer.
Ownership resolution follows the same rule. `network_alerts.ownership` and the admin inventory API resolve a public address to its guest through metadata for a floating IP and through the resource columns for an additional one; an additional IP missing from either is an attacked or abusing address that resolves to nobody, and `resolve_target_owners` fails closed.
Not automated: nothing writes the address into the guest. Proxmox `ipconfig0` carries one address, so the customer configures the extra one inside the OS and the platform only permits it at the firewall — the same deal as a floating IP, and both panels say so.
**Rotating a primary address.** `apps/billing/ip_rotation.py`, on the same Network tab, `@admin_required` with `rotate_primary_ip` re-checking `is_admin`. Support gets an address off a blocklist or out of an attack without rebuilding the server, which is what its absence forced. It is *not* a grant: the primary is the address the platform itself configures, so a rotation writes the guest and takes the old address away, and the panel leads with that rather than with the new address.
* **The guest is rewritten, the NIC is not.** `proxmox_adapter.apply_primary_ip_to_guest` PUTs `ipconfig0` only. Rewriting `net0` without the MAC makes Proxmox regenerate it (see `hotplug_private_nic`), so `rotatable_ip_addresses_for_vm` restricts candidates to pools resolving to the bridge the guest already uses — the cluster and public-pool rule of a grant, plus that.
* **Cloud-Init only reads `ipconfig0` at boot.** The customer is unreachable between the rotation and the restart, so the form offers the restart in the same submit and says so either way. A restart that will not queue (a stopped or busy server) is a warning, not a failed rotation: the address lands at the next boot regardless.
* **The old address stays `RESERVED`,** exactly as a revoked additional IP does, and for the same reason — the guest can still source from what is still in its ipset. `schedule_reverse_dns_release` runs on the old address, because a rotation ends the customer's hold on it.
* **`IPAllocationPurpose.PRIMARY` is registered in `_IP_FILTER_SYNC_KINDS`** (key `primary_ip_sync`), so rotation reuses the state machine, the retry budget, the `ip_filter_sync` sweep and the **Make available** check rather than growing a third copy. That is only safe because `allocation_filter_cleanup_is_pending` treats a released allocation with **no sync record** as owing nothing: destroy and local-delete free the address outright — the guest holding the ipset went with it — and so does every primary released before rotation existed. Without that rule, registering the purpose would strand every address a destroyed server ever held.
**Reverse DNS.** PTR records for LayerOne's own address space, published to Cloudflare, editable by the customer who holds the address.
PTR authority follows the IP allocation, not the domain, and **reverse zones are per `/24`** (Cloudflare supports `/24` and `/16` for IPv4 and nibble boundaries for IPv6; reverse delegation itself only cuts on those boundaries). A `/22` is four zones, not one. `IPPool.network_cidr` is arbitrary, so the pool is the wrong key for a zone lookup — hence `ReverseDnsZone`, keyed on the prefix and resolved by **longest prefix match** (`reverse_dns.zone_for_address`).
* `ReverseDnsConfig` — one row, Fernet-encrypted Cloudflare token, TTL, two switches. `is_enabled` is the transport (off = nothing is published or removed by anyone, staff included); `client_editing_enabled` is only the customer-facing switch, so support can keep publishing while the client page is closed.
* `ReverseDnsZone.zone_name` is **derived** from the prefix in `clean()`, never typed: a hand-entered name that disagrees with the network writes PTRs into a zone that resolves for nobody and looks published from our side. Non-boundary prefixes are a form error.
* `IPAddress.reverse_dns` means **what is published**. It is no longer editable on `IPAddressForm`: a free-text field that writes the column without calling the API turns it into "what somebody typed once", and it is the value the release sweep and the console read to decide what still needs doing. **A failed sync deliberately does not update it.**
Two rules:
1. **A PTR is only as trustworthy as whoever set it.** A customer can name any hostname, and an unchecked PTR is how a spammer borrows someone else's reputation from one of our IPs. Customer-submitted hostnames must **forward-confirm** (their A/AAAA resolves back to the IP) before publishing. Staff are not held to that check, because the cases support gets called about are exactly the ones where the forward record is not in place yet. The boundary is *who is asking*, so `set_reverse_dns` takes `enforce_forward_confirmation` as a **required keyword** — a new client-facing entry point cannot get the safe behaviour by forgetting, and there is no setting that turns the check off.
2. **A PTR outlives the allocation unless something deletes it.** Reverse DNS lives on the address, so an address handed to the next customer keeps naming the previous one. Every path that ends an allocation calls `schedule_reverse_dns_release` (`client_services.release_floating_ip`, `proxmox_adapter._release_primary_ip`, `views/common._release_vm_primary_ip_for_local_delete`). It runs on `transaction.on_commit`, not inline: release holds row locks inside `transaction.atomic` and a Cloudflare round trip there holds them open for a network call, and deferring also means a rolled-back release never deletes a live record. The "is anyone still on this address" check re-runs at commit time, because that is exactly the state the enclosing transaction is changing. The clear is **best effort and never raises** — a customer must be able to hand back an address while Cloudflare is unreachable — and `billing.reconcile_reverse_dns` (hourly) is the safety net, finding addresses with a hostname and no active allocation. **Both halves are needed.**
Console: Reverse DNS is a **tab under IP addresses**, not a Settings page, because the zone a PTR lands in is a property of the prefix and the prefix is on that page. Each tab keeps its own url name. **No reverse-DNS route is on `SUPPORT_CONSOLE_ROUTES`.** The per-zone **Test** button reads the zone back and compares the name Cloudflare reports against the derived one — the check that catches a zone ID pasted from the wrong zone, i.e. credentials that work perfectly while writing PTRs nobody can resolve.
Operational setup per `/24`: create the zone in Cloudflare named `...in-addr.arpa` (free plan under 200 PTR records, so a fully populated `/24` needs a paid plan), add it here with its zone ID and press Test, then **set the two Cloudflare nameservers on the reverse delegation at ARIN** — nothing resolves until this is done and nothing in the app can tell that it has not been. The API token is separate from the email one and needs only `Zone.DNS:Edit`.
Not done: no default PTR is generated at allocation time (worth revisiting for mail deliverability, at the cost of pushing every `/24` onto a paid plan immediately), and IPv6 works by the same code path but has not been exercised against a real `ip6.arpa` delegation.
## 5.9 Hypervisor firewall and firewall groups
**The per-VM hypervisor firewall** is editable from the instance Network tab (`/client/services//network/` and the operator copy). The database is intended state (`VirtualMachineFirewall` + ordered `VirtualMachineFirewallRule`); Proxmox `/firewall/options` and `/firewall/rules` are applied state. The client API exposes the same service layer.
**`ipfilter-net0` is not this feature.** Provisioning and floating-IP sync write that ipset so a guest cannot spoof a source address it was not assigned. The customer path **never** lists, creates, edits or deletes `/firewall/ipset*`, **never** sets `ipfilter=0` or `enable=0`, and **never** accepts `+ipfilter-net0` (or any ipset name) as a rule source or destination. **There is no "turn the firewall off" control, because that would disable anti-spoofing.**
Inbound and outbound default to ACCEPT. Switching inbound to DROP without an enabled inbound ACCEPT covering TCP 22 requires `confirm_ssh_lockout`; the browser console remains out-of-band recovery.
**Provision and reinstall synchronization.** Both workflows explicitly send `policy_in=ACCEPT` and `policy_out=ACCEPT` when enabling the VM firewall. Previously they sent only `enable=1` and `ipfilter=1`, and **Proxmox's unset inbound policy defaults to DROP** while the portal displayed Allow from the database defaults — and fresh VMs skipped the customer-policy overlay because no firewall row existed yet. Before boot, `configure-network` now creates the default policy row if needed and applies the intended policies and the complete customer rule set, replacing inherited template rules while preserving saved customer policies on reinstall. The overlay uses the provisioning runner's client and logged action, so an explicit live run cannot silently fall back to the interactive save's dry-run setting. A client-visible placeholder may still lack node/VMID when this runs; the overlay copies that identity from the live plan (or the succeeded select-resources summary on retry) before applying, so it does not skip a guest whose clone already ran. Apply failures are persisted on the firewall and **fail the network step before `start-vm`**; retry reapplies intended state.
**Port rules require a protocol on the hypervisor.** The portal stores protocol Any (blank) with a destination port as a TCP/UDP match. Proxmox's `/firewall/rules` schema requires `proto` whenever `dport` is set; posting `dport` alone is HTTP 400 (`'dport' requires this property`) and used to fail reinstall `configure-network`. Apply therefore posts two rules (TCP then UDP) for that intended row. ICMP never carries a port.
**Firewall groups** live in **Networking → Security**, below Aegis, and share reusable rules across account-owned VPS devices. `FirewallGroup` holds an account-owned name, description and inbound/outbound default policies; `FirewallGroupRule` holds ordered rules using the existing validation; `FirewallGroupMembership` assigns groups to VMs. A device can have **eight groups, 40 rules per group**.
Composition: groups compose in **ascending creation ID**, rules keep their position order, first match wins, and unmatched traffic uses DROP if any assigned group sets that direction to DROP, otherwise ACCEPT. **Renaming a group or attaching it in a different order never changes precedence.**
The individual `VirtualMachineFirewall` policy and rules **remain saved** and are excluded from effective policy while any group is assigned; removing the last assignment restores them. The per-device page shows effective policies, rule provenance and links to assigned groups, **without mutation forms** — and the shared service layer also blocks individual edits, so client, operator and API requests cannot bypass the UI (individual firewall PUT/POST/DELETE return **409** while groups are assigned).
Apply and recovery: mutations validate and lock the account, group and affected VMs before changing intended state. **Account locks allow unrelated foreign-key references**, so device lifecycle activity cannot deadlock against a group change merely by recording an account event. Membership requires the same billing account and an eligible VM with no active provisioning job; duplicate assignment is idempotent; deletion requires removing assignments first. Intended changes and an unsynchronized marker **commit before remote fanout**, and each apply then locks its VM and reads the **latest committed** effective policy, so an older caller cannot overwrite a newer group's rules. Each VM keeps its own sync timestamp and error; failed applies preserve intended state, expose a customer-safe error and offer a POST retry. These are **synchronous** firewall applies, without a new job dispatcher. Destroyed devices are skipped during fanout and their stale memberships can be explicitly detached; deleting devices stay blocked until their job finishes. Policy and membership changes that risk SSH lockout require confirmation, including a restore to a saved deny policy.
`GET /api/v1/servers//firewall` reports effective configuration, a `managed_by_groups` boolean and a `groups` array of ID/name pairs; group rules carry `group_id` and `group_name`. Client pages are at `/client/firewall-groups/`, operator pages under `/console/billing/firewall-groups/`, **off `SUPPORT_CONSOLE_ROUTES`**.
## 5.10 Private networking
Customers create **account-scoped private networks** (Proxmox SDN VNets in the existing VXLAN zone), attach their own VMs, and may deploy **without a public IPv4**. An isolated server has no SNAT and no inbound path: access is the browser VNC console, or another of their VMs dual-homed on the same VNet.
Addressing is customer-picked RFC1918 (`/29`–`/24`) and can be changed later as long as every member IP and the optional gateway still sit inside the new range. **The CIDR is committed before isolation is pushed**, so saving the same CIDR again retries that hypervisor write when the VNet is still ERROR. With **Automatically assign IPs with Cloud-Init** on (the default) the address is written into Cloud-Init; with it off the NIC still attaches and guests use DHCP or in-guest config. The panel does **not** create SDN subnets, DHCP or IPAM objects. Changing the CIDR does not rewrite addresses already on members.
An optional **VNet gateway** is the LAN IPv4 of a customer firewall VM (pfSense or similar) already on that network, written as Cloud-Init `gw=` on private NICs only when Cloud-Init is assigning IPs. **It is not a Proxmox SDN SNAT, subnet or WAN.** Blank (the default) omits `gw`.
**Isolation is platform policy.** Customers do not create or edit VNet FORWARD rules — there is no WAN or gateway on the VNet itself, so a customer FORWARD table does not make sense. On create, sync and re-apply the panel writes a **platform** VNet firewall: default FORWARD **DROP** (north-south), one FORWARD **ACCEPT** whose source and destination are that VNet's CIDR (east-west between members, including a gateway VM on the same CIDR), and one trailing FORWARD **DROP** with no source or destination so the deny is a visible rule and not only the default policy. **`isolate-ports` is never enabled** — that would block guest-to-guest and kill the product. Dual-homed VMs still NAT or forward **in-guest** via their public NIC.
**`ipfilter-net0` is still public-NIC anti-spoofing.** Private NICs never get `ipfilter-netN`. The panel never creates `ipfilter-net1`, never sets `ipfilter=0` or `enable=0`, never accepts ipset names as VNet firewall sources, and never enables `isolate-ports`. VNet firewall is **FORWARD-only** (Proxmox has no IN/OUT on a VNet), and stock `pve-firewall` **ignores** those FORWARD rules — they need nftables `proxmox-firewall` on the hypervisor.
NIC and Cloud-Init contract:
| Deploy | Public NIC | Private NIC(s) | ipfilter | Cloud-Init |
| -------------- | ------------------------------------------ | -------------------------------------------- | --------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| Public only | `net0` `vmbr0`, firewall=1 | — | `ipfilter-net0` | `ipconfig0` public + gw |
| Public + VNets | as today | each VNet on `net1+`, `mtu=1450`, firewall=1 | `ipfilter-net0` only | `ipconfig0` public+gw; each private `ipconfigN=ip=…/nn[,gw=that VNet]` only when that VNet assigns IPs |
| Private only | — | first VNet on `net0`, more on `net1+` | skip creating `ipfilter-net0`; keep `enable=1`; omit `ipfilter` | private `ipconfigN` as above (no public nameservers) |
| No NIC | `PUT delete=` of cloned `netN`/`ipconfigN` | attach after deploy | skip creating `ipfilter-net0`; keep `enable=1`; omit `ipfilter` | no `ipconfig` |
Clone copies the template NICs, so a no-public-IP guest **`PUT`s `delete=`** for leftover `netN`/`ipconfigN` (the same method as unplug): `POST /config` ignores `delete`, which is why a Cloud-Init POST cannot strip them. Live apply reads the clone and deletes only keys actually present. **MTU 1450 is a `netN` property — do not put `mtu=` on `ipconfigN`**; Proxmox's schema there is `ip`/`gw`/`ip6`/`gw6` only and rejects extra keys with HTTP 400. Public IPv4 still owns `net0` when present; each attached VNet fills the next free `netN`; detach frees that slot and does **not** compact the others, so a later attach may reuse the hole.
A guest may attach to several of the account's VNets; **the platform does not route between them** and inter-VNet inspection is in-guest. Checkout and the API allow **public IP, VNet + IP, both, or neither** — neither is a no-NIC guest so a private interface can be attached after deploy — and those choices are part of `create_order_intake`'s duplicate-order fingerprint.
Caps: default **10** VNets per billing account (`BillingAccount.private_network_limit`; create refuses at the cap and Support can raise it from the client Account tab, writable on `SUPPORT_CONSOLE_ROUTES`), RFC1918 `/29`–`/24`, overlapping CIDRs across accounts allowed (VXLAN isolates L2), several private NICs per VM but not two attachments to the same VNet, generated VNet ids `n{id:07d}` (customer names go in `alias`), VNI range default **10000–19999** stored on the cluster row, one optional gateway IPv4 per VNet.
Local dry-run records VNets as ready, persists the platform isolation policy, and does not call Proxmox SDN. Live create is refused while `sdn_enabled` is off.
The client **Network** hub is one sidebar entry with these destinations: `/client/firewall-groups/`, `/client/network/private/`, `/client/network/floating-ips/`, `/client/network/reverse-dns/`, and **Bandwidth usage** at `/client/billing/bandwidth/`. Per-VM networking is a sibling of Firewall at `/client/services//network/`. Operator copies stay off `SUPPORT_CONSOLE_ROUTES`.
Before enabling `sdn_enabled` on a production cluster (human steps): leave the zone as-is and record its id on the cluster; grant the token `SDN.Allocate` and `SDN.Audit` on that zone; confirm `PUT /cluster/sdn` (apply) is safe on the node, since **every new VNet requires an SDN apply before VMs can attach**; enable nftables `proxmox-firewall`; **re-verify the VM firewall and `ipfilter-net0` after switching to nftables**; keep the datacenter firewall on; reserve the VNI range for the panel.
Not in v1: adding or removing a public IP after deploy, platform NAT/egress, IPv6, SDN DHCP/IPAM/subnets, customer-managed VNet FORWARD rules, and billing suspend/unsuspend as a power action. ISO attach and eject on a guest, and customer ISO upload, are [5.19](/platform/vps/images-and-backups.md#id-519-iso-library). Operator ISO ingest onto shared storage is also [5.19](/platform/vps/images-and-backups.md#id-519-iso-library). Backups and snapshots are [5.20](/platform/vps/images-and-backups.md#id-520-backups-and-snapshots).
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.layeronecloud.com/platform/vps/networking.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.