A tour of my home network

Published on 2024-09-10 by ADAM

My home network is an IPv6-mostly network. It's "ready for the future". In most parts of it, I've retired legacy IP addressing entirely. It took a lot of work to get to that point. I'll detail the various steps of how I got here in a future set of blog posts. In this post, I'll give a brief outline of what my network looks like. We'll take this tour "from the outside-in" starting at the ISP uplink and then working inward. There's a lot to cover here -- I live in a 1k SqM house on 12k SqM of land.

Let's start with the ISP. My home ISP is Verizon FIOS. They provide both legacy IP and IPv6. FIOS is ASN 701. This means that they have the following allocations (ignoring legacy IP allocations):

And they appear to advertise the following ranges out of their second allocation:

Astute observers will notice that dig aaaa blahg.nerdland.org doesn't come from any of ASN 701's allocations -- instead it's 2001:470:e1da:1ab::b106 (The last two bytes spell out "blog" when you squint at it -- cute, right?) And yet this site IS being hosted out of my house. How this works will be explained later in this post. (If you trace who owns 2001:470::/32, you'll instantly figure out the secret.)

So, Verizon assigns me a /56, specifically one like: 2600:40WW:XXYY:ZZ00::/56. The WW:XXYY:ZZ part I'm not revealing, for two different reasons. First, of course, is privacy. Second, while the WW:XX part seems to stay static for me, which would reveal information about my location, the YY:ZZ is dynamically allocated. Thus from time to time, Verizon change it on me (and it would invariably become out-of-date on this page). This happens either because my router stays down for maintenance long enough to lose the reservation, or because folks at Verizon just like annoying me. Perhaps both?

It's widely known that Verizon assigns your IPv6 via DHCPv6 and a Prefix Delegation of /56 is granted. Further, like every other ISP providing Legacy IP addresses to its customers, DHCPv4 is used to provision an IPv4 address. Thankfully, Verizon doesn't appear to use CG-NAT for my area, which makes certain use cases far less painful.

I do not use Verizon's provided CPE gear except for their ONT ("modem", if you like). Specifically, I am not using their router. There is a known problem with their router and certain Intel network adaptors it uses -- thus Verizon has been disabling IPv6 on a lot of their customers... Hopefully, this problem is resolved soon -- the internet has to keep moving in the right direction.

My router is a "custom job". It's a regular amd64/PC running stock FreeBSD. PFSense and other related projects are absolutely fantastic products. I strongly recommend them; however, I like to have total control. Further, doing this all by hand is very educational.

The router also runs a DNS server (BIND) for the network, a DHCP server (ISC DHCPD), and a few other things (detailed a bit later). The router has 5 1-gigabit network cards in it. One, of course, is hooked up to the Verizon ONT. That interface has a public legacy IP address, but it never is provisioned with a public IPv6 address. (This is perfectly normal.) The other 4 are hooked up as follows:

  1. Attached to a TP-Link 18-port Gigabit PoE managed switch.
  2. Attached to an Allied TeleSyn 6-port Gigabit managed switch. (Dumpster find!)
  3. Attached to a Dell PowerConnect 2724 24-port Gigabit managed switch. (Dumpster find!)
  4. Direct Cat-6A connection to the machine hosting this site.

The FreeBSD router uses pf for firewalling on all protocols and NAT (eeeeew!!) for legacy IP. The first local network card is setup for 12 VLANs. The second network card has 4 VLANs. The third network card is setup for 3 VLANs. The direct connection doesn't use VLANs. For each of the three switch-connected interfaces, a single VLAN is setup for infrastructure management purposes -- these interfaces get legacy IP addresses only. Sadly, the switches only can be provisioned over IPv4. They get addresses for themselves on these VLANs. Some pf rules are in effect to prevent traffic to these VLANs from unauthorized sources.

The remaining VLANs all get modern IPs, and a small subset also get legacy IPs (Dual Stack). The Allied TeleSyn provides network for my home-office, the office cubby in my server room, and part of the phone system in my server room. Each of those functions has a dedicated VLAN.

The Dell switch provides network for the two halves of the lab in the server room. One half is "modern equipment". The other half is the "zoo of the ancients". Each of those is a dedicated VLAN. The "modern equipment" VLAN runs modern machines. The "zoo of the ancients" has a dedicated router providing legacy internet services to some of the very old curiosities. There's a few PowerPC Macs (a few G3s, a few G4s, a few G5s), a few SUN UltraSparc IIi systems, two DEC Alphas, an SGI Indy, an RS/6000, a dozen mc680x0 Macs, a few mc680x0 NeXTs, a Dreamcast, and a few other things. Since many of these systems all run ancient OSes without modern IP support, those exist on their own network and switch, behind that second router. These machines have no real access to the wider world. One can SSH into the router and then SSH from there into any of these other machines.

The TP-Link switch provides most network infrastructure for the rest of the house & family. One VLAN (untagged, so not really a vlan) is "standby" with Dual Stack in case I ever screw up vlan provisioning in the switches -- I can disable certain vlan settings in the wifi controllers and recover service in case I make a mistake. The 10 remaining VLANs provide WiFI for various purposes (no legacy IPs unless specified otherwise):

  1. Primary WiFi Network
  2. Guest Network
  3. IP Phone Network -- more on this later (Dual Stack)
  4. Streaming Device Network (Dual Stack)
  5. Internet of Things Network (Dual Stack)
  6. Home Security System (Dual Stack)
  7. Printers (Dual Stack)
  8. Video Game Consoles (Dual Stack)
  9. "Legacy Home" (Dual Stack)

Sadly, because I use Roku streaming on some TVs, I have to provide legacy IP for the streaming network. And, depressingly, myriad IoT devices seem to be cranky without legacy IP connections. And Nintendo hasn't discovered IPv6 at all. And I have not yet setup my IP phone system for legacy-free networks. The "Legacy Home" network is for any home users with devices that need to connect to legacy IP services via IPv4 literals and the devices do not have CLAT support (using a VPN on Windows, for instance). Thus, Printers have to be available on legacy IP -- I've had problems trying to get RDP sessions on Windows over legacy IP to talk to local printers over modern IP. VPNs and Windows for office work -- mostly I get to avoid these, most of the time. But my wife can't. IoT and Windows/VPN seem to be the major remaining legacy IP holdouts in my home network.

The Wireless infrastructure is provided by a handful of strategically placed TP-Link Omada APs. Most are EAP 225s and there's a few outdoor Omada APs (I forget the model). Most are powered via PoE from the TP-Link switch. An Omada SDN controller links and manages all of the APs. This system provides one SSID for each of the 9 VLANs described above. The SDN adds and removes VLAN tags on traffic from each appropriate VLAN.

The router runs Tayga to provide NAT64 services for all devices in the house. The DNS server on the router is setup for DNS64. An IPv6 tunnel to Hurricane Electric is also running on the router. This provides my access to 2001:470:e1da::/48. I rather like this allocation, as when I squint at the 70:e1da part it looks almost like Zelda, one of my favourite video game series. My router sends ::/0 out directly over the Verizon FIOS link. This causes a problem, as Verizon won't (for some sensible reasons) route IP traffic with a source address from outside my allocation. They want to prevent spoofing and to prevent misconfiguration. But global traffic bound for 2001:470:e1d::/48 emerges from the tunnel in my router, so there's that half of the link already. From there, it's already capable of sending it to various machines in this lab. The 2001:470:e1da:1ab::/64 (1ab looks like lab) subnet is dedicated to the lab. And for this webserver, I assigned 2001:470:e1da:1ab::b106 (looks like blog) as a cool-looking 6-address. When my webserver responds to your traffic, it normally would come back via the Verizon direct route. However, instead I have to setup dual fibs on my router and a few pf rules to cause traffic originating from my HE mapped network to go back out over the HE tunnel. Thus only really the web traffic goes over the HE tunnel, but my normal home and business traffic go out over the proper 6-link for Verizon.

My DNS server presents itself to the wider internet over both modern and legacy IP. I do still run a "split-horizon" DNS, however. This is because in addition to the 2000::/3 GUA addresses I provision internally, I also run fd00::/8 ULA addresses internally. This is to permit me to keep static addresses for machines in my network (and not have to alter DNS records every time Verizon decides to change my public address allocations). I can ssh router.local, even when public network addresses change, just like I used to do when running legacy IP. I'm moving towards making all publicly advertised machines expose addresses from 2001:470:e1da::/48, just like this webserver does. This webserver is an experiment of that technique. However, the primary limitation there is that I must make sure that all machines have 2600:4000::/24 allocations such that I can craft most traffic to go out over the Verizon link.