How I moved 45,000 endpoints to 802.1X without anyone noticing
Founding a Cisco ISE initiative from scratch, the architecture decisions that carried it from a 4-node cluster to 14 nodes across five global datacenters, and the monitor-first rollout that grew into 60K+ endpoints behind 802.1X.
The riskiest project I ever ran produced no noise at all. The first three client deployments took roughly eight months end to end, and they were the proving ground for a process that has since put tens of thousands of endpoints behind 802.1X authentication on Cisco ISE. The help desk queue stayed flat the whole way. That was the goal from the first design meeting. Network access control that announces itself is network access control that gets rolled back.
The question nobody could answer
The initiative started with client requirements: three programs needed 802.1X, contractually. But the reason I pushed it far past those three was a question that kept bothering me. What, exactly, is on our network right now?
Before ISE, the honest answer was “whatever plugged in.” Our AAA story was Cisco ACS 5.x handling device administration, which meant the routers and switches authenticated their admins while the endpoints themselves were never asked to prove anything. Ask three engineers for an inventory and you got three different lists, none of which matched the switch configs. That gap stays invisible right up until an auditor or an incident makes it very visible.
I handled the ACS-to-ISE migration myself, and that became the entry point for everything that followed: ISE 2.3, patched through 2.4, then a parallel 3.x environment built alongside the old one so the platform could move forward without betting production on an in-place upgrade. More on that pattern later, because it decided how everything else went.
Three decisions that carried the project
Sizing for the end state. We started with a 4-node ISE 2.3 cluster: two combined admin and monitoring nodes, two policy service nodes. By the time the 3.x environment matured it was a 14-node distributed deployment: dedicated admin pair, dedicated monitoring pair, and two policy service nodes in each of five global datacenters across North America and Asia. The rule I held was to size for the enforcement phase, not the pilot, because resizing a live authentication platform is slow, careful work best done zero times.
Policy sets built around protocols, with populations inside. The 3.x policy structure put security first by splitting on access method: wired 802.1X, wireless 802.1X, wired MAB, wireless MAB, guest. Device populations, corporate laptops, IP phones, printers, and the long tail of things with an Ethernet jack, lived as authorization logic inside those sets. Device administration got its own TACACS policy sets organized by platform family: IOS and IOS-XE, Nexus, wireless controllers, the management platforms, ASA firewalls, Palo Alto. Protocols are stable; populations and platforms grow. The structure absorbed both without a redesign.
Failure behavior decided up front. What does a switch port do at 2am when it cannot reach ISE? We wrote that answer before any port enforced. Access ports fail closed, because a NAC platform that fails open is a suggestion. Cisco phones authenticate with their manufacturer-installed certificates and the ports trust them accordingly. And every network device kept a single local break-glass account for the day TACACS is unreachable. If you cannot say what happens when authentication dies, you are not ready to turn it on.
Monitor mode is where the truth lives
Every port started in open mode: authenticate everything, block nothing, log the difference. Those logs became the most valuable artifact of the entire project. They caught a steady stream of broken supplicants, which usually meant vendor drivers that needed updating before they could speak enterprise 802.1X properly. Devices that could not be fixed moved to MAB with an endpoint identity group wired into the policy set rules, so access stayed tight no matter how a device connects. The same logs proved out the policy logic itself, showing every authentication matching the rule it was designed to match before anything enforced.
The staging worked on the Windows side too. Test OUs carried the 802.1X GPO settings first, so supplicant behavior was validated on a handful of machines before the production OU design rolled the same policies to everything else. Every line in the failed-auth report was an outage that never happened.
Enforcement in waves that earned their size
The first client rollout started with a batch of two or three PCs. Machines moved into the production OUs during change windows, which is harmless ahead of the port configs, and once AD replication settled we enforced in batches of ten until the first 250 machines were done. As the process proved itself the batches grew to fifty at a time, running across multiple geographies at once, because some client programs span delivery centers in different regions. Everything production-facing rode a change request with post-implementation support and testing attached, and when something did pop up, it got fixed fast enough that the rollout kept its reputation for being flawless. The perception was the product of the response time, not the absence of problems.
The tooling grew up alongside the batches. The first site ran on config templates I kept in a notepad. Those became templates in Cisco DNA Center, pushing consistent configs to IOS and IOS-XE access switches in a fraction of the time. And when the process was solid enough that it no longer needed me hovering over it, the network org made the right call: stop rolling out by client, start rolling out by site. It is the same habit I use in my homelab, change one thing and watch before changing the next. Production did not change the instinct, only the size of the log files.
What it added up to
Today that platform authenticates 60K+ endpoints, wired and wireless, and the number keeps climbing as more wireless devices join. It carried a Duo integration that put MFA in front of ISE administration, and it became the authorization brain for remote access too, with the VPN headends authenticating against Azure MFA and then handing authorization to ISE posture checks. The enterprise rollout continues past my chapter of it, moving authentication from MSCHAP and PEAP to certificate-based EAP-TLS, which is exactly what should happen to a platform built to outlive its founder.
What I would tell my younger self: the switch configuration was the smallest part. The project was won in the sequencing. Visibility before enforcement, monitor mode before closed mode, small batches before big ones. Get the order right and the endpoints move quietly.
People sometimes ask what a successful NAC deployment feels like. It feels like nothing happened. That is the product.