The migration workbook: how I make risky network changes boring

The numbers first: a global wireless estate of about 900 access points and 190 WLANs serving around 10,000 wireless clients a day, migrated from cloud-managed Meraki to sixteen Cisco Catalyst 9800 controllers across four regional datacenters, with a AAA migration from ISE 2.4 to a parallel-built 3.x environment running underneath. Users noticed nothing. Not being noticed was the goal, and the method that got us there is dull on purpose.

Twenty years of enterprise changes have convinced me that risky work goes wrong for a boring reason: someone touched a system they had not fully inventoried. Everything in this method exists to make that impossible.

Audit first, opinions later

Before the migration touched anything, I audited the entire Meraki estate: which SSIDs existed at which locations, what authentication each one used, how the sites actually differed from each other. Not the documentation. The running configs. That inventory became the raw material for two designs at once, the 9800 controller architecture and the ISE policy sets that would authenticate everything behind it.

Here’s the catch with audits: they feel like delay. Days of pulling configs while nothing visibly moves. That discipline is precisely what buys the quiet cutover later, so I defend the audit time the way I defend the maintenance window itself.

The audits did not stop at cutover either. Once the 9800s were in production we audited again and tuned what the real world exposed: idle timers that logged people out over lunch when they locked their laptops, RRM values that needed adjusting for particular buildings, band-steering options per site. Wireless environments are shaped by walls and density and whatever construction happened last quarter, so post-deployment tuning is not a failure of the design. It is a scheduled part of it, sometimes more than once.

The design underneath

Each regional datacenter got the same four-controller layout: two controllers in an HA pair carrying the internal WLANs, a third standing alone with an identical configuration, ready to absorb access points during failures or maintenance, and a dedicated anchor controller in the DMZ for guest traffic, reached over mobility tunnels. Four regions, sixteen controllers, and a hard rule that access points join controllers in their own region. Hauling CAPWAP tunnels from India or the Philippines back to a US controller stresses the tunnel design and invites problems that never need to exist. The access points themselves mapped one to one against the Meraki fleet, with counts increased where a site’s density earned it.

Guest wireless shows the design philosophy in its purest form: balance security against availability, then give each segment exactly what it needs. Guests terminate on the DMZ anchor, isolated from the inside of the network. From that segment we denied all RFC1918 address space except the ISE nodes, DNS, and DHCP. Internet access was restricted to well-known web ports and VPN services, because the most common legitimate thing a guest needs is a path back to their own company’s VPN. Nothing about any segment was accidental. Every path a packet could take was a decision somebody made on purpose.

Write it down so anyone can run it

Every piece of the migration got documented and published to Confluence as it stabilized: controller inventories with hostnames and addressing, deployment instructions, guides for building each type of wireless network, a full design document for the guest and DMZ wireless path. The point was never the documents. The point was that the next engineer could execute the same change the same way without me in the room.

Two rules governed every change regardless of who ran it. A change is not ready until its verification and its rollback are written, both before the window opens. And no change does two things. When something misbehaves during a cutover, single-purpose changes tell you exactly what to undo. Compound changes turn the undo into guesswork.

Verification has a name and a face

Early windows captured client counts per SSID so we had numbers to compare against. But the verification that mattered most was live. Cutovers ran on scheduled calls where high-profile users and client contacts got a white-glove pass: they worked their real workflows while we watched the authentication flows land in ISE in real time. Anything that surfaced got handled on the call, or scheduled with the right people before we hung up. After onboarding, issues flowed through tickets and a Webex channel until they went quiet.

“It works” is not verification. A named person doing their actual job on the new platform while you watch the auth logs, that is verification.

Cut over with the radios

The cutover design is the part I would keep above all else. Every site’s new Cisco environment was fully pre-staged: APs onboarded, WLANs built, RADIUS configured, radios off. Dark hardware, ready to broadcast. The cutover itself was almost embarrassingly small: cut PoE to the Meraki APs, enable the Cisco radios, confirm the SSIDs are broadcasting, test authentication, watch the flows in ISE.

We ran corporate wireless site by site, since it looks the same worldwide, and held a stricter per-site, per-program sequence for client-specific SSIDs so each program migrated whole. The posture inside a window was fix forward. Running two wireless platforms in parallel is not a safety net, it is an interference generator, two systems fighting over the same channels and airspace. The Meraki side had to go dark for the new environment to perform, so the honest plan was to make the new environment work, with re-powering the old APs held as the break-glass option that never got used.

The AAA side ran on the same pre-staging principle. Every 9800 was built with both ISE environments’ policy service nodes configured, old and new, selectable per SSID. Moving a WLAN from ISE 2.4 to 3.x was a selection, not a rebuild. And for the thousand-device TACACS estate, I built the AAA migration project itself, turning what had been a manual, device-by-device repointing exercise into a defined, repeatable process. Design the migration path into the platform on day one and the actual migration becomes configuration, not surgery.

Why boring wins

This pattern works for any change where the failure mode is “a building full of people cannot work,” whether the system is wireless, AAA, or routing. Audit the real state, pre-stage everything that can be staged, publish the runbook so the process outlives you, verify with real humans doing real work, and decide your rollback posture while you are rested.

Oddly enough, I learned the shape of this from guitar practice. You play a passage at half speed until it is clean, because speed hides mistakes and slowness exposes them. A cutover window is the same instrument. If a change only succeeds when you rush it, it does not succeed.

The migration ran quietly, the docs outlived the project, and in a network holding 99.99% uptime, a quiet cutover beats an impressive one every time.