The situation
The Cybersecurity and Infrastructure Security Agency trains the people who defend federal networks. Much of that training is hands-on: students work inside realistic environments where they configure firewalls, investigate intrusions, analyze logs, and handle live malicious code. Those environments, called cyber ranges, are expensive to run and difficult to get right.
CISA's ranges were supplied by a third-party vendor as a managed service. The agency paid approximately $50,000 every month, and it paid that amount whether a course was in session or the labs were sitting empty between classes. For a training program, that is the worst possible cost profile: demand comes in bursts around scheduled courses, but the bill never changes.
The cost was only part of the problem. Instructors were building their lesson plans around what the platform could do, rather than around what students needed to learn.
Why the existing approach failed
The vendor platform was built on large, pre-built virtual machine images. Every course received the same network: domain controllers, file servers, and other infrastructure, whether or not the lesson needed them. That design produced four problems that compounded each other.
- Always-on billing. Fixed infrastructure was paid for continuously, regardless of use.
- Slow preparation. Because the images were so large, preparing a single student's environment took two to four hours. Students waited, and exercises were deferred.
- One network for every course. Instructors could not shape an environment around a specific lesson, tool, or threat scenario without an expensive reconfiguration.
- No visibility and no exit. The agency could not see usage, attribute cost, or audit behavior, and the proprietary design offered no path to extend or migrate the platform.
Each problem pointed at the same root cause. The labs were treated as permanent machines to be rented, when they should have been temporary environments to be built.
The approach and the decisions behind it
I proposed replacing the rented machines with labs defined entirely as code, built for each student when needed and destroyed when finished. I wrote the first version of the platform myself, from the provisioning engine to the dashboards, to prove the idea before asking anyone to staff it. A handful of early decisions shaped everything that followed.
- Build labs from code, not from snapshots.
- Every environment is a version-controlled Terraform template. A lab is assembled from exactly the parts its course needs, which makes it small, fast to build, and identical every time.
- Make idle free.
- If labs exist only while someone is using them, an empty week costs nothing. Shutting services down became a routine, zero-cost action rather than a decision anyone had to weigh.
- Give every course its own range.
- Instead of one network for all courses, each course runs in its own isolated range: a Windows enterprise with Active Directory and Exchange for one class, industrial firewalls for another, a SIEM and log pipeline for a third.
- Deliver everything through the browser.
- Students reach Linux and Windows desktops through Apache Guacamole in a single browser tab. There is nothing to install, no VPN, and no credential for a student to mishandle.
- Choose portability over convenience.
- Several AWS-native services were used during development but deliberately kept out of the product, so the platform could be deployed to AWS GovCloud or Azure Government and operated by agencies with strict service restrictions.
- Put instructors in control.
- Self-service dashboards let instructors launch, monitor, and retire labs without filing a request with an engineering team or a vendor.
How it works
From a student's point of view, the experience is a single action: open the course and click to launch the lab. Behind that click, the platform does the following.
- The student launches the lab from inside the course. The platform checks whether the student already has a lab and, if so, reconnects them to it where they left off.
- A lab is assigned or built. If a ready lab is waiting in the warm standby pool, it is assigned immediately. Otherwise the request goes on a queue, and the range processor assembles and applies the Terraform for that student, typically in under thirty seconds.
- The lab is isolated. Every student gets dedicated machines and a dedicated security boundary, named and tracked by a unique 32-character identifier. Outbound network access is decided at the infrastructure level, which is what makes it safe to study real malware.
- The desktop arrives in the browser. Randomized credentials are generated and injected directly into the gateway. The student never sees or handles them.
- The lab is released. After inactivity or at the end of the course, it is returned to the pool or destroyed, and its cost stops.
The system is organized in four layers, each deployed as code and each depending on the one beneath it: the cloud organization and identity controls, a central control plane, the application platform, and the individual course ranges. Every network, platform and range alike, follows the same five-tier subnet pattern, so security reviewers evaluated one design rather than dozens.
Making cost visible
The financial change was not a discount on the old bill. It was a different cost model. Under the vendor platform, cost was a fixed monthly figure with no breakdown. Under the new platform, nothing is billed while nothing is running, and every running lab is recorded from the moment it is provisioned to the moment it is released.
Shutting a range down costs nothing, and restarting it costs nothing until students arrive.
For the first time, the agency could answer questions it had never been able to ask: what a particular course costs to deliver, what one student's lab cost over a multi-day class, how spending moved month to month and quarter to quarter. Because each allocation carries its own running cost, budget owners could forecast the next quarter from real usage and defend the figure with data.
The visibility changed behavior as well. Instructors began sizing labs deliberately, and because shutting a service down cost nothing to reverse, no one hesitated to do it.
The hard problems, and how they were solved
Earning an Authority to Operate
Nothing runs inside DHS without an Authority to Operate, and no SaaS product had ever been granted one. I wrote the security documentation, architecture diagrams, and compliance evidence myself. The design helped make the case: hardened container images from the Department of Defense's Iron Bank registry, no credentials exposed to users, strict isolation between students and courses, controlled outbound access, and central audit logging. The platform became the first SaaS product to receive an ATO from DHS and CISA.
Working within GovCloud's limits
AWS GovCloud is deliberately restrictive. It cannot host public DNS zones, and it limits access from ordinary tooling. Public DNS was placed in a commercial account and pointed into GovCloud, and all infrastructure changes run through a dedicated, access-restricted executor inside the GovCloud management account. DNS records are created in a second phase, only after the infrastructure they point to is confirmed to be running.
Many students starting at once
Classes start on a schedule, so demand arrives all at once. Provisioning requests are placed on a durable queue with a fast cache in front, which allows many labs to be built in parallel without blocking the interface. A warm standby pool absorbs the first wave so that most students never wait at all.
Keeping credentials away from students
Credentials are the usual weak point in training environments. Here they are randomized per lab, injected directly into the browser gateway, and never shown to the student or stored in the student dashboard. Access URLs are tied to the student's unique identifier and cannot be bookmarked or reused.
Course content without version control
The agency's learning system, Moodle, assumes every edit happens directly in production. It has no staging, no versioning, and no rollback. The management system added a content pipeline so course authors could develop, test, and promote material safely across environments.
Sizing the browser gateways
Remote-desktop gateways are memory-intensive, and courses differ widely in what they ask of them. Capacity was set per course rather than globally, and reduced for the heaviest ranges when monitoring showed memory pressure, so a demanding class could not degrade a lighter one.
Outcomes
The rented platform was retired. In its place, CISA had a system it controlled, whose cost it could see, and which it could shape around its courses.
- The idle bill disappeared. The roughly $50,000 monthly charge was replaced by a model in which an idle range costs nothing.
- Waiting disappeared. Lab preparation fell from two to four hours to under thirty seconds, and to effectively zero for students served from the standby pool.
- Cost became a management tool. Spending could be traced by student, course, month, and quarter.
- Courses became faster to build. New courses could be launched in days rather than weeks, each with an environment designed for its subject. The platform has hosted nine distinct ranges, from Windows enterprise defense to industrial control systems.
- The platform supported hundreds of concurrent students across isolated environments.
- It opened doors beyond the project. Its success contributed to the consulting firm that delivered it being awarded a follow-on federal contract worth more than $100 million.
My role
I invented the platform and built the original version single-handedly: the provisioning engine, the Terraform templates, the APIs, and the dashboards. Once it was proving itself, I brought in a DevOps engineer, then gradually grew a small team. Together we matured the invention into the platform the agency relies on.
Along the way I led the team's agile practice, trained engineers who were new to AWS, brought in a designer for the learning-system experience, and wrote the ATO package. I worked directly with instructors to turn learning objectives into lab designs, served as the primary technical contact for DHS, CISA, and FEMA stakeholders, and worked with the consulting firm's chief technology, information, and operating officers on the platform's direction.