Work / Backup Exec

Case study · Arcada Software, later Seagate and Symantec

Building Backup Exec from the hardware up

How a small founding team built one of the industry's first comprehensive backup products, and how the work closest to the hardware helped carry it to the top of its market.

Short link: fredlackey.com/backup-exec
Full. Incremental. Differential.

Executive brief

The problem

As businesses moved their critical data onto shared network servers, they needed backup that worked reliably across many kinds of tape hardware and captured more than just files.

What was built

Backup Exec: scheduled full, incremental, and differential backups across multiple servers, built on device drivers that let it work with a wide range of tape drives.

The outcome

Arcada grew from a handful of developers to more than a hundred people, was acquired by Seagate, and Backup Exec became the world's leading backup software before its later sale to Symantec.

Foundingmember of the original development team
100+employees, grown from a handful of developers
#1backup software worldwide under Seagate
Twoacquisitions: Seagate, then Symantec

The situation

Businesses were connecting their offices on Novell NetWare, and the information that ran them was moving off individual desks and onto shared servers. That concentration made the data more useful and far more exposed. A single failed disk could take an entire office's records with it.

Arcada Software was founded, with backing from an executive in the tape-backup industry, to build one of the industry's first comprehensive backup solutions. I joined as part of the small founding development team.

Why backup was harder than it looked

Backing up a networked server was not simply a matter of copying files to tape.

  • The hardware varied widely. Tape drives from different manufacturers behaved differently, and software that worked with one often failed with another.
  • Files were not the whole picture. A NetWare server's bindery and directory services describe the network itself: its users, groups, and permissions. A backup that restored the files but not the directory left an organization unable to use them.
  • Backups depended on people. Without reliable scheduling, protection depended on someone remembering to run the job and change the tape.
  • Networks had more than one server. Protecting an organization meant reaching across several machines and platforms, not just one.

The approach and the decisions behind it

Own the layer closest to the hardware.
I wrote low-level SCSI device drivers in C and C++ so Backup Exec could communicate reliably with a wide range of tape devices. Broad, dependable hardware support is invisible when it works and decisive when a customer is choosing a product.
Protect the network, not just the files.
Backup Exec was engineered to capture NetWare bindery and directory-services data alongside file data, so a restore brought back a working network rather than a pile of files.
Make protection routine.
Scheduled full, incremental, and differential backups let administrators balance the length of nightly jobs against the number of tapes needed to restore.
Use what the hardware already does well.
Support for the tape drives' native compression put more data on each tape without spending server time to compress it.
Reach every server from one place.
Communication protocols allowed a single backup job to span multiple servers, and lightweight agents for UNIX and NetWare compressed and sent data from the source machine to improve backup and restore times.

How it fit together

The Backup Exec stack Four layers. At the top, the servers being protected: NetWare servers with their bindery and directory services, Windows NT, and UNIX through agents. Below them, the Backup Exec engine with scheduling, multi-server jobs, and compression. Below that, the SCSI device drivers, the layer Fred wrote. At the bottom, the tape drives. Servers being protected NetWare with bindery and directory services · Windows NT · UNIX through agents Backup Exec engine Scheduling: full, incremental, differential · multi-server jobs · compression SCSI device drivers C and C++ · the layer I wrote Tape drives many manufacturers, many behaviors
Figure 1. Where the driver work sat: between the product and the physical media. Swipe the diagram sideways to see all of it.

The product began on NetWare and later expanded to Windows NT and Windows desktop systems; I helped build those later versions as well. A close technical relationship with Microsoft brought Backup Exec drivers and services into Windows itself, alongside its place in the NetWare world.

From the code to the customer

As Arcada grew, I moved into technical support leadership. We supported customers around the clock with hardware selection, system assembly, installation, and the practical work of getting backups to run. I made a point of carrying what customers told us back to the engineering team, so real-world failures and requests shaped the product.

One problem had no off-the-shelf answer: getting driver updates into customers' hands. I built a content management and distribution system in Visual Basic, ASP, and SQL Server to publish drivers through Wildcat BBS and CompuServe. It solved, for its time, the version-distribution and dependency problems that modern package systems handle today.

Outcomes

Handfulof founding developers
100+employees as the company grew
#1backup software worldwide under Seagate
2acquisitions: Seagate, then Symantec
  • The company grew from a handful of developers to more than a hundred employees.
  • Seagate acquired Arcada, and the business became Seagate Software's Network Storage Management Group.
  • Backup Exec became the world's leading backup software under Seagate's ownership.
  • Symantec later acquired Backup Exec, where it continued as a mainstay of data protection.

My role

I was part of the founding development team. I wrote the SCSI tape device drivers, engineered bindery and directory-services backup, built the scheduling and compression support, wrote UNIX and NetWare agents, and helped bring the product to Windows NT. As the company grew, I led technical support and built the internal systems that delivered drivers to customers.

Build the part closest to the metal well, listen to the people using it, and get the fixes into their hands.
What Backup Exec taught me

Next step

Why it still matters

The lesson carries over to almost every product I have worked on since. The least visible layer, whether device drivers, provisioning engines, or data pipelines, usually decides whether a product earns trust. And the fastest route to a better product is a short path from the customer's problem to the engineer who can fix it.

Read the Homeland Security case study