When Sakeena Fiza describes her work as a validation engineer at NVIDIA, she does so in terms more befitting a detective story than a world-class engineering lab.
“Validation engineers look in the shadows and shine a light into every corner,” Fiza said. “Every time we get a system, our first thought is: how can it break?”
And when it does?
“I always like to think of it as a mystery to solve,” she said.
At NVIDIA, the systems Fiza and her colleagues in the data center systems engineering lab investigate are the engines of the AI era. Her work begins before the rest of the world knows a product exists — in the lab — when a new system first receives power.
Components are brought up one by one, boards are integrated, firmware and software teams swarm, and engineers watch for the first signs of life.
One of Fiza’s earliest and most enduring memories of working at NVIDIA is the collective joy she experienced when she saw the NVIDIA Rubin GPU working for the first time at a system level.
“It literally just said, ‘NVIDIA Corporation Device,’” she recalls. “And everyone’s cheering and celebrating because it’s the first time in the world that a Rubin GPU enumerated at a system level.”
Those moments, electric as they are, are only the beginning. From there, the system must be made resilient: from tray to rack to cluster to production line to customer AI factory.
Fiza describes validation — the process of ensuring a physical device works correctly before mass production begins — as becoming “the first customers for the product,” exercising hardware to its limits in a range of real-world conditions before anyone else has to depend on it.
“The goal is to always catch issues before customers catch it,” she said.
Fiza arrived at NVIDIA after earning her bachelor’s degree at the University of California, Irvine, where she studied computer science and engineering. Her path into hardware was the result of an accumulating fascination with systems.
Growing up in Dubai, she was introduced to coding via the Logo programming language, prompting future forays into systems design that included building Mars rovers at a high school robotics camp and working on unmanned aerial vehicles in college.
What drew her to work with data center systems was the chance to work with the whole machine. At NVIDIA, she said, validation sits at exactly that intersection: firmware, hardware, software, mechanical design, thermal behavior, manufacturing and customer experience.
“I get to be a mechanical engineer when I want to be,” she said. “I get to be an electrical engineer when I want to be. I get to be a firmware engineer when I want to be.”
The failures she chases can be immense or microscopic. A rack-scale issue might involve high-speed signaling, thermal margins or power integrity. Another might come down to a screw tightened too far or the level of dust in a customer facility.
“The solution can be elusive,” Fiza said. “We have to follow the clues, ignore the red herrings and know where to look.”
When a log shows how something failed, Fiza’s job is to discover why. Validation engineers reproduce the issue, vary the conditions, investigate firmware, remove mechanical variables, probe signals, study scope shots and narrow the possible causes.
A single board may contain tens of thousands of components; a rack may approach half a million. Those parts must not merely coexist. They must behave as one system under stress, at scale, in the complex realities of production and deployment across diverse AI factory configurations.
“I wish people understood how complex the hardware is that AI needs to run on,” Fiza said.
For Fiza, the pressure of the work is inseparable from the pleasure of it. Bring-up, she said, is “like the Avengers assembling”: architects, designers, software engineers, firmware engineers, validation engineers, all in the room, racing toward a working system.
“One thing I know when I come to work is I’m never alone,” she said.
To be a validation engineer is to practice a disciplined kind of suspicion: believe a system can work, then try to conceive of every way it might not. The job requires the doggedness of a great detective, as well as the diagnostic abilities of a general practitioner and the temperament of someone who meets catastrophic failure in the way others might a crossword.
Each project brings a new puzzle, a new failure mode and, in turn, a chance to make the next system better.
“With the products we have in the pipeline, I’m so excited,” Fiza said. “They’re going to change the world.”
