Bill Dally

Bill Dally studied electrical engineering at Virginia Tech and Stanford, then took a PhD in computer science at Caltech in 1986, where he worked on how messages move between the processors of a parallel computer. In eleven years at MIT his group built two experimental machines, the J-Machine and the M-Machine, designed to make communication and synchronization between processors cheap. At Stanford from 1997 he chaired the computer science department from 2005 to 2009. The Queen Elizabeth Prize foundation credits his Stanford team with the network architecture, signaling, routing and synchronization technology "found in most large parallel computers today."

He had consulted for NVIDIA since 2003 and became its chief scientist in January 2009, building the company's research lab. He told Caltech's alumni magazine that a breakfast with Andrew Ng, who described a neural network running on 16,000 processor cores that had taught itself to recognize cats in YouTube videos, convinced him NVIDIA's GPUs could do the same work with far fewer chips. He then told NVIDIA's executive staff, "This is going to take off. This is going to be really big."

On 12 September 2023 Dally testified for NVIDIA before the Senate Judiciary subcommittee on privacy and technology, where he argued that "AI is a software program, not a nuclear reactor" and that no country or company controls a chokepoint in AI development. In a June 2026 lecture in Singapore he said that cutting arithmetic precision from 32 bits to 4 had roughly quadrupled energy efficiency at each step, and that the computing used to train leading models had grown about 10 million times since AlexNet ran on two GPUs in 2012. He shared the 2025 Queen Elizabeth Prize for Engineering with Bengio, Hinton, Hopfield, Huang, LeCun and Li, and in March 2026 appeared at NVIDIA's GTC conference in conversation with Google's Jeff Dean.