A gentle introduction to hardware architectures for DNN acceleration

“The synergy between the large datasets in the cloud and the numerous computers that power it has enabled remarkable advancements in machine learning, especially in DNNs. […] That changed in 2013 when a projection in which Google users searched by voice for three minutes per day using speech recognition DNNs would double Google datacenters’ computation demands.” This presentation will introduce the concepts behind the hardware architectures used to support current growth in machine learning, including GPUs and TPUs.

Norman P. Jouppi, Cliff Young, Nishant Patil, David Patterson. 2018. A Domain-Specific Architecture for Deep Neural Networks.