Automatic Differentiation in Machine Learning: Survey and Revisionist History
1 : Maynooth University
-
Website
* : Corresponding author
Department of Computer Science, Maynooth University, Maynooth, Co. Kildare -
Ireland
Automatic Differentiation (AD) is the mechanical transformation of computer code implementing a numeric function to also exactly and efficiently calculate various derivatives. AD had its first successes at the dawn of the computer age, and has continued to develop. In this talk we will explain the basic ideas of AD. We will also explore where it can be of assistance in rapidly and correctly implementing and analyzing complicated models, and where manual assistance is still required to attain desirable efficiencies.