Automatic differentiation in R

I’ve been working on a small R package that does automatic differentiation. It takes a function that computes an arithmetic expression as input and outputs a function that computes the derivative of the expression. You can check it out on GitHub.

I got inspired to write it a few weeks ago when one of our PhD students gave a talk on automatic differentiation. I didn’t attend the talk, but remembered playing around with it as a meta-program in C++ templates ages ago. Now that I am writing a book on meta-programming in R, I thought it would be a cool example to include there—and I have included it in the chapter I just finished. I gave it to a student as a project, but I am not patient enough to let someone else program it, so I have also done it myself.

It is actually a nice exercise to do. Differentiation is pretty simple to program. You just follow the rules you learned in calculus for the arithmetic operations and apply the chain rule for function calls. Nothing complicated there. To make it a meta-program in R, though, you need to know how to work with expressions and how to inspect functions to correctly apply the chain rule. While this is not particularly hard, this example is great at getting around the various corners of working with expressions.

Unless I think up something else to add, I think the meta-programming book will be done after one more chapter. After that, I will take a short break from the R books. I will get back to them in a few weeks, I imagine, but I have a few other projects to focus on before then. Including proof-reading my data science book—that should arrive next week and then I have to get through it in a week before it goes to the printer.

I haven’t decided yet what the next R book should be. I’m thinking either functional data structures and algorithms or embedded domain-specific languages. Let me know what you think.

Author: Thomas Mailund

My name is Thomas Mailund and I am a research associate professor at the Bioinformatics Research Center, Uni Aarhus. Before this I did a postdoc at the Dept of Statistics, Uni Oxford, and got my PhD from the Dept of Computer Science, Uni Aarhus.

2 thoughts on “Automatic differentiation in R”

  1. I guess it is symbolic differentiation, yes. I have also played with the dual number representation, but for meta programming this solution was a better example.

Leave a Reply