https://tomroth.dev/tags/autodiff/Recent content in autodiff on Tom RothHugo -- gohugo.ioenThu, 16 Feb 2023 00:00:00 +0000https://tomroth.dev/compgraph3/Thu, 16 Feb 2023 00:00:00 +0000https://tomroth.dev/compgraph3/This is the third part of a series on creating a computational graph in Python. You can go back to part one and part two. There’s a bit of code in this post. Here is the final code in np_wrapping.py. The module for numpy_autograd is here. In the last post we created computational graphs for a function, but it was a bit hard to use. We also had these problems:https://tomroth.dev/compgraph2/Wed, 15 Feb 2023 00:00:00 +0000https://tomroth.dev/compgraph2/This is the second post in a series on computational graphs. You can go back to the previous post or ahead to the next post. If you’d like to see what we are working towards in these posts, here is the Github link: Last time we looked at computational graphs and their use in autodiff packages. looked at the autodiff problem and the structure of the grad function in autograd showed how Python breaks down expressions to create computational graphs created a simple graph manually using a simplified Node class How do we automatically create a computational graph for a function?https://tomroth.dev/compgraph1/Tue, 14 Feb 2023 00:00:00 +0000https://tomroth.dev/compgraph1/This is the first post in a series on computational graphs. You can go to the next post or skip ahead to the third post. If you’d like to see what we are working towards in these posts, here is the Github link: Computation graphs have many uses. Here I’ll be presenting it as they used for autodifferentiation. For this post, I’ll assume you have some familiarity with backpropagation and autodifferentiation (also known as autodiff).https://tomroth.dev/backprop-toposort/Sun, 01 Dec 2019 00:00:00 +0000https://tomroth.dev/backprop-toposort/Neural networks need to update their weights with respect to a loss function. Rather than compute these weight updates manually, today we usually use automatic differentiation packages and tools. We call these autodiff systems. To do their job, autodiff systems construct computational graphs of the function they’re trying to find the gradient of. These computational graphs are key to backpropagation. I won’t go into how these graphs are constructed here.