https://tomroth.dev/tags/neural-networks/Recent content in neural networks on Tom RothHugo -- gohugo.ioenSun, 01 Dec 2019 00:00:00 +0000https://tomroth.dev/backprop-toposort/Sun, 01 Dec 2019 00:00:00 +0000https://tomroth.dev/backprop-toposort/Neural networks need to update their weights with respect to a loss function. Rather than compute these weight updates manually, today we usually use automatic differentiation packages and tools. We call these autodiff systems. To do their job, autodiff systems construct computational graphs of the function they’re trying to find the gradient of. These computational graphs are key to backpropagation. I won’t go into how these graphs are constructed here.