Module 7 · before

Autograd = automatic chain rule

6 questions on what this module teaches. Guessing is fine: this is your starting point, not a test. Take the same check again at the end to see what you learned.

1. Predict: y = x × x + x at x = 2 reaches x by three roads, with rates 2, 2 and 1. This loop tries to collect x’s gradient. What does it print?

x_grad = 0
for rate in [2, 2, 1]:
    x_grad = rate
print(x_grad)

2. Predict: a / b is built as a × b⁻¹. What does this print?

class Value:
    def __init__(self, data, children=(), local_grads=()):
        self.data = data
        self._local_grads = local_grads
    def __pow__(self, n):
        return Value(self.data ** n, (self,), (n * self.data ** (n - 1),))

y = Value(4.0) ** -1
print(y.data, y._local_grads)

3. Predict: L = a × b + a. What does this print?

class V:
    def __init__(self, data, kids=(), rates=()):
        self.data, self.grad, self.kids, self.rates = data, 0, kids, rates
    def __add__(self, o): return V(self.data + o.data, (self, o), (1, 1))
    def __mul__(self, o): return V(self.data * o.data, (self, o), (o.data, self.data))

a, b = V(3.0), V(4.0)
c = a * b
L = c + a
L.grad = 1
for v in [L, c]:   # reverse topological order
    for kid, rate in zip(v.kids, v.rates):
        kid.grad += rate * v.grad
print(a.grad, b.grad)

4. Your backward() runs without errors, but every gradient, even for the loss’s direct inputs, comes out 0. Which line is most likely missing?

5. Predict: no Value class here, just nudging. Use the chain rule to work out what it prints.

def z(x):
    y = 2 * x + 1
    return y * y

h = 1e-6
print(round((z(1 + h) - z(1 - h)) / (2 * h), 3))

6. You compute s = a + b once, then L = s × s. How many distinct nodes are in the graph, and what would counting give without a visited set?

Your answers are saved with a random id, not your name, so we can see which lessons work. Privacy