Module 6 · before
Learning = walking downhill 6 questions on what this module teaches. Guessing is fine: this is your starting point, not a test. Take the same check again at the end to see what you learned.
1. Predict: what does this print? def f(x):
return x * x * x
h = 0.001
print(round((f(2 + h) - f(2 - h)) / (2 * h), 2)) 6.0 24.0 8.0 12.0
2. Predict: the valley (x − 3)² has its bottom at x = 3. Starting at 0 with a learning rate of 1.0, where is the knob after 3 steps? def slope(x):
return 2 * (x - 3) # the slope of (x - 3) ** 2
x = 0.0
for step in range(3):
x = x - 1.0 * slope(x)
print(x) 6.0 -6.0 0.0 3.0
3. Gradient descent stops moving a knob once it reaches the bottom of the valley, even though nobody tells it to stop. Why? The slope there is 0, so the step lr × slope is 0 The loss has reached 0, so there is nothing left to learn The learning rate shrinks to 0 by itself near the bottom The knob has reached its largest allowed value
4. Predict: what does this print? import math
scores = [0.0, math.log(3)]
exps = [math.exp(s) for s in scores]
total = sum(exps)
print([round(e / total, 2) for e in exps]) [0.25, 0.75] [0.0, 1.0] [0.5, 0.5] [0.33, 0.67]
5. A model has 100 knobs, and you measure its gradient with two-sided nudges, (f(x + h) − f(x − h)) / 2h, for each knob. How many loss evaluations does one step cost? 200 2 100 101
6. A friend’s embedding model gives each of the 27 symbols 8 numbers in wte, and scores the 27 possible next symbols with lm_head. How many knobs does it have? 432 216 729 64
Answer all 6 to see your result Your answers are saved with a random id, not your name, so we can see which lessons work. Privacy