1.5 Reading a dataset

open('names.txt') opens the file; looping over it gives one line at a time. Each line (except the very last) ends with an invisible newline character, written '\n'. So the first line is 'emma\n': it looks like 4 letters, but its length is 5.

line.strip() removes spaces and newlines from both ends. It gives back a new, clean text and leaves line unchanged, so save the result: name = line.strip(). A line that was blank (or only spaces) strips down to the empty text '', and if name != '': skips it.

In the lab you put this loop inside a function, load(path), so it works on any file. The checks also try it on a small messy file with blank lines.

microgpt writes the same loop as one compact line (line 19). This is the kind of line you ran in lesson 1.1. Here are the two side by side:

docs = []
for line in open('input.txt'):          # the loop
    if line.strip():                    # the check
        docs.append(line.strip())       # what goes in

docs = [line.strip()                    # what goes in
        for line in open('input.txt')   # the loop
        if line.strip()]                # the check

That’s a list comprehension: a for loop that builds a list, written as “what goes in, the loop, the check” inside square brackets. (if line.strip(): is short for “if what’s left isn’t empty”.) microgpt calls its file input.txt; here it’s names.txt, the same 32,033 names. Lines 15 to 18 download the file if it’s missing; in your browser it’s already there.

Fill the loop inside load: strip each line, and if what’s left isn’t empty, append it to docs. Press Check my code: you should have 32,033 names, and the small messy file should come back as ['ann', 'bob', 'cy'].

Lines 14 to 19 and 21: microgpt’s dataset section. (Line 20 shuffles it; that’s lesson 3.4.)