Automate the Boring Stuff with Go, Ch. 8: Files
Part of Automate the Boring Stuff with Go, a beginner-friendly guide to programming in Go.
Updated for Go 1.27 in October 2026. What changed: this chapter was three code listings with no words. It now explains the handful of file operations everything else is built from, and the three projects are rewritten from scratch with tests.
In this chapter we focus on files: the management of the files on your own laptop or desktop.
Think of the mundane tasks you do with files: backing them up, finding the ones eating your disk, renaming a batch of them. Each is a few lines of Go once you know four operations: list a folder, walk a tree, read a file, and copy, move or delete one. The code is in the repo under ch8.
Let’s get going.
The four operations
List a folder
os.ReadDir gives you the entries of one folder, files and sub-folders, sorted by name:
// list one folder, files and sub-folders, with sizes
package main
import (
"fmt"
"os"
)
func main() {
entries, err := os.ReadDir(".")
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
for _, e := range entries {
info, err := e.Info()
if err != nil {
continue
}
kind := "file"
if e.IsDir() {
kind = "dir "
}
fmt.Printf("%s %8d %s\n", kind, info.Size(), e.Name())
}
}
An entry knows its name and whether it is a folder; e.Info() fetches the size and modification time when you need them.
Walk a tree
To visit everything under a folder, including sub-folders of sub-folders, use filepath.WalkDir. You give it a function and it calls that function once for every file and folder it finds:
// walk a whole tree and count files by extension
package main
import (
"fmt"
"io/fs"
"os"
"path/filepath"
)
func main() {
root := "."
if len(os.Args) > 1 {
root = os.Args[1]
}
counts := map[string]int{}
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
if err != nil {
return err // stop on the first folder we cannot read
}
if d.IsDir() && d.Name() == ".git" {
return fs.SkipDir // do not descend into git's internals
}
if !d.IsDir() {
counts[filepath.Ext(path)]++
}
return nil
})
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
fmt.Println(counts)
}
Returning fs.SkipDir from inside a folder tells the walk not to descend into it. Returning any other error stops the walk and hands that error back to you. Always use filepath.Join and friends rather than gluing paths together with +: they know about \ on Windows and / everywhere else.
Read and write a file
For a small file, os.ReadFile and os.WriteFile do the whole job in one call. For a file of any size, open it and read one line at a time with bufio.Scanner:
// read a file line by line, and write a file in one call
package main
import (
"bufio"
"fmt"
"os"
"path/filepath"
"strings"
)
func main() {
dir, err := os.MkdirTemp("", "read-lines")
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
defer os.RemoveAll(dir) // clean up the temp folder when main returns
path := filepath.Join(dir, "todo.txt")
// os.WriteFile creates or replaces the file in one call; 0o644 is "owner can write, all can read"
if err := os.WriteFile(path, []byte("buy milk\nwalk dog\n\nfix bike\n"), 0o644); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
f, err := os.Open(path)
if err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
defer f.Close()
// bufio.Scanner hands you one line at a time, however big the file is
sc := bufio.NewScanner(f)
n := 0
for sc.Scan() {
line := strings.TrimSpace(sc.Text())
if line == "" {
continue
}
n++
fmt.Printf("%d. %s\n", n, line)
}
if err := sc.Err(); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
Two habits to notice: defer f.Close() right after a successful os.Open, so the file is closed whatever happens later; and checking sc.Err() after the loop, because the scanner stops quietly when it hits a problem.
Copy, move, delete
// copy, move and delete, the three verbs of file management
package main
import (
"fmt"
"io"
"os"
"path/filepath"
)
// copyFile streams src into dst, so a 10 GB file never has to fit in memory.
func copyFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
if _, err := io.Copy(out, in); err != nil {
out.Close()
return err
}
return out.Close() // the Close error matters: it is when the last bytes hit the disk
}
func run() error {
dir, err := os.MkdirTemp("", "verbs")
if err != nil {
return err
}
defer os.RemoveAll(dir)
original := filepath.Join(dir, "notes.txt")
if err := os.WriteFile(original, []byte("keep this\n"), 0o644); err != nil {
return err
}
// copy
backup := filepath.Join(dir, "notes.bak")
if err := copyFile(original, backup); err != nil {
return fmt.Errorf("copy: %w", err)
}
// move (rename works across folders on the same disk; copy then delete otherwise)
archive := filepath.Join(dir, "archive")
if err := os.MkdirAll(archive, 0o755); err != nil {
return err
}
moved := filepath.Join(archive, "notes.txt")
if err := os.Rename(original, moved); err != nil {
return fmt.Errorf("move: %w", err)
}
// delete one file, then a whole tree
if err := os.Remove(backup); err != nil {
return fmt.Errorf("delete: %w", err)
}
entries, _ := os.ReadDir(dir)
for _, e := range entries {
fmt.Println("left in the folder:", e.Name())
}
return os.RemoveAll(archive)
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
io.Copy streams from a reader to a writer in chunks, so copying a 10 GB file never loads 10 GB. os.Rename is a move; it is instant on the same disk and fails across disks, in which case you copy and then delete. os.Remove deletes one file or empty folder; os.RemoveAll deletes a whole tree, and there is no undo, so look twice at what you pass it.
Back up files to a zip
The first project copies a whole folder into a numbered zip file next to it. Run it again and you get the next number, so you keep a history.
// backup-to-zip copies a whole folder into a numbered zip file next to it. Run it again
// and you get the next number, so you keep a history of backups.
//
// go run ./ch8/projects/backup-to-zip ./ch8/projects/backup-to-zip/sample
package main
import (
"archive/zip"
"errors"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"strings"
)
// NextZipName returns "<folder>_1.zip", or _2, _3, ... whichever does not exist yet.
func NextZipName(folder string) (string, error) {
folder = filepath.Clean(folder)
for n := 1; ; n++ {
name := fmt.Sprintf("%s_%d.zip", folder, n)
_, err := os.Stat(name)
if errors.Is(err, fs.ErrNotExist) {
return name, nil
}
if err != nil {
return "", fmt.Errorf("check %s: %w", name, err)
}
}
}
// BackupToZip writes every regular file under folder into zipPath, with paths relative
// to the folder's parent so the archive unpacks into a folder of the same name.
func BackupToZip(folder, zipPath string) (count int, err error) {
out, err := os.Create(zipPath)
if err != nil {
return 0, fmt.Errorf("create %s: %w", zipPath, err)
}
defer func() {
if cerr := out.Close(); err == nil && cerr != nil {
err = cerr
}
}()
zw := zip.NewWriter(out)
base := filepath.Base(filepath.Clean(folder))
err = filepath.WalkDir(folder, func(path string, d fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if d.IsDir() {
return nil
}
rel, err := filepath.Rel(folder, path)
if err != nil {
return err
}
if err := addFile(zw, path, filepath.ToSlash(filepath.Join(base, rel))); err != nil {
return err
}
count++
return nil
})
if err != nil {
return count, fmt.Errorf("walk %s: %w", folder, err)
}
if err := zw.Close(); err != nil {
return count, fmt.Errorf("finish zip: %w", err)
}
return count, nil
}
func addFile(zw *zip.Writer, path, nameInZip string) error {
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
w, err := zw.Create(nameInZip)
if err != nil {
return err
}
_, err = io.Copy(w, f)
return err
}
func run(args []string) error {
if len(args) != 1 {
return errors.New("usage: backup-to-zip <folder>")
}
folder := strings.TrimSuffix(args[0], string(filepath.Separator))
info, err := os.Stat(folder)
if err != nil {
return err
}
if !info.IsDir() {
return fmt.Errorf("%s is not a folder", folder)
}
zipPath, err := NextZipName(folder)
if err != nil {
return err
}
n, err := BackupToZip(folder, zipPath)
if err != nil {
return err
}
fmt.Printf("backed up %d file(s) to %s\n", n, zipPath)
return nil
}
func main() {
if err := run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
$ go run ./ch8/projects/backup-to-zip ./ch8/projects/backup-to-zip/sample
backed up 4 file(s) to ch8/projects/backup-to-zip/sample_1.zip
NextZipName is the walk-a-tree idea in miniature: try _1.zip, _2.zip, until one does not exist. BackupToZip is WalkDir plus archive/zip, with each file’s path made relative so the archive unpacks into a folder with the original’s name.
Find files taking up lots of space
The second project walks a folder and lists every file at or above a size, largest first. Sizes read the way people write them, 500KB, 100MB, 2GB.
// find-large-files walks a folder and lists every file at or above a size, largest
// first. Sizes read the way people write them: 500KB, 100MB, 2GB.
//
// go run ./ch8/projects/find-large-files -min 100MB ~/Downloads
package main
import (
"errors"
"flag"
"fmt"
"io/fs"
"os"
"path/filepath"
"slices"
"strconv"
"strings"
)
// Entry is one file that met the threshold.
type Entry struct {
Path string
Size int64
}
var units = []struct {
suffix string
mult int64
}{
{"GB", 1 << 30}, {"MB", 1 << 20}, {"KB", 1 << 10}, {"B", 1},
}
// ParseSize turns "100MB", "2gb", "512" (bytes) into a byte count.
func ParseSize(s string) (int64, error) {
s = strings.ToUpper(strings.TrimSpace(s))
for _, u := range units {
if num, ok := strings.CutSuffix(s, u.suffix); ok {
n, err := strconv.ParseFloat(strings.TrimSpace(num), 64)
if err != nil || n < 0 {
return 0, fmt.Errorf("bad size %q", s)
}
return int64(n * float64(u.mult)), nil
}
}
n, err := strconv.ParseInt(s, 10, 64)
if err != nil || n < 0 {
return 0, fmt.Errorf("bad size %q", s)
}
return n, nil
}
// HumanSize formats bytes with one decimal and the largest fitting unit.
func HumanSize(n int64) string {
for _, u := range units {
if n >= u.mult && u.mult > 1 {
return fmt.Sprintf("%.1f %s", float64(n)/float64(u.mult), u.suffix)
}
}
return fmt.Sprintf("%d B", n)
}
// FindLarge returns the files under root whose size is at least min, largest first.
// Folders it cannot read are skipped, not fatal.
func FindLarge(root string, minSize int64) ([]Entry, error) {
var found []Entry
err := filepath.WalkDir(root, func(path string, d fs.DirEntry, walkErr error) error {
if walkErr != nil {
if d != nil && d.IsDir() {
return fs.SkipDir
}
return nil
}
if d.IsDir() {
return nil
}
info, err := d.Info()
if err != nil {
return nil
}
if info.Size() >= minSize {
found = append(found, Entry{Path: path, Size: info.Size()})
}
return nil
})
if err != nil {
return nil, fmt.Errorf("walk %s: %w", root, err)
}
slices.SortFunc(found, func(a, b Entry) int { return int(b.Size - a.Size) })
return found, nil
}
func run(args []string) error {
flags := flag.NewFlagSet("find-large-files", flag.ContinueOnError)
minFlag := flags.String("min", "100MB", "smallest size to report")
if err := flags.Parse(args); err != nil {
return err
}
if flags.NArg() != 1 {
return errors.New("usage: find-large-files [-min 100MB] <folder>")
}
minSize, err := ParseSize(*minFlag)
if err != nil {
return err
}
entries, err := FindLarge(flags.Arg(0), minSize)
if err != nil {
return err
}
for _, e := range entries {
fmt.Printf("%10s %s\n", HumanSize(e.Size), e.Path)
}
fmt.Fprintf(os.Stderr, "%d file(s) at or above %s\n", len(entries), HumanSize(minSize))
return nil
}
func main() {
if err := run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
$ go run ./ch8/projects/find-large-files -min 1KB ./ch9
7.2 KB ch9/projects/census-by-county/census.xlsx
6.9 KB ch9/projects/update-produce-prices/produceSales.xlsx
6.5 KB ch9/projects/read-workbook/example.xlsx
...
12 file(s) at or above 1.0 KB
(Point it at ~/Downloads with -min 100MB for the real use.)
Two choices worth copying into your own programs. A folder you are not allowed to read is skipped, not fatal: for a disk-cleanup tool, one unreadable system folder should not stop the report. And the size parsing lives in its own function, ParseSize, with a table of units, which is what makes it easy to test (look at main_test.go next to it).
Rename dates on files
The third project finds file names containing an American date, MM-DD-YYYY, and renames them to the European order, DD-MM-YYYY. Renaming is the kind of thing you want to see before you do, so it only prints the plan unless you pass -apply.
// rename-dates finds file names that contain an American date (MM-DD-YYYY) and renames
// them to the European order (DD-MM-YYYY). It only prints the plan unless you pass
// -apply, because renaming is the kind of thing you want to see before you do it.
//
// go run ./ch8/projects/rename-dates ./reports
// go run ./ch8/projects/rename-dates -apply ./reports
package main
import (
"errors"
"flag"
"fmt"
"os"
"path/filepath"
"regexp"
)
// americanDate matches 05-21-2026 and 5-21-2026, with the surrounding text captured so
// the rest of the name survives.
var americanDate = regexp.MustCompile(`^(.*?)(\d{1,2})-(\d{1,2})-(\d{4})(.*)$`)
// EuropeanName returns the file name with its date flipped to DD-MM-YYYY, and whether the
// name contained a plausible American date at all.
func EuropeanName(name string) (string, bool) {
m := americanDate.FindStringSubmatch(name)
if m == nil {
return name, false
}
before, month, day, year, after := m[1], m[2], m[3], m[4], m[5]
if !plausible(month, 1, 12) || !plausible(day, 1, 31) {
return name, false
}
return before + day + "-" + month + "-" + year + after, true
}
func plausible(field string, lo, hi int) bool {
n := 0
for _, r := range field {
n = n*10 + int(r-'0')
}
return n >= lo && n <= hi
}
// Plan lists the renames that would happen in folder (non-recursive).
func Plan(folder string) (map[string]string, error) {
entries, err := os.ReadDir(folder)
if err != nil {
return nil, fmt.Errorf("read %s: %w", folder, err)
}
plan := map[string]string{}
for _, e := range entries {
if e.IsDir() {
continue
}
if newName, ok := EuropeanName(e.Name()); ok && newName != e.Name() {
plan[e.Name()] = newName
}
}
return plan, nil
}
func run(args []string) error {
flags := flag.NewFlagSet("rename-dates", flag.ContinueOnError)
apply := flags.Bool("apply", false, "actually rename (default: print the plan)")
if err := flags.Parse(args); err != nil {
return err
}
if flags.NArg() != 1 {
return errors.New("usage: rename-dates [-apply] <folder>")
}
folder := flags.Arg(0)
plan, err := Plan(folder)
if err != nil {
return err
}
if len(plan) == 0 {
fmt.Println("nothing to rename")
return nil
}
for from, to := range plan {
fmt.Printf("%s -> %s\n", from, to)
if !*apply {
continue
}
if err := os.Rename(filepath.Join(folder, from), filepath.Join(folder, to)); err != nil {
return fmt.Errorf("rename %s: %w", from, err)
}
}
if !*apply {
fmt.Println("(dry run; pass -apply to rename)")
}
return nil
}
func main() {
if err := run(os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
$ go run ./ch8/projects/rename-dates ./reports
invoice_05-21-2026.pdf -> invoice_21-05-2026.pdf
(dry run; pass -apply to rename)
The regex captures the text before the date, the three date parts, and the text after, so the rest of the name survives. plausible rejects month 13 and day 40, which keeps a part number like 13-40-2026 from being mangled. Dry run by default is a habit worth keeping for any program that changes files: it costs one flag and saves you from the one time the pattern matches something you did not expect.
Conclusion
List, walk, read, copy-move-delete: that is the whole toolkit, and the three projects are those four operations arranged differently. From here on, every chapter reads or writes files of some kind.