Automating System-Level Tasks

Python is an excellent glue language: it runs other programs, copies and renames files, walks directories, sets environment variables, and makes small scripts into durable automation. The relevant modules are os, pathlib, shutil, subprocess, sys and platform. Between them you can replace most shell one-liners with Python code that is cross-platform and easy to reason about.

For anything file- or path-related, start with pathlib. Path.home(), Path.cwd(), p.iterdir(), p.glob("*.py"), p.rglob("**/*.txt") give you most of what os.path + os.listdir used to provide, but in a clearer object-oriented style.

Running another program is the job of subprocess. subprocess.run(["ls", "-la"], check=True, capture_output=True, text=True) is the modern one-stop API. check=True raises if the process returns a non-zero exit code; capture_output=True makes result.stdout available; text=True decodes output as str instead of bytes.

For copying, moving and deleting, shutil wraps the fiddly platform details: shutil.copy2 preserves metadata, shutil.move handles cross-device moves, shutil.rmtree removes a folder recursively. Use them instead of writing file-by-file loops.

Walking directories and reading env

Path.iterdir() lists a directory's immediate children; Path.rglob("pattern") walks recursively. Combine with p.is_file(), p.is_dir(), and p.stat().st_size to build real tools. For very large trees, os.scandir is faster but more low-level.

Environment variables live in os.environ, a regular dict. Read with os.environ.get("VAR", "default"); setting os.environ["VAR"] = "x" affects only this process and its future children.

Calling other programs safely

Pass the command as a list of strings (["git", "status"]), not a single string. This skips shell quoting pitfalls and avoids command injection. If you truly need the shell, set shell=True and pass only trusted input.

Prefer subprocess.run(..., check=True) over os.system. It gives you return codes, captured output, and a proper exception on failure. Use subprocess.Popen only for long-running or streaming processes.

System-task tools.

Automating System-Level Tasks code example

The script creates a small folder tree, inspects it, runs an external Python command, and cleans up.

# Lesson: Automating System-Level Tasks
import os
import platform
import shutil
import subprocess
import sys
from pathlib import Path
from tempfile import TemporaryDirectory


print("python :", sys.version.split()[0])
print("system :", platform.system(), platform.release())
print("home   :", Path.home())
print("cwd    :", Path.cwd())
print("env PATH first 60 chars:", os.environ.get("PATH", "")[:60], "...")

with TemporaryDirectory() as tmp:
    root = Path(tmp)
    (root / "a").mkdir()
    (root / "a" / "hello.txt").write_text("hi", encoding="utf-8")
    (root / "b").mkdir()
    (root / "b" / "note.py").write_text("print('from b')", encoding="utf-8")

    # Walk recursively
    for p in sorted(root.rglob("*")):
        marker = "DIR " if p.is_dir() else f"{p.stat().st_size:3d}B"
        print(f"  {marker}  {p.relative_to(root)}")

    # Copy and move
    shutil.copy2(root / "a" / "hello.txt", root / "b" / "hello_copy.txt")
    print("after copy:", sorted(p.name for p in (root / "b").iterdir()))

    # Run a child Python program
    result = subprocess.run(
        [sys.executable, "-c", "print('hello from subprocess')"],
        check=True, capture_output=True, text=True,
    )
    print("subprocess:", result.stdout.strip())
    print("returncode:", result.returncode)

print("temp cleaned:", not Path(tmp).exists())

Four tasks in one script:

1) `platform`/`sys` give you runtime facts without touching the filesystem.
2) `pathlib.rglob` walks the tree; `p.stat().st_size` queries file size.
3) `shutil.copy2` preserves metadata during copy.
4) `subprocess.run(check=True)` raises on non-zero exit, so it fails loudly.

Automate a small file search and an external command.

import subprocess, sys
from pathlib import Path

# Example A: count lines across all .py files under a folder
def count_lines(root: Path) -> int:
    return sum(1 for f in root.rglob("*.py") for _ in f.read_text(encoding="utf-8").splitlines())

# Example B: run a second Python snippet and grab stdout
result = subprocess.run(
    [sys.executable, "-c", "import sys; print(sys.version_info[:2])"],
    capture_output=True, text=True, check=True,
)
print("child said:", result.stdout.strip())

Small checks for pathlib and platform.

from pathlib import Path
import sys
assert Path("a/b.txt").suffix == ".txt"
assert Path("a") / "b.txt" == Path("a/b.txt") or Path("a") / "b.txt" == Path("a\\b.txt")
assert isinstance(sys.version_info.major, int)

Output depends on your OS but looks like:

python : 3.12.4
system : Linux 6.5.0
home   : /home/you
cwd    : /home/you/demo
env PATH first 60 chars: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/ ...
  DIR   a
    2B  a/hello.txt
  DIR   b
   15B  b/note.py
after copy: ['hello_copy.txt', 'note.py']
subprocess: hello from subprocess
returncode: 0
temp cleaned: True