Managing Code Packages

A package in Python is a folder of modules treated as a single importable unit. When you run import project.utils, Python finds a folder named project and a file named utils.py inside it. Packages are how you keep a growing codebase organized once a single .py file stops being enough.

The classic recipe is simple: create a folder, drop an empty __init__.py inside, and put your modules there. The __init__.py tells Python "this folder is a regular package". Since Python 3.3 you can also use namespace packages (folders without __init__.py), but regular packages are easier to reason about and what most projects use.

Packages nest. A folder called project/api/routes.py is imported as project.api.routes. Inside, you use either absolute imports (from project.api.routes import get_user) or relative ones (from .routes import get_user). Absolute imports are preferred for application code; relative imports are handy inside tightly coupled sub-packages.

Beyond organizing code, packages are how you publish libraries. Once you write a pyproject.toml that names your package, tools like build and twine (or hatch / flit) package it into a wheel and upload it to PyPI. The details are out of scope for this lesson; the key idea is that the same directory layout powers both in-project organization and distribution.

Creating a package and its layout

The conventional layout is a top-level folder that shares the project name, an inner folder with the source package name, and tests in a sibling folder. Example: myproj/src/myproj/__init__.py plus myproj/tests/. The src/ layout prevents accidental imports of unpackaged code during development.

Sub-packages are just more folders with __init__.py. Group by feature, not by layer: myproj/auth, myproj/payments age much better than myproj/models, myproj/views.

Controlling the public API

Anything defined in a module's top level is importable from outside. You can make the public surface explicit with __all__ = ["foo", "Bar"] at the top of the module; from module import * then imports only those names. Even without *, __all__ is a documentation hint: “these are the intended exports”.

Keep private helpers underscored: _internal_thing. That single underscore is a widely respected convention meaning "not part of the public API". Linters and IDEs honor it.

The pieces that make a folder into a package.

Managing Code Packages code example

The script builds a tiny package in a temporary directory, imports from it, and demonstrates __all__.

# Lesson: Managing Code Packages
import importlib
import sys
import textwrap
from pathlib import Path
from tempfile import TemporaryDirectory


with TemporaryDirectory() as tmp:
    pkg = Path(tmp) / "mypkg"
    pkg.mkdir()
    (pkg / "__init__.py").write_text(
        '__all__ = ["greet"]\nfrom .core import greet, _secret\n',
        encoding="utf-8",
    )
    (pkg / "core.py").write_text(textwrap.dedent('''
        def greet(name: str) -> str:
            return f"hello {name}"

        def _secret() -> str:
            return "hidden"
    ''').strip(), encoding="utf-8")

    sys.path.insert(0, str(tmp))
    try:
        mypkg = importlib.import_module("mypkg")
        print("greet:  ", mypkg.greet("ana"))
        print("secret?:", mypkg._secret())  # still accessible, just conventionally private
        print("__all__:", mypkg.__all__)

        # `from mypkg import *` only brings in names in __all__
        ns: dict = {}
        exec("from mypkg import *", ns)
        public = [n for n in ns if not n.startswith("__")]
        print("imported by *:", public)
    finally:
        sys.path.pop(0)
        for key in [k for k in sys.modules if k.startswith("mypkg")]:
            del sys.modules[key]

print("temp dir cleaned up:", not Path(tmp).exists())

Study the script step by step:

1) Creating `__init__.py` is what makes `mypkg` a package.
2) The package re-exports `greet` from a submodule; consumers don't care where it lives.
3) `_secret` is importable but the underscore signals 'private' to readers and tools.
4) `__all__` controls `from pkg import *`; it is a hint and a tool, not a hard wall.

Practice a two-module package with a private helper.

# Typical layout (pseudo file tree):
#   myproj/
#     __init__.py        # """public API"""
#     core.py            # public functions
#     _utils.py          # private helpers
#
# myproj/__init__.py
#     from .core import greet
#     __all__ = ["greet"]
#
# myproj/core.py
#     from . import _utils
#     def greet(name):
#         return _utils.format("hello", name)
#
# myproj/_utils.py
#     def format(prefix, name):
#         return f"{prefix}, {name}!"
print("layout sketch above")

Assertions focused on import semantics.

import sys
import importlib
# Plain standard library imports (already packages)
assert importlib.import_module("json") is sys.modules["json"]
assert "encode" in dir(importlib.import_module("json"))
assert hasattr(importlib.import_module("json"), "__path__") is False  # json is a module, not a package

Running prints something like:

greet:   hello ana
secret?: hidden
__all__: ['greet']
imported by *: ['greet']
temp dir cleaned up: True