Modules

Python modules are one of the main abstraction layer available and probably the

most natural one. Abstraction layers allow separating code into parts holding

related data and functionalities.

For example, a layer of a project can handle interfacing with user actions,

while another would handle low-level manipulation of data. The most natural way

to separate these two layers is to regroup all interfacing functionalities

in one file, and all low-level operations in another file. In this case,

the interface file need to import the low-level file. This is done with the

`import` and `from ... import` statements.

As soon as you use `import` statements you use modules, either builtin modules

such as `os` and `sys`, or third-party modules you have installed in your

environment, or project's internal modules.

Nothing special is required for a Python file to be a module, but the import

mechanism need to be understood in order to use this concept properly and avoid

some issues.

Concretely, the `import modu` statement will look for the proper file, which is

`modu.py` in the same directory as the caller if it exists. If it is not

found, the Python interpreter with search for `modu.py` in the "path"

recursively and raise an ImportError exception if it is not found.

Once `modu.py` is found, the Python interpreter will execute the module in an

isolated scope. Any top-level statement in `modu.py` will be executed,

including other imports if any. Function and classes definitions are stored in

the module's dictionary.

Then modules variables, functions and classes will be available to the caller

through the module's namespace, a central concept in programming that is

particularly helpful and powerful in Python.

In many languages, a `include file` directive is used by the preprocessor to

take all code found in the file and 'copy' it in the caller's code. It is

different in Python: the included code is isolated in a module namespace, which

means that you generally don't have to worry that the included code could have

unwanted effect, eg override an existing function with the same name.

It is possible to simulate the more standard behavior by using a special syntax

of the import statement: `from modu import *`. This is generally considered bad

practice, **using import * makes code harder to read and dependencies less

compartimented**.

Using `from modu import func` is a way to pinpoint the function you want to

import and put it is the global namespace. While much less harmful than `import

*` because it shows explicitely what is imported in the global namespace, it's

advantage over a simpler `import modu` is only that it will save some typing.

**Very bad**

.. code-block:: python

**Better**

.. code-block:: python

**Best**

.. code-block:: python

As said in the section about style, readability is one of the main feature of

Python. Readability means to avoid useless boilerplate text and clutter,

therefore some efforts are spent trying to achieve a certain level of brevity.

But terseness and obscurity are the limits where brevity should stop: being

able to tell immediately from where comes a class or a function, as in the

`modu.func` idiom, improves greatly code readability and understandability in

most cases but the simplest single file projects.

Packages --------

Python provides a very straightforward packaging system, which is simply an

extension of the module mechanism to a directory.

Any directory with a __init__.py file is considered a Python package. The

different modules in the package are imported in a similar manner as plain

modules, will a special behavior for the __init__.py file, that is used to

gather all package-wide definitions.

A file modu.py in the directory pack/ is imported with the statement `import

pack.modu`. This statement will look for a __init__.py file in `pack`, execute

all its top-level statements. Then it will look for a file `pack/modu.py` and

execute all its top-level statements. After these operations, any variable,

function or class defined in modu.py is available in pack.modu namespace.

A commonly seen issue is to add too many code and functions in __init__.py

files. When the project complexity grows, there may be sub-packages and

sub-sub-packages in a deep directory structure, and then, import a single item

from a sub-sub-package will require to execute all __init__.py file met while

descending the tree.

Leaving a __init__.py file empty is considered normal and even a good pratice,

if the package's modules and sub-packages do not need to share any code.

Lastly, a convenient syntax is available for importing deeply nested packages:

`import very.deep.module as mod` allow to use `mod` in place of the verbose

repetition of `very.deep.module` in front of each calls to module items.