Calculated file paths

MP 87: What are they, and why should you use them?

Note: I've been working on the styling of code blocks in technical posts. They should look better than they did previously, and be more aligned with what's discussed in the text. There's still some work to do; if they're unreadable for you, please let me know and I'll address it. You can reply to this email, or send a message to eric@mostlypython.com.

When I show people how to read and write files, I usually start with relative file paths. That's nice because the code looks simple, but the resulting program behaves inconsistently across different editors and environments.

When working with file paths, it's much safer and more consistent to work with calculated file paths. With a calculated file path, you tell Python how to find the file, and it works out the absolute path. Absolute paths almost always work, so your code should work wherever and however it's run.

Using a relative file path

Here's an example program that reads from a file, in about as little code as you can reasonably write:

from pathlib import Path

path = Path("coffees.txt")
contents = path.read_text()

print(contents)

Here's my directory layout:

I have a folder called projects/, where I keep a large collection of projects. Within that folder I have another folder called coffee_project/, which contains all the files for my latest project about coffee.

As long as the file coffees.txt is saved in the same directory as coffee_reader.py, this program will work in many cases. This is an example of a relative file path, where the location of coffees.txt is specified relative to the location of coffee_reader.py.

Here's an example of coffee_reader.py working in VS Code:

If you open the folder coffee_project/ in VS Code, the simple version of coffee_reader.py works.

Here's an example of coffee_reader.py not working in VS Code:

If you open the folder projects/ in VS Code, the simple version of coffee_reader.py no longer works. This time, it generates a FileNotFoundError.

Why do relative file paths sometimes fail?

There's a pretty brief explanation for why running coffee_reader.py sometimes works, and other times fails in an editor like VS Code. When you open the folder coffee_project/ in VS Code, it uses that folder as the working directory. It looks for coffees.txt in coffee_project/, and it's able to find it there.

When you open the folder projects/ in VS Code, it uses that folder as the working directory. It looks for coffees.txt in projects/, but the file doesn't exist in that folder.

Running from the terminal

It's helpful to see what happens when running this program from a terminal as well:

coffee_project$ python coffee_reader.py 
Hair Bender
El Inierto Bourbon
Trapper Creek

When you run coffee_reader.py from within the coffee_project/ directory, the program works.

But that changes if you run it from a different directory:

projects$ python coffee_project/coffee_reader.py 
Traceback (most recent call last):
  ...
FileNotFoundError: [Errno 2] No such file or directory: 'coffees.txt'

When you run it from the projects/ directory, you get the same FileNotFoundError we saw earlier.

Quick Fix: Use an absolute file path

A quick fix is to use the full file path on your system, called the absolute path. For me, the absolute path to coffees.txt is:

/Users/eric/projects/coffee_project/coffees.txt

Here's coffee_reader.py, using this path:

from pathlib import Path

path = Path(
    "/Users/eric/projects/coffee_project/coffees.txt")
contents = path.read_text()

print(contents)

This version of coffee_reader.py works regardless of what directory is open in VS Code, or what directory you run it from in a terminal:

projects$ python coffee_project/coffee_reader.py 
Hair Bender
El Inierto Bourbon
Trapper Creek

This solves the issue on my system, but it almost certainly won't work on anyone else's system. It's extremely unlikely they're going to have the same absolute path to coffees.txt, especially considering there's a username in this path.

Better fix: Calculating the full path

There's a much better, more resilient solution to the issue of how to define a file path in Python. To do this, we can use the special variable __file__, often read as "dunder file". This variable is a reference to the absolute path of the current .py file.

The __file__ variable only makes sense in the context of a .py file, so we can't experiment with it in a terminal session. Here's a simple file that lets us explore what we can do with __file__:

from pathlib import Path

path = Path(__file__)
print(path)

Here's the output, showing the full path to dunder_file_demo.py:

coffee_project$ python dunder_file_demo.py 
/Users/eric/projects/coffee_project/dunder_file_demo.py

coffee_project$ cd ..
projects$ python coffee_project/dunder_file_demo.py 
/Users/eric/projects/coffee_project/dunder_file_demo.py

The important thing to notice here is that the output is the same whether this is run from coffee_project/ or projects/. The value of __file__ is always the full path to that file, wherever it's run from.

Building a path to coffees.txt

Now that we know the path to coffees.py, we can figure out the path to coffees.txt. They're both in the same directory, which is the parent directory of coffees.py. Every Path object has a parent attribute:

from pathlib import Path

path = Path(__file__).parent
print(path)

Now path points to coffee_project/:

coffee_project$ python dunder_file_demo.py
/Users/eric/projects/coffee_project

Now that we have a path to the parent folder, we can reach coffees.txt:

from pathlib import Path

path = Path(__file__).parent / "coffees.txt"
print(path)

This generates the absolute path to coffees.txt:

coffee_project$ python dunder_file_demo.py
/Users/eric/projects/coffee_project/coffees.txt

Bringing this back to the original coffee_reader.py, we now have:

from pathlib import Path

path = Path(__file__).parent / "coffees.txt"
contents = path.read_text()

print(contents)

This version of coffee_reader.py works on any system, when run from any directory. It will work in VS Code regardless of how coffee_reader.py is accessed, and in any other Python editor as well.

Defining a project_root path

In most projects, there are multiple levels in the directory structure. It can be helpful to define a project_root path, and then build other paths from that root path.

Imagine we move the coffees.txt file to a folder called coffees/:

Here's how you might access coffees.txt in a real-world project:

from pathlib import Path

project_root = Path(__file__).parent
path = project_root / "coffees" / "coffees.txt"
contents = path.read_text()

print(contents)

Here we use __file__ and parent to get the path to the coffee project's root directory, coffee_projects/. We then build the path we want to work with by starting with project_root. In a larger project, project_root can be defined in one place, and all other path definitions will be much shorter and more readable.

Conclusions

Sometimes it can make sense to use relative file paths. If you're the only one working on a project, and you know relative file paths will work consistently with how you run your project, go ahead and use them.

If other people are going to use your project, or you're going to be running it on different systems, using a calculated file path will almost certainly be a much more resilient approach.

Resources

You can find the code files from this post in the mostly_python GitHub repository.