Tracking Issue for normalize_lexically

Feature gate: #![feature(normalize_lexically)]

This is a tracking issue for Path::normalize_lexically that is used to normalize a path, including .. parent references, without touching the filesystem.

Public API

// std::path

impl Path {
    pub fn normalize_lexically(&self) -> Result<PathBuf, NormalizeError> {
}

Steps / History

API Change Proposal
Implementation: Implement normalize_lexically #134696
Final comment period (FCP)¹
Stabilization PR

Unresolved Questions

Should PathBuf have a method that normalizes in-place? Normalizing a path just removes components.
Should this disallow resolving to the empty path or the single . path.
Should this only allow paths without a root or prefix? It may be surprising that base_path.join(user_path.normalize_lexically()?) can still escape the base path.

The last two can be summarised as: should this strictly require paths to be sub-paths?

https://std-dev-guide.rust-lang.org/feature-lifecycle/stabilization.html ↩

Rollup merge of rust-lang#134696 - ChrisDenton:normalize-lexically, r…

Unrolled build for #134696

The basic implementation is now implemented in nightly if people would like to experiment.

Rollup merge of rust-lang#134696 - ChrisDenton:normalize-lexically, r…

Should this take &mut self and normalize in-place? Normalizing a path just removes components.

This could be done for PathBuf, but &mut Path cannot actually delete anything, since the resulting path has to be the same length. Maybe it could be done in a way similar to partition_dedup where all the components are moved to the end, but that would be confusing and a PathBuf-based method sounds much better API-wise.

We could have a function both on Path and PathBuf. The only issue would be choosing names to distinguish them. normalize_lexically is already a bit of a mouthful (somewhat intentionally since it shouldn't be what people reach for by default).

How would you do a function on Path that removes components of the path in-place, given what I mentioned?

No, I meant have a function on Path that returns a PathBuf and a separate function on PathBuf that does in-place normalisation.

Ah, yeah, that makes sense. I was mostly commenting on the note at the end that taking &mut self would only work on PathBuf, meaning that it would be worthwhile to have both.

I've updated it to suggest adding a method to PathBuf instead. If anyone wants to implement that then feel free, I probably won't get to it for awhile.

I'm not sure that it makes sense for this operation to be fallible. Given an input of "abc/../../def", an output of "../def" seems perfectly reasonable. Even /../etc, which seems like it should be an error, is silently resolved to /etc on Linux.

There is absolutely a strong use case for checking if a path escapes the pwd, but normalize_lexically:

a) isn't a one-stop shop for this because it will happily represent absolute paths; and
b) isn't an efficient solution to that problem because it allocates.

With that in mind, I suggest always returning a PathBuf rather than Result<PathBuf, _>.

On that note, does it make sense to return a Cow<'_, Path> to avoid allocating when no change is made? This is already done for eg. with_trailing_sep().

I definitely agree that it would be reasonable for this API to return Cow<'_, Path>, alongside my previous recommendation to add a version that just applies to PathBuf in-place.

Also in agreement that it would be reasonable for the final result to contain relative paths, assuming that any .. are at the beginning of the path. I mean, after all, abc is a relative path.

There was no intention to make this API to touch filesystem at all.

My point is that lexical normalisation is a part of link traversal, but not all of it. And it's better to move this responsibility to the caller as all situations might be different.

E.g. .. in one context could mean "went beyond safety boundary, in other case it's just a path on a filesystem.

If there's an intention to have such api, there's a method of Path to check if it's a relative path which fully covers the second safety part in their interpretation.

The only path which is in a gray area is empty path which may or may not be considered as equivalent of ..

@eirnym

@eirnym i still guess I don't totally follow. If you want to represent the traversal there's std::fs::canonicalize.

E.g. .. in one context could mean "went beyond safety boundary, in other case it's just a path on a filesystem.

Really, the safety provided by lexical normailzation is for containment is limited as even a normalized path could have a symlink that points to a prior path. I still think it's valuable in an application to reduce/remove .. as it makes reasoning about the logic easier. It reduces the entropy. But it can also cut the other way i.e. a .. can "escape" a symlink such that folding it lexically does not produce an equivalent path as a physical traversal. Ruby's File.expand_path both follows physical links and folds as it goes. I think that's safer but I think there's some subtle dragons there too as you can wind up in situations where the traversal stops due to a problem like a broken symlink.

Most languages seem to have a number of slightly different APIs with slightly different behavior. That's why I'm suggesting we try to lock down what "is" and isn't the job of this current feature. That way we can have strong confidence when a PR comes in to accept or reject it. And if someone needs a thing that isn't provided here, it would be a hint to start the process over with a new API with a new stated purpose and goal. I don't think we can have a single API make everyone 100% happy for all their needs.

The only path which is in a gray area is empty path which may or may not be considered as equivalent of ..

There's a few ambiguous cases in the existing implementation beyond this one (spelled out some differences in a few of my links). I think it's more useful to define this behavior and declare it produces something other programs can use.

The only think I'd like that lexical normalisation won't think for me in what context I'd like to use it. I'd like that relative paths were valid, even they follow above current folder.

While most of the choices seam reasonable/acceptable I have a few notes.

Absolute Escape

If lexical normalization should return an Err for any path it should be ̀the absolute root escapes, as that can't be check easily before or after.

I.e.

Absolute escape "/../.." → "/" (pin to root)

Absolute escape multiple "/../../../a" → "/a" (pin to root)

should result in Err.

Relative Escape

For me a relative path can fail to be normalized .. is perfectly fine at the beginning of a normalized path.
So rather than

So a normalized relative path ALWAYS starts with a . and …

A normalized relative path either starts with either a "." or at least one ".." .
"." may only appear as the first component (after an optional prefix)
".." may only appear as the first component (after an optional prefix) or after itself

I.e.

Relative escape ”.." → Err
Relative escape ”../anything/else" → Err

shouldn't result in Err. If someone wants to reject these checking for .. after normalization can be done easily.
Though if you insist that these should Err as long as the Err provides the what you call a partially normalized path I could live with that.

Plain relative

I don't think it is good that we deviate here from all other referenced implementation

Plain relative "a" → "./a"

Though I don't feel strongly about it as it should be easy to check for so that one can skip normalization in that case.

Paths with Prefix

The examples are missing e.g. windows paths with prefix.
Basically I think it should act as
1. if the path has a prefix remove it
2. perform normalization as described before
3. if 1. removed a prefix add it back

I've started a discussion with the libs team on zulip. I'd like to keep that discussion focused on the broad objective of this method rather than the specific details. Without some agreement on what normalize_lexically is for we're unlikely to make progress.

Nomination for meeting discussion, with nomination shifting from #159863 to this.

So, collecting feedback from libs meeting:

We are in agreement that this should ignore symlinks, and that a/b/.. is always a.
We are in agreement that there are no parents of the root, so, /.. is just /. (+ for other prefixes as well, noting that C:.. is not C:\)
We know that this is not enough for security, and that you should not rely solely on this for security reasons. We could potentially find a resource to link here detailing some of the potential issues and why this isn't enough.

We haven't figured out trailing slashes, yet, and it feels like this might be a place where we want to have extra options, or just not trim the trailing separator and let users do it themselves.

Just to be explicit, the feeling in the libs meeting is that this function should just normalize the path and not do anything else, like restrict it to the root. E.g. a/../.. should be ../.

As clarafon said, security of paths is a much more complex issue and we can't handle them all in one function. It's also not specific to lexical normalization. This kind of normalization may be one component of securing paths but it may also not be. Even in the case that it is part of securing paths, this one function can't do everything people want without growing a lot of options so it's better for it to do one thing so it can then be composable.

path: Replace `normalize` with component-based normalization in `Path…

A few thoughts following on from the libs team review (specifically, "do one thing", "not a one-stop security solution", "don't restrict to [pwd]", "no parents of root"): it seems to refute the use cases for returning either an enum or a Result from normalize_lexically.

I favour splitting the descriptive use cases into two new methods:

impl Path {
    pub fn is_normalized(&self) -> bool;
    pub fn lexical_bound(&self) -> LexicalBound;
    // And change the signature to: 🐮
    pub fn normalize_lexically(&self) -> Cow<'_, Path>;
}
pub enum LexicalBound {
    Absolute, // "/xyz"
    Outer, // "../xyz"
    Pwd, // "." (only)
    Inner, // "xyz"
    Empty, // "" (only)
}

Checking is_normalized and lexical_bound is trivial and can always be done without allocating. Users should be encouraged to use these lightweight descriptive functions except when they're going to use the output of the heavy transformative function.
Given an ideal scenario where all files and dirs exist and none are symlinks, the normalized path and un-normalized path should behave identically, ie. we should preserve trailing slashes because File::open(path) and File::open(path.normalize_lexically()) should return the same NotADirectory error (assuming the input has a trailing slash).
If is_normalized() == true, normalize_lexically should always return the path unchanged.
No one is going to agree on what is and is not a valid path because everyone's use case is different. Since there's a reasonable output to any input, normalize_lexically should be infallible, and further decisions about what is and is not valid for a particular use case should be made using the existing and proposed descriptive functions.
path.lexical_bound() == path.normalize_lexically().lexical_bound() in all cases.

Navigation Menu

Repository navigation

Tracking Issue for `normalize_lexically` #134694

Public API

Steps / History

Unresolved Questions

44 remaining items

Absolute Escape

Relative Escape

Plain relative

Paths with Prefix

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Tracking Issue for normalize_lexically #134694

Description

Public API

Steps / History

Unresolved Questions

Footnotes

Activity

ChrisDenton commented on May 27, 2025

clarfonthey commented on Aug 13, 2025

ChrisDenton commented on Aug 13, 2025

clarfonthey commented on Aug 13, 2025

ChrisDenton commented on Aug 13, 2025

clarfonthey commented on Aug 13, 2025

ChrisDenton commented on Aug 13, 2025

MikkelPaulson commented on Feb 11, 2026

clarfonthey commented on Feb 12, 2026

44 remaining items

eirnym commented on Sep 11, 2026

schneems commented on Sep 11, 2026

eirnym commented on Sep 11, 2026

Skgland commented on Sep 11, 2026

Absolute Escape

Relative Escape

Plain relative

Paths with Prefix

ChrisDenton commented on Sep 11, 2026

clarfonthey commented on Sep 13, 2026

clarfonthey commented on Sep 22, 2026

ChrisDenton commented on Sep 22, 2026

MikkelPaulson commented on Nov 1, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Tracking Issue for `normalize_lexically` #134694