Skip to content

Conversation

@SteveL-MSFT
Copy link
Member

@SteveL-MSFT SteveL-MSFT commented Feb 27, 2021

PR Summary

When using Move-Item between DFS paths, the FileSystemProvider will decide whether to use the .NET Move() or a Copy() and Delete() operation based on comparing the root path of the source and destination. This will work whenever the source and destination are on the same volume, but results in an Access Denied error message if not. In the case of DFS paths, they look like SMB share paths but typically are on different volumes so the move fails with Access Denied.

There is a Win32 API that could be used to determine if a UNC path is DFS, but it appears there is a timeout of around 1 second when the path is NOT DFS, so that's too big of an impact for normal usage.

Fix is to try MoveTo() as that is much faster on the same volume. If that fails, we check the HResult to see if it's Access Denied (0x80070005) and if so (as this is what is returned in DFS scenario) we fallback to CopyAndDelete().

Tested manually on a VM with DFS shares.

PR Context

Reported through Microsoft customer support

PR Checklist

@ghost ghost assigned anmenaga Feb 27, 2021
@SteveL-MSFT
Copy link
Member Author

Going to change this to try Move() first, then if AccessDenied, then try CopyDelete().

@iSazonov iSazonov added the CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log label Mar 2, 2021
@SteveL-MSFT SteveL-MSFT marked this pull request as draft March 2, 2021 19:48
@SteveL-MSFT
Copy link
Member Author

Making this a draft PR for now, the customer tried a private and still had initial problem after latest change. Trying to determine if they got a different HResult.

@iSazonov
Copy link
Collaborator

iSazonov commented Mar 3, 2021

Looking Directory.MoveTo() implementation https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L119-L137
I think we can throw for ERROR_PATH_NOT_FOUND and ERROR_ALREADY_EXISTS but fall back for other exceptions.

@SteveL-MSFT
Copy link
Member Author

SteveL-MSFT commented Mar 3, 2021

@iSazonov Looks like there can be different HResult for DFS, so can't rely on that. Based on https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/System.IO.FileSystem/src/System/IO/FileSystem.Windows.cs#L133, I think we can just try copydelete on an IOException.

@SteveL-MSFT SteveL-MSFT marked this pull request as ready for review March 3, 2021 15:13
@iSazonov
Copy link
Collaborator

iSazonov commented Mar 3, 2021

If we look GetExceptionForWin32Error()
https://github.com/dotnet/runtime/blob/79ae74f5ca5c8a6fe3a48935e85bd7374959c570/src/libraries/Common/src/System/IO/Win32Marshal.cs#L25
I think we should fall back for IOException and do not fall back for FileNotFoundException, DirectoryNotFoundException, PathTooLongException which is IOException too.

GitHub
.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps. - dotnet/runtime

@iSazonov iSazonov self-requested a review March 3, 2021 17:32
@SteveL-MSFT
Copy link
Member Author

@iSazonov I see, that makes sense, will update.

@SteveL-MSFT
Copy link
Member Author

Need to think about this some more, the FQErrorID is now different on Linux vs Windows which is not acceptable. Maybe I can find out if there is a limited set of HResult we can check against.

@SteveL-MSFT SteveL-MSFT marked this pull request as draft March 3, 2021 19:56
@iSazonov
Copy link
Collaborator

iSazonov commented Mar 4, 2021

We already discussed this with @rjmholt . See https://github.com/PowerShell/PowerShell/pull/13044/files#r446770600

I guess .Net unify the behavior (exceptions) and we can have one code path for all platforms.

I see the test It "Verify Move-Item across devices for directory" is not correct too. It came from from #13044 . I suggest to re-review #13044 now since the code doesn't work and isn't tested correctly neither Windows nor Unix.

GitHub
PR Summary Fixes #13039. Currently Move-Item does not support cross-mount moves on Unix, because it doesn't check properly whether given paths lie on the same filesystem. This PR fixes the chec...

@SteveL-MSFT
Copy link
Member Author

@iSazonov can you be specific on your concerns about that test?

@SteveL-MSFT SteveL-MSFT marked this pull request as ready for review March 4, 2021 23:11
@SteveL-MSFT SteveL-MSFT requested a review from rjmholt March 4, 2021 23:11
@iSazonov
Copy link
Collaborator

iSazonov commented Mar 5, 2021

@iSazonov can you be specific on your concerns about that test?

I'd expect Move-Item -Path $dir -Destination $destination -ErrorAction Stop throws in the test.

https://github.com/rjmholt/PowerShell/blob/7a2c1fb06cfddff1bb9f2535197154ff143aeefa/test/powershell/Modules/Microsoft.PowerShell.Management/FileSystem.Tests.ps1#L134-L152


And I guess .Net team unified exceptions in the API for all platforms and we could have one code path here. Both return codes fall in IOException and we need only to ignore FileNotFoundException, DirectoryNotFoundException, PathTooLongException.

We must take into account that there are many ways to mount a different file system / volume and they may have different return codes.

@SteveL-MSFT
Copy link
Member Author

@iSazonov so that test case relies on a testhook to simulate doing a move across devices/volumes. It succeeds because the test hook forces it to throw the exception that results in CopyAndDelete() to be used.

In one of my commits, I tried the approach of ignoring FileNotFoundException, DirectoryNotFoundException, PathTooLongException but that resulted in another FullyQualifiedErrorId to change whereas the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

@iSazonov
Copy link
Collaborator

iSazonov commented Mar 6, 2021

@iSazonov so that test case relies on a testhook to simulate doing a move across devices/volumes. It succeeds because the test hook forces it to throw the exception that results in CopyAndDelete() to be used.

Sorry, my bad. With -Stop parameter I expect -Should -Not -Throw -Because ""

In one of my commits, I tried the approach of ignoring FileNotFoundException, DirectoryNotFoundException, PathTooLongException but that resulted in another FullyQualifiedErrorId to change whereas the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

We had similar discussion in #13634 (comment)
I tried to make minimal changes. But after some conversations we concluded that driver error codes is not reliable way in that context and I made more large refactoring.
So here I also expect that using driver codes is the last thing we should do.
We could think about moving to OneDrive but this works. WebDav? Other Cloud services? Etc.?
It is good if a driver supports moving, but if not? What code will it return? We do not know. I suppose you could always try copying.

My proposal:

        private void MoveDirectoryInfoUnchecked(DirectoryInfo directory, string destinationPath, bool force)
        {
            try
            {
                if (InternalTestHooks.ThrowExdevErrorOnMoveDirectory)
                {
                    throw new IOException("Invalid cross-device link", hresult: TEST_CODE);
                }

                directory.MoveTo(destinationPath);
            }
            catch (Exception exception)
            {
                switch (exception)
                {
                    case FileNotFoundException:
                    case DirectoryNotFoundException:
                    case PathTooLongException:
                        throw;
                    case IOException:
                        CopyAndDelete(directory, destinationPath, force);
                        break;
                    default:
                        throw;
                }
            }
        }

Copy link

@anmenaga anmenaga left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the current implementation checking for specific HResult which is specific to the Unix error of moving across devices and Windows error of moving across volumes has less impact. I was also concerned that other specific IOExceptions beyond those ignored types would fallback to CopyAndDelete() when it didn't in the past and might cause unintended side effects.

I agree with this.

@anmenaga anmenaga merged commit 22b3653 into PowerShell:master Mar 8, 2021
@SteveL-MSFT SteveL-MSFT deleted the dfs-move branch March 9, 2021 05:16
@ghost
Copy link

ghost commented Mar 16, 2021

🎉v7.2.0-preview.4 has been released which incorporates this pull request.:tada:

Handy links:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants