-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Cleanup Utils.FileExists(), Utils.DirectoryExists() and Utils.ItemExists() methods #7129
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cleanup Utils.FileExists(), Utils.DirectoryExists() and Utils.ItemExists() methods #7129
Conversation
27fa002 to
a62483f
Compare
|
WIP - still 6 commits upcoming... |
e525be4 to
bb1aa8a
Compare
|
@daxian-dbw Please look the issue with Unix compiling. Yesterday the commit (UpdatableHelpSystem.cs) was compiled without errors. Today it failed. I tryed compile the code on CI Appveyor and get the same error. Locally I can compile without problem and IntelliSense works well in VS Code. Tracking Issue https://github.com/dotnet/corefx/issues/30626 |
|
I have only one commit related to Unix but can not push it until travis-ci problem will be resolved. |
|
I tested locally all versions - msi and zip packages - 2.1.300 and 2.1.301 and the project is compiled well. |
8c2fa88 to
4c84b21
Compare
|
Rebased to move to .Net Core 2.1.1. |
Ah, this is probably because the compiler resolves You can use the full type name The CI failure now is different though, it's complaining about |
4b4fe4d to
f281765
Compare
|
@daxian-dbw Thanks for help! The PR is ready for review. |
f281765 to
c53aaa3
Compare
|
Rebased to resolve a merge conflict. |
adityapatwardhan
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Signed of with 1 non-blocking comment.
| { | ||
| } | ||
| catch (UnauthorizedAccessException) | ||
| if (!Directory.Exists(drive.Root)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should this be
return !DirectoryExists(drive.Root)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Seems original code is easier to understand.
|
@daxian-dbw Can we merge the PR? |
|
@iSazonov Let me quickly go through the changes. |
| } | ||
| } | ||
| #else | ||
| if (Utils.FileExists(path)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part was in windows-only section. You moved it above at the beginning of the method, but that's not made "compile-only-windows"
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assumed that we will always receive the path with the right case. (Seems we get the path in code not from user). So the first block will always return a value on all platforms. The code below will only search the file on Unix with ignoring case.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, it makes sense to to always have that check on all plats. #closed
| // We use 'FileInfo.Attributes' (not 'FileInfo.Exist') | ||
| // because we want to get exceptions | ||
| // like UnauthorizedAccessException or IOException. | ||
| if ((int)item.Attributes != -1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you please explain how do you decide which Utils.DirectoryExists or Utils.FileExists can be replaced with Directory.Exists and File.Exists (swallow all exceptions), and which ones need to continue throw exceptions? I wonder why we want to get exceptions in this method.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to keep the logic of the previous code. And I tried to make as few changes as possible. I'm afraid that this PR will grow many times if we try to correct the previous logic in all places where Utils.DirectoryExists or Utils.FileExists exist.
Here I'm afraid that we'll get issues in LoadStringFromPath() .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you think this method GetFilePath(string path) should throw the exception from checking the file existence?
I tried to keep the logic of the previous code
The previous code just used Utils.FileExists and Utils.DirectoryExists like everywhere else. Here, you didn't replace them to File.Exists and Directory.Exists but check the existence in a way that may throw exception. I want to know what your decision is based on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Utils.FileExists and Utils.DirectoryExists can throw but File.Exists and Directory.Exists don't throw.
If calling code catch exceptions and ignore them we can safely replace Utils.FileExists and Utils.DirectoryExists with File.Exists and Directory.Exists.
Here calling code don't catch and ignore exceptions so I used item.Attributes.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If calling code catch exceptions and ignore them we can safely replace Utils.FileExists and Utils.DirectoryExists with File.Exists and Directory.Exists.
Here calling code don't catch and ignore exceptions so I used item.Attributes.
Does this mean for all the places you changed to File.Exists or Directory.Exists, you have checked that the calling code actually caches exceptions and ignore them?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I did this commit by commit.
[daxian-dbw] #Closed
| if (fspDynamicParam != null && fspDynamicParam.FollowSymlink) | ||
| { | ||
| tracker = new InodeTracker(directory.FullName); | ||
| tracker = new InodeTracker(((DirectoryInfo)fsinfo).FullName); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do you need to cast fsinfo to DirectoryInfo to access FullName?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see FullName is in different code paths in the classes:
https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem/src/System/IO/FileInfo.cs#L29
https://github.com/dotnet/corefx/blob/master/src/System.IO.FileSystem/src/System/IO/DirectoryInfo.cs#L38
The same for Name in comment below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Here is a quick check of the properties, and it looks the casting is not needed.
void Main()
{
FileSystemInfo info1 = GetFileSystemInfo(@"C:\arena\temp.txt");
FileSystemInfo info2 = GetFileSystemInfo(@"C:\arena");
Console.WriteLine("info1.Name: {0}", info1.Name);
Console.WriteLine("info1.FullName: {0}", info1.FullName);
Console.WriteLine("info2.Name: {0}", info2.Name);
Console.WriteLine("info2.FullName: {0}", info2.FullName);
}
static FileSystemInfo GetFileSystemInfo(string path)
{
if (File.Exists(path))
{
return new FileInfo(path);
}
else if (Directory.Exists(path))
{
return new DirectoryInfo(path);
}
return null;
}Console Output:
info1.Name: temp.txt
info1.FullName: C:\arena\temp.txt
info2.Name: arena
info2.FullName: C:\arena
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, we don't need casting because the value is evaluated in object creation time and here we only retrive the value.
Fixed.
[daxian-dbw] #Closed
| fileInfo.Name, | ||
| fileInfo.FullName, | ||
| ((FileInfo)fsinfo).Name, | ||
| ((FileInfo)fsinfo).FullName, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same here, do you need to cast?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, we don't need casting because the value is evaluated in object creation time and here we only retrive the value.
Fixed.
[daxian-dbw] #Closed
| } | ||
| else | ||
| WriteItemObject(fileInfo, path, false); | ||
| WriteItemObject((FileInfo)fsinfo, path, false); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No need to cast here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
[daxian-dbw] #Closed
| path = NormalizePath(path); | ||
|
|
||
| return Utils.DirectoryExists(path); | ||
| return Directory.Exists(path); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
same question, why we ignore all exceptions here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This behavior corresponds to the xml description of the IsItemContainer method. We use result in MakePath or to get DirectoryInfo/FileInfo - so we always get specific error later. Seems we could cleanup using IsItemContainer() - it turns out we make a lot of extra kernel calls.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. I guess it makes sense to swallow all exceptions here. #closed
| { | ||
| isContainer = true; | ||
| fileSystemInfoShell = PSObject.AsPSObject(new DirectoryInfo(path)); | ||
| fileSystemInfoShell = PSObject.AsPSObject((DirectoryInfo)fsinfo); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no need to cast here
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
[daxian-dbw] #Closed
| { | ||
| // Maybe the path is a file name so try a FileInfo instead | ||
| fileSystemInfoShell = PSObject.AsPSObject(new FileInfo(path)); | ||
| fileSystemInfoShell = PSObject.AsPSObject((FileInfo)fsinfo); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ditto.
Actually, this if/else block can be removed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
[daxian-dbw] #Closed
| } | ||
|
|
||
| if (fileSystemInfoShell != null) | ||
| { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fileSystemInfoShell still could be null, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why? We call PSObject.AsPSObject(fsinfo) where fsinfo != null.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, you moved the code within if (fsinfo != null). I missed that. #Closed
|
|
||
| // If the directory exists, just return it. | ||
| // Otherwise fallback to previous behavior and let provider handle it. | ||
| if (Directory.Exists(userPath)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you please explain why having this change? And this might not be the right place for the new code, because a new collection is just allocated above, but you might return without using the collection at all.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this PR I went through so many code ways that I do not remember how I got here.
As far as I see the idea was simple - do not perform an analysis if the path already exists.
Fixed - I return without using the collection.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From a quick browsing of the code after this, it seems the new code you added treats the userPath as a literal path without consulting context.SuppressWildcardExpansion. Say userPath is /blah/a[bc], and it happens to have a folder named a[bc].
I'm not that familiar with the file system code, so there could be other concerns on this early return. I suggest to focus on the topic of this PR, and leave optimization to a separate issue/PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The code was removed.
[daxian-dbw] #Closed
| path = NormalizePath(path); | ||
|
|
||
| PSObject results = new PSObject(); | ||
| PSObject fileSystemInfoShell = null; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This variable now should be moved in if (fsinfo != null), since it's only used in the if block.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed.
[daxian-dbw] #Closed
|
|
||
| // If the directory exists, just return it. | ||
| // Otherwise fallback to previous behavior and let provider handle it. | ||
| if (Directory.Exists(userPath)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
From a quick browsing of the code after this, it seems the new code you added treats the userPath as a literal path without consulting context.SuppressWildcardExpansion. Say userPath is /blah/a[bc], and it happens to have a folder named a[bc].
I'm not that familiar with the file system code, so there could be other concerns on this early return. I suggest to focus on the topic of this PR, and leave optimization to a separate issue/PR.
|
@daxian-dbw Could you please continue? |
|
src/System.Management.Automation/engine/SessionStateDriveAPIs.cs, line 966 at r1 (raw file):
Is this going to suppress IO and UnAuthorized Exceptions like the old code? |
|
src/System.Management.Automation/namespaces/FileSystemProvider.cs, line 136 at r1 (raw file):
Why don't you cast fsinfo to a
|
TravisEz13
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 9 of 9 files at r1.
Reviewable status:complete! all files reviewed
Yes, we preserve old code logic.
Here we create objects of different types. |
|
@iSazonov |
| // We use 'FileInfo.Attributes' (not 'FileInfo.Exist') | ||
| // because we want to get exceptions | ||
| // like UnauthorizedAccessException or IOException. | ||
| if (fileList.Length > 0 && (int)fileList[0].Attributes != -1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The original code doesn't check for existence for the files returned from Direcotry.GetFiles. Why do you add the check now? Can't we trust direcotryInfo.GetFiles?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please also respond to this comment: #7129 (comment)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@iSazonov Can you please respond to the first comment in this thread (#7129 (comment))?
That's the only open comment left now :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry, GitHub sometimes tricky hides comments and I can not find where to type response :-)
You are right - the check is dead code - I removed it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great, thanks! #Closed
PR Summary
Continue #6929
Replace #7100
Cleanup code to replace Utils.FileExists(), Utils.DirectoryExists() and Utils.ItemExists() methods with .Net Core methods.
PR Checklist
.h,.cpp,.cs,.ps1and.psm1files have the correct copyright headerWIP:to the beginning of the title and remove the prefix when the PR is ready.[feature]if the change is significant or affects feature testsThis change is