Inconsistencies between Get-ChildItem and Remove-Item with respect to -Recurse and -Include

Andrew Stanton (@StingyJack)

Update: Based on Andrew Stanton (@StingyJack)'s feedback, part of this issue may be related to a longstanding known Remove-Item problem currently documented as:

When it is used with the Include parameter, the Recurse parameter might not delete all subfolders or all child items. This is a known issue.

Get-ChildItem allows the following idioms to look for matching files on all levels of the target directory's hierarchy:

Get-ChildItem *.txt -Recurse
Get-ChildItem .  -Recurse -Include *.txt

By contrast, these idioms currently do not work with Remove-Item:

Steps to reproduce

Set-Location ~

Get-ChildItem *.txt -Recurse
Get-ChildItem . -Recurse -Include *.txt

Remove-Item *.txt -Recurse -WhatIf
Remove-Item . -Recurse -Include *.txt -WhatIf

Expected behavior

All 4 commands should output / target the same set of files, across the directory hierarchy.

Actual behavior

The Get-ChildItem commands behave as expected.

The Remove-Item commands do not:

Remove-Item *.txt -Recurse -WhatIf only targets items directly in the target directory (does not recurse)
- This behavior is sensible, but it deviates from Get-ChildItem's behavior - see Get-ChildItem -Recurse treats literals passed to -Path like wildcard expressions #5699
Remove-Item . -Recurse -Include *.txt -WhatIf fails.
E.g., on macOS: Remove-Item : Cannot remove the item at '/Users/jdoe' because it is in use.; i.e., an attempt is made to blindly remove ., without considering the -Include wildcard pattern.
- Note that the problem does not occur if you target a directory by name or path, from a different directory (rather than with . from inside it).

Environment data

PowerShell Core v6.0.0-alpha (v6.0.0-alpha.18) on macOS 10.12.4
PowerShell Core v6.0.0-alpha (v6.0.0-alpha.18) on Microsoft Windows 10 Pro (64-bit; v10.0.14393)

Remove-Item also does not appear to either sort the items for deletion by leaf-first or at least check if the item exists before deleting it.

The example pattern is to use Get-ChildItem and pipe the results

Get-ChildItem -Path "c:\temp\folder\* -Recurse | Remove-Item

but this will get "directory is not empty" and other errors, and you have to run the same thing 2 or more times to actually clean the folder out.

Its also does not carry over the behavior of rd /s or rmdir where a directory and all its contents must be removed. These two causes lead to code like the following being necessary to have a better chance (but not guarantee) of working.

if (Test-Path $folderPath)
{
  $items = Get-ChildItem -Path $folderPath -Recurse
  foreach ($item in $items)
  {
    if (Test-Path $item)
    {
      Remove-Item $item -Recurse -Force -Confirm:$false
    }
  }
  Remove-Item $folderPath -Force -Confirm:$false
}

Andrew Stanton (@StingyJack)

Hello Andrew Stanton (@StingyJack)

It seems that the issue is bigger than that (prior to Windows 10 1903, if I understood correctly)

This video seems to explain well the bug: https://www.youtube.com/watch?v=uhRWMGBjlO8&feature=youtu.be&t=455

And someone made a terrifying script 🤯 about it: https://stackoverflow.com/questions/53553729/cannot-remove-item-the-directory-is-not-empty/53561052#53561052

function Remove-FileSystemItem {
  <#
  .SYNOPSIS
    Removes files or directories reliably and synchronously.

  .DESCRIPTION
    Removes files and directories, ensuring reliable and synchronous
    behavior across all supported platforms.

    The syntax is a subset of what Remove-Item supports; notably,
    -Include / -Exclude and -Force are NOT supported; -Force is implied.

    As with Remove-Item, passing -Recurse is required to avoid a prompt when 
    deleting a non-empty directory.

    IMPORTANT:
      * On Unix platforms, this function is merely a wrapper for Remove-Item, 
        where the latter works reliably and synchronously, but on Windows a 
        custom implementation must be used to ensure reliable and synchronous 
        behavior. See https://github.com/PowerShell/PowerShell/issues/8211

    * On Windows:
      * The *parent directory* of a directory being removed must be 
        *writable* for the synchronous custom implementation to work.
      * The custom implementation is also applied when deleting 
         directories on *network drives*.

    * If an indefinitely *locked* file or directory is encountered, removal is aborted.
      By contrast, files opened with FILE_SHARE_DELETE / 
      [System.IO.FileShare]::Delete on Windows do NOT prevent removal, 
      though they do live on under a temporary name in the parent directory 
      until the last handle to them is closed.

    * Hidden files and files with the read-only attribute:
      * These are *quietly removed*; in other words: this function invariably
        behaves like `Remove-Item -Force`.
      * Note, however, that in order to target hidden files / directories
        as *input*, you must specify them as a *literal* path, because they
        won't be found via a wildcard expression.

    * The reliable custom implementation on Windows comes at the cost of
      decreased performance.

  .EXAMPLE
    Remove-FileSystemItem C:\tmp -Recurse

    Synchronously removes directory C:\tmp and all its content.
  #>
    [CmdletBinding(SupportsShouldProcess, ConfirmImpact='Medium', DefaultParameterSetName='Path', PositionalBinding=$false)]
    param(
      [Parameter(ParameterSetName='Path', Mandatory, Position = 0, ValueFromPipeline, ValueFromPipelineByPropertyName)]
      [string[]] $Path
      ,
      [Parameter(ParameterSetName='Literalpath', ValueFromPipelineByPropertyName)]
      [Alias('PSPath')]
      [string[]] $LiteralPath
      ,
      [switch] $Recurse
    )
    begin {
      # !! Workaround for https://github.com/PowerShell/PowerShell/issues/1759
      if ($ErrorActionPreference -eq [System.Management.Automation.ActionPreference]::Ignore) { $ErrorActionPreference = 'Ignore'}
      $targetPath = ''
      $yesToAll = $noToAll = $false
      function trimTrailingPathSep([string] $itemPath) {
        if ($itemPath[-1] -in '\', '/') {
          # Trim the trailing separator, unless the path is a root path such as '/' or 'c:\'
          if ($itemPath.Length -gt 1 -and $itemPath -notmatch '^[^:\\/]+:.$') {
            $itemPath = $itemPath.Substring(0, $itemPath.Length - 1)
          }
        }
        $itemPath
      }
      function getTempPathOnSameVolume([string] $itemPath, [string] $tempDir) {
        if (-not $tempDir) { $tempDir = [IO.Path]::GetDirectoryName($itemPath) }
        [IO.Path]::Combine($tempDir, [IO.Path]::GetRandomFileName())
      }
      function syncRemoveFile([string] $filePath, [string] $tempDir) {
        # Clear the ReadOnly attribute, if present.
        if (($attribs = [IO.File]::GetAttributes($filePath)) -band [System.IO.FileAttributes]::ReadOnly) {
          [IO.File]::SetAttributes($filePath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
        }
        $tempPath = getTempPathOnSameVolume $filePath $tempDir
        [IO.File]::Move($filePath, $tempPath)
        [IO.File]::Delete($tempPath)
      }
      function syncRemoveDir([string] $dirPath, [switch] $recursing) {
          if (-not $recursing) { $dirPathParent = [IO.Path]::GetDirectoryName($dirPath) }
          # Clear the ReadOnly attribute, if present.
          # Note: [IO.File]::*Attributes() is also used for *directories*; [IO.Directory] doesn't have attribute-related methods.
          if (($attribs = [IO.File]::GetAttributes($dirPath)) -band [System.IO.FileAttributes]::ReadOnly) {
            [IO.File]::SetAttributes($dirPath, $attribs -band -bnot [System.IO.FileAttributes]::ReadOnly)
          }
          # Remove all children synchronously.
          $isFirstChild = $true
          foreach ($item in [IO.directory]::EnumerateFileSystemEntries($dirPath)) {
            if (-not $recursing -and -not $Recurse -and $isFirstChild) { # If -Recurse wasn't specified, prompt for nonempty dirs.
              $isFirstChild = $false
              # Note: If -Confirm was also passed, this prompt is displayed *in addition*, after the standard $PSCmdlet.ShouldProcess() prompt.
              #       While Remove-Item also prompts twice in this scenario, it shows the has-children prompt *first*.
              if (-not $PSCmdlet.ShouldContinue("The item at '$dirPath' has children and the -Recurse switch was not specified. If you continue, all children will be removed with the item. Are you sure you want to continue?", 'Confirm', ([ref] $yesToAll), ([ref] $noToAll))) { return }
            }
            $itemPath = [IO.Path]::Combine($dirPath, $item)
            ([ref] $targetPath).Value = $itemPath
            if ([IO.Directory]::Exists($itemPath)) {
              syncremoveDir $itemPath -recursing
            } else {
              syncremoveFile $itemPath $dirPathParent
            }
          }
          # Finally, remove the directory itself synchronously.
          ([ref] $targetPath).Value = $dirPath
          $tempPath = getTempPathOnSameVolume $dirPath $dirPathParent
          [IO.Directory]::Move($dirPath, $tempPath)
          [IO.Directory]::Delete($tempPath)
      }
    }

    process {
      $isLiteral = $PSCmdlet.ParameterSetName -eq 'LiteralPath'
      if ($env:OS -ne 'Windows_NT') { # Unix: simply pass through to Remove-Item, which on Unix works reliably and synchronously
        Remove-Item @PSBoundParameters
      } else { # Windows: use synchronous custom implementation
        foreach ($rawPath in ($Path, $LiteralPath)[$isLiteral]) {
          # Resolve the paths to full, filesystem-native paths.
          try {
            # !! Convert-Path does find hidden items via *literal* paths, but not via *wildcards* - and it has no -Force switch (yet)
            # !! See https://github.com/PowerShell/PowerShell/issues/6501
            $resolvedPaths = if ($isLiteral) { Convert-Path -ErrorAction Stop -LiteralPath $rawPath } else { Convert-Path -ErrorAction Stop -path $rawPath}
          } catch {
            Write-Error $_ # relay error, but in the name of this function
            continue
          }
          try {
            $isDir = $false
            foreach ($resolvedPath in $resolvedPaths) {
              # -WhatIf and -Confirm support.
              if (-not $PSCmdlet.ShouldProcess($resolvedPath)) { continue }
              if ($isDir = [IO.Directory]::Exists($resolvedPath)) { # dir.
                # !! A trailing '\' or '/' causes directory removal to fail ("in use"), so we trim it first.
                syncRemoveDir (trimTrailingPathSep $resolvedPath)
              } elseif ([IO.File]::Exists($resolvedPath)) { # file
                syncRemoveFile $resolvedPath
              } else {
                Throw "Not a file-system path or no longer extant: $resolvedPath"
              }
            }
          } catch {
            if ($isDir) {
              $exc = $_.Exception
              if ($exc.InnerException) { $exc = $exc.InnerException }
              if ($targetPath -eq $resolvedPath) {
                Write-Error "Removal of directory '$resolvedPath' failed: $exc"
              } else {
                Write-Error "Removal of directory '$resolvedPath' failed, because its content could not be (fully) removed: $targetPath`: $exc"
              }
            } else {
              Write-Error $_  # relay error, but in the name of this function
            }
            continue
          }
        }
      }
    }
}

Stack Overflow
Cannot remove item, The Directory is not empty
When using the Remove-Item command, even utilizing the -r and -Force parameters, sometimes the following error message is returned: Remove-Item : Cannot remove item C:\Test Folder\Test Folder\Ta...

Mickaël Tricot (@mickaeltr)

Mickaël Tricot (@mickaeltr) Your referring to #8211, which will ultimately only be resolved once the WinAPI file-removal calls become synchronous (you can use the function as a workaround, but it has limitations).

Michael Klement (@mklement0)

Thanks Michael Klement (@mklement0) for the link and explanation.

This issue has not had any activity in 6 months, if this is a bug please try to reproduce on the latest version of PowerShell and reopen a new issue and reference this issue if this is still a blocker for you.

This issue has been marked as "No Activity" as there has been no activity for 6 months. It has been closed for housekeeping purposes.

Navigation Menu

Repository navigation

Inconsistencies between Get-ChildItem and Remove-Item with respect to -Recurse and -Include #3727

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Inconsistencies between Get-ChildItem and Remove-Item with respect to -Recurse and -Include #3727

Description

Steps to reproduce

Expected behavior

Actual behavior

Environment data

Activity

StingyJack commented on Feb 8, 2018

mickaeltr commented on May 13, 2020

mklement0 commented on May 13, 2020

mickaeltr commented on May 13, 2020

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 16, 2023

microsoft-github-policy-service commented on Nov 23, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions