WinAPI MoveFile used for renaming keeps Existing file

I am using MoveFile for WinAPI in my C code to rename a file. The file is renamed to the same parent directory in the same volume. But, intermittently I see MoveFile keeping the existing original file and creating the new renamed file as well. Hence, we have both the files.

The exit code returned is non zero which denotes operation was successful

Pseudo Code snippet for better referrence.

int
file_move(wchar_t *existing_file, wchar_t *dest_file)
{
    if (exists(dest_file) == 0) {
        rc = MoveFile(existing_file, dest_file);
        if (rc == 0) {
           rc = GetLastError();
           printf("File could not be renamed with error %u", rc);
        }
        printf("File was renamed.");
    }
}

exists checks whether a file path is present or not.

Ideally, the original file should not be present and only the renamed file should be kept.

The issue can be reported because the original files, if present are moved to different location on completion of renaming. We can also see the renamed files based on logs.

Files with same pattern name but just different name e.g. file_2_name.sqlite files are renamed properly.

The renaming is done to change the extension of the file. The return value was also non-zero signifying no errors.