Skip to content

rspack: virtual folder deletion is not resilient when multiple compilers are run in parallel #537

@elbywan

Description

@elbywan

Environment

pnpx envinfo --system --npmPackages unplugin --binaries

System:
  OS: macOS 15.6
  CPU: (10) arm64 Apple M1 Max
  Memory: 1.57 GB / 64.00 GB
  Shell: 5.9 - /bin/zsh
Binaries:
  Node: 20.15.1 - ~/.volta/tools/image/node/20.15.1/bin/node
  Yarn: 4.2.2 - ~/.volta/tools/image/yarn/4.2.2/bin/yarn
  npm: 10.7.0 - ~/.volta/tools/image/node/20.15.1/bin/npm
  pnpm: 10.14.0 - ~/.volta/bin/pnpm
npmPackages:
  unplugin: 2.3.7 => 2.3.7 

Reproduction

# create a new project
mkdir unplugin-repro-multiprocess-rspack-virtual
cd unplugin-repro-multiprocess-rspack-virtual
# init and add packages
pnpm init
pnpm add -D @rspack/core unplugin
# create src files
mkdir src
cat << EOF > src/one.js
import "virtual:console.log('one')"
EOF
cat << EOF > src/two.js
import "virtual:console.log('two')"
EOF
# create a dummy plugin that resolves and load virtual files
cat << EOF > virtual-code-plugin.js
const { createUnplugin } = require("unplugin");

module.exports = createUnplugin((delay = 0) => ({
  name: "virtual-code-plugin",
  async resolveId(id) {
    if (id.startsWith("virtual:")) {
        await new Promise(resolve => setTimeout(resolve, delay));
        return id
    }
  },
  load: {
    filter: {
        id: /^virtual:/
    },
    handler(id) {
        return id.slice(8);
    }
  }
}));
EOF
# create a build script that forks the process and runs the compiler in parallel twice
cat << EOF > build.js
const child_process = require('node:child_process');
const process = require('node:process');
const rspack = require('@rspack/core');
const VirtualCodePlugin = require('./virtual-code-plugin');

let isChild = process.argv[2] === 'child';
if(!isChild) { child_process.fork(process.argv[1], ['child']) }

const compiler = rspack({
    entry: isChild ? { one: './src/one.js' } : { two: './src/two.js' },
    target: 'web',
    plugins: [VirtualCodePlugin.rspack(isChild ? 0 : 500)],
    devtool: false
});

compiler.run(function(err, stats) {
    if (err) {
        console.error(err);
    } else {
        console.log(stats.toString());
    }
    compiler.close((error) => {
        if (error) {
            console.error(error);
        }
    });
});
EOF
# the build will crash, because the child process will delete the virtual folder while the parent is still running
node build.js

Describe the bug

When running multiple rspack compilers in separate processes (in parallel), the node_modules/.virtual folder will get deleted by the first process that completes - making the pending processes fail since they won't be able to open their own virtual files as the folder is shared.

Logs

node build.js
asset one.js 481 bytes [emitted] (name: one)
runtime modules 730 bytes 5 modules
cacheable modules 54 bytes
  ./src/one.js 36 bytes [built] [code generated]
  ./node_modules/.virtual/virtual%3Aconsole.log('one') 18 bytes [built] [code generated]
Rspack 1.4.11 compiled successfully in 324 ms
assets by status 314 bytes [cached] 1 asset
runtime modules 93 bytes 2 modules
./src/two.js 36 bytes [built] [code generated] [1 error]

ERROR in ./src/two.js 1:0-35
  × Error: ENOENT: no such file or directory, open '/Users/julien.elbaz/dev/experiments/unplugin-repro-multiprocess-rspack-virtual/node_modules/.virtual/virtual%3Aconsole.log('two')'
  │     at async open (node:internal/fs/promises:633:25)
  │     at async Object.writeFile (node:internal/fs/promises:1207:14)
  │     at async FakeVirtualModulesPlugin.writeModule (/Users/julien.elbaz/dev/experiments/unplugin-repro-multiprocess-rspack-virtual/node_modules/.pnpm/unplugin@2.3.7/node_modules/unplugin/dist/utils-D30tqOw3.cjs:34:3)
  │     at async /Users/julien.elbaz/dev/experiments/unplugin-repro-multiprocess-rspack-virtual/node_modules/.pnpm/unplugin@2.3.7/node_modules/unplugin/dist/index.cjs:746:10
  │

Rspack 1.4.11 compiled with 1 error in 670 ms

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions