Skip to content

fix(sed): avoid extra newline when joining multiple files - #1261

Merged
nfischer merged 2 commits into
shelljs:mainfrom
Hashim1999164:fix/sed-multi-file-double-newline
Oct 9, 2026
Merged

nfischer merged 2 commits into
shelljs:mainfrom
Hashim1999164:fix/sed-multi-file-double-newline

Conversation

@Hashim1999164

@Hashim1999164 Hashim1999164 commented Jul 22, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • sed was joining multi-file results with an extra newline, so files that already ended with \n produced a blank line between them (hello1\n\nhello2\n).
  • Join file outputs directly, matching unix sed/cat concatenation.
  • Update multi-file tests and add a regression covering trailing-newline inputs.

Fixes #1068

Test plan

  • npx ava test/sed.js
  • Manual check: sed on file1.txt + file2.txt yields hello1\nhello2\n (not a double newline)

Concatenating sed results with an extra newline inserted a blank line
between files that already ended with a newline. Match unix sed/cat by
joining file outputs directly.

Fixes shelljs#1068.
@nfischer nfischer added fix Bug/defect, or a fix for such a problem bash compat Compatibility issues with bash or POSIX behavior labels Oct 3, 2026
Comment thread src/sed.js Outdated
return sed.join('\n');
// Concatenate like unix sed/cat: do not insert an extra newline between files.
// Each file's own trailing newline (if any) is already preserved by split/join above.
return sed.join('');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about files that don't have a trailing newline? Do we need to add one?

Can you show me a bash command that I can test on my Linux machine to prove the correct behavior one way or another?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nah we shouldnt invent a newline if the file didnt have one. join('') just concatenates whatever each file already produced, so a missing trailing newline stays missing.

the double newline bug was only for files that already end with \n. heres a quick repro on linux:

tmp=$(mktemp -d)
printf 'hello1\n' > "$tmp/c"
printf 'hello2\n' > "$tmp/d"
printf 'hello1' > "$tmp/a"
printf 'hello2' > "$tmp/b"

# files WITH trailing newline: sed and cat both give a single \n between
sed 's/x/x/' "$tmp/c" "$tmp/d" | od -c
# hello1 \n hello2 \n

# files WITHOUT trailing newline: cat stays glued, gnu sed inserts \n between
cat "$tmp/a" "$tmp/b" | od -c
# hello1 hello2
sed 's/x/x/' "$tmp/a" "$tmp/b" | od -c
# hello1 \n hello2

shelljs sed already keeps each files own ending through the split/join path, so joining with '' matches the trailing newline case (and cat). adding a separator would just bring the blank line back for normal \n terminated files.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you just demonstrated that we do need to insert the newline separator though. Your example with $tmp/a and $tmp/b shows that the two files are not simply concatenated; they are separated by a newline delimiter.

I'm aligned that we don't need to insert a double newline. But I think we do need to sometimes insert a single newline if one did not already exist. Can you check that we have test coverage for this?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good catch, youre right. bash sed does put a newline between the files, i was overcorrecting.

pushed a follow up: if the previous chunk already ends with a newline we just concatenate, otherwise we insert one. so no trailing newline files become hello1\nhello2 with no extra blank line.

Comment thread test/sed.js
`${t.context.tmp}/file2`);
t.falsy(shell.error());
t.is(result.code, 0);
t.is(result.toString(), 'hello1\nhello2');

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in the other comment thread, I believe this test was already correct: it should have a single newline between the two files, and no newline at the end.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agreed, restored that expectation. test is back to hello1\nhello2 with no trailing newline.

Unix sed puts a single newline between file outputs. Keep that delimiter when a file has no trailing newline, but do not insert a second one when it already ends with \\n.
@Hashim1999164

Copy link
Copy Markdown
Contributor Author

good catch, youre right. bash sed does put a newline between the files, i was overcorrecting.

pushed a follow up: if the previous chunk already ends with a newline we just concatenate, otherwise we insert one. so no trailing newline files become hello1\nhello2, and files that already end with \n stay hello1\nhello2\n with no blank line in the middle.

@codecov

codecov Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.17%. Comparing base (f364da6) to head (08c7745).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1261      +/-   ##
==========================================
+ Coverage   97.10%   97.17%   +0.07%     
==========================================
  Files          36       36              
  Lines        1518     1523       +5     
==========================================
+ Hits         1474     1480       +6     
+ Misses         44       43       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@nfischer
nfischer merged commit 77b506f into shelljs:main Oct 9, 2026
17 checks passed
@nfischer nfischer added this to the v0.10.x milestone Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bash compat Compatibility issues with bash or POSIX behavior fix Bug/defect, or a fix for such a problem

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unexpected double newline in shell.sed() output

2 participants