Repository navigation
fix(sed): avoid extra newline when joining multiple files - #1261
Conversation
Concatenating sed results with an extra newline inserted a blank line between files that already ended with a newline. Match unix sed/cat by joining file outputs directly. Fixes shelljs#1068.
| return sed.join('\n'); | ||
| // Concatenate like unix sed/cat: do not insert an extra newline between files. | ||
| // Each file's own trailing newline (if any) is already preserved by split/join above. | ||
| return sed.join(''); |
There was a problem hiding this comment.
What about files that don't have a trailing newline? Do we need to add one?
Can you show me a bash command that I can test on my Linux machine to prove the correct behavior one way or another?
There was a problem hiding this comment.
nah we shouldnt invent a newline if the file didnt have one. join('') just concatenates whatever each file already produced, so a missing trailing newline stays missing.
the double newline bug was only for files that already end with \n. heres a quick repro on linux:
tmp=$(mktemp -d)
printf 'hello1\n' > "$tmp/c"
printf 'hello2\n' > "$tmp/d"
printf 'hello1' > "$tmp/a"
printf 'hello2' > "$tmp/b"
# files WITH trailing newline: sed and cat both give a single \n between
sed 's/x/x/' "$tmp/c" "$tmp/d" | od -c
# hello1 \n hello2 \n
# files WITHOUT trailing newline: cat stays glued, gnu sed inserts \n between
cat "$tmp/a" "$tmp/b" | od -c
# hello1 hello2
sed 's/x/x/' "$tmp/a" "$tmp/b" | od -c
# hello1 \n hello2shelljs sed already keeps each files own ending through the split/join path, so joining with '' matches the trailing newline case (and cat). adding a separator would just bring the blank line back for normal \n terminated files.
There was a problem hiding this comment.
I think you just demonstrated that we do need to insert the newline separator though. Your example with $tmp/a and $tmp/b shows that the two files are not simply concatenated; they are separated by a newline delimiter.
I'm aligned that we don't need to insert a double newline. But I think we do need to sometimes insert a single newline if one did not already exist. Can you check that we have test coverage for this?
There was a problem hiding this comment.
good catch, youre right. bash sed does put a newline between the files, i was overcorrecting.
pushed a follow up: if the previous chunk already ends with a newline we just concatenate, otherwise we insert one. so no trailing newline files become hello1\nhello2 with no extra blank line.
| `${t.context.tmp}/file2`); | ||
| t.falsy(shell.error()); | ||
| t.is(result.code, 0); | ||
| t.is(result.toString(), 'hello1\nhello2'); |
There was a problem hiding this comment.
As discussed in the other comment thread, I believe this test was already correct: it should have a single newline between the two files, and no newline at the end.
There was a problem hiding this comment.
agreed, restored that expectation. test is back to hello1\nhello2 with no trailing newline.
Unix sed puts a single newline between file outputs. Keep that delimiter when a file has no trailing newline, but do not insert a second one when it already ends with \\n.
|
good catch, youre right. bash sed does put a newline between the files, i was overcorrecting. pushed a follow up: if the previous chunk already ends with a newline we just concatenate, otherwise we insert one. so no trailing newline files become hello1\nhello2, and files that already end with \n stay hello1\nhello2\n with no blank line in the middle. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1261 +/- ##
==========================================
+ Coverage 97.10% 97.17% +0.07%
==========================================
Files 36 36
Lines 1518 1523 +5
==========================================
+ Hits 1474 1480 +6
+ Misses 44 43 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Summary
sedwas joining multi-file results with an extra newline, so files that already ended with\nproduced a blank line between them (hello1\n\nhello2\n).sed/catconcatenation.Fixes #1068
Test plan
npx ava test/sed.jssedonfile1.txt+file2.txtyieldshello1\nhello2\n(not a double newline)