Bash IFS causes unexected results with array output

Question

When IFS has its default value and an array is printed without quotes, the interpreted value doesn't have quotes, but when IFS doesn't have a space it it, it does.

Using an echo web server and curl to demonstrate why this makes a difference:

bash-5.2$ echo $BASH_VERSION 
+ echo '5.2.21(1)-release'
5.2.21(1)-release
bash-5.2$ declare -a testit=([0]="-H" [1]="foo: bar")
+ testit=(['0']='-H' ['1']='foo: bar')
+ declare -a testit
bash-5.2$ declare -p testit
+ declare -p testit
declare -a testit=([0]="-H" [1]="foo: bar")
bash-5.2$ IFS=$' \t\n'
+ IFS='         
'
bash-5.2$ declare -p IFS
+ declare -p IFS
declare -- IFS=$' \t\n'
bash-5.2$ echo ${testit[@]}
+ echo -H foo: bar
-H foo: bar
bash-5.2$ curl [::1] ${testit[@]}
+ curl '[::1]' -H foo: bar
GET / HTTP/1.1
Host: [::1]
User-Agent: curl/8.5.0
Accept: */*

curl: (6) Could not resolve host: bar
bash-5.2$ echo "${testit[@]}"
+ echo -H 'foo: bar'
-H foo: bar
bash-5.2$ curl [::1] "${testit[@]}"
+ curl '[::1]' -H 'foo: bar'
GET / HTTP/1.1
Host: [::1]
User-Agent: curl/8.5.0
Accept: */*
foo: bar

bash-5.2$ IFS=$'_\t\n'
+ IFS='_        
'
bash-5.2$ declare -p IFS
+ declare -p IFS
declare -- IFS=$'_\t\n'
bash-5.2$ echo ${testit[@]}
+ echo -H 'foo: bar'
-H foo: bar
bash-5.2$ curl [::1] ${testit[@]}
+ curl '[::1]' -H 'foo: bar'
GET / HTTP/1.1
Host: [::1]
User-Agent: curl/8.5.0
Accept: */*
foo: bar

bash-5.2$ echo "${testit[@]}"
+ echo -H 'foo: bar'
-H foo: bar
bash-5.2$ curl [::1] "${testit[@]}"
+ curl '[::1]' -H 'foo: bar'
GET / HTTP/1.1
Host: [::1]
User-Agent: curl/8.5.0
Accept: */*
foo: bar

Why does the removal of the space from IFS (regardless of where you put the space, and regardless of whether you replace the space with another character; underscore, for example) cause this different behavior? The only value of IFS on output is the first character, and it doesn't matter where the space is to cause the unexpected behavior.

Do yourself a favor. Don't use POSIX shells like Bash for anything other than trivial job control. Use a modern shell like Elvish or Fish (which don't have an IFS variable and all the problems it causes) or a programming language like Python or Go for this type of thing. — Kurtis Rader
– Kurtis Rader, Commented Jan 22, 2024 at 2:53
@KurtisRader I didn't know Elvish, until now... I've read this: Elvish Shell, then tried (in bash): curl -sL xkcd.com/{1100..1111}/info.0.json |jq -r '[ ( .num|tostring ),.title ]|join(": ")' then I still don't understand why to reinvent wheel... — F. Hauri - Give Up GitHub
– F. Hauri - Give Up GitHub, Commented Jan 22, 2024 at 7:03
You didn't spend even a second learning how to use Elvish @F.Hauri-GiveUpGitHub. The point of my recommending using Elvish is that it has native support for JSON. Which means you don't need to use the jq command. And your "don't understand why to reinvent the wheel" comment is also hard to understand. The whole point of my prior comment to use a shell like Elvish is that it avoids the problematic IFS splitting of POSIX shells like Bash. — Kurtis Rader
– Kurtis Rader, Commented Jan 22, 2024 at 7:17
It's very hard to understand what the code in your question is intended to show us. In particular I have no idea why there are curl commands in it if you're just asking why array contents get printed in different ways. Please edit your question to show a test script we can copy/paste to replicate your problem, not just a bunch of separate commands with stderr and stdout output interleaved. Also, don't name variables test as that's the name of a command and so obfuscates your code at best. — Ed Morton
– Ed Morton, Commented Jan 22, 2024 at 21:15
“Using an echo web server and curl to demonstrate why this makes a difference:” — Larry
– Larry, Commented Jan 24, 2024 at 0:30

tjm3772 · Accepted Answer · 2024-01-22 02:47:33Z

4

IFS is used to split the result of a parameter expansion, per Word Splitting in the manual:

The shell scans the results of parameter expansion, command substitution, 
and arithmetic expansion that did not occur within double quotes for word splitting.

The shell treats each character of $IFS as a delimiter, and splits 
the results of the other expansions into words using these characters 
as field terminators.

When IFS contains a space, the unquoted expansion foo: bar gets split into the two words foo: and bar. When you remove the space from IFS this splitting does not occur, so the debug output displays 'foo: bar' in quotes to indicate that this is a single word on the command line and not two words like the space might otherwise indicate. You would also get the quotes in the debug output if IFS contains a space but you quote the variable expansion to disable word splitting.

user@host$ var='foo: bar'
user@host$ set -x
user@host$ echo $var
+ echo foo: bar
foo: bar
user@host$ echo "$var"
+ echo 'foo: bar'
foo: bar
user@host$ declare -p IFS
+ declare -p IFS
declare -- IFS="    
"

answered Jan 22, 2024 at 2:47

tjm3772

3,2444 silver badges15 bronze badges

Sign up to request clarification or add additional context in comments.

1 Comment

Larry Over a year ago

Clearly I had a misunderstanding of where/what IFS is used for.

glenn jackman · Accepted Answer · 2024-01-22 11:40:10Z

1

IFS only affects arrays when you use * as the index, and you put the parameter expansion in double quotes:

$ a=(foo bar baz)
$ IFS=":"
$ printf '%s\n' ${a[@]}
foo
bar
baz
$ printf '%s\n' ${a[*]}
foo
bar
baz
$ printf '%s\n' "${a[@]}"
foo
bar
baz
$ printf '%s\n' "${a[*]}"
foo:bar:baz

answered Jan 22, 2024 at 11:40

glenn jackman

249k42 gold badges233 silver badges363 bronze badges

2 Comments

tjm3772 Over a year ago

I don't believe this is quite the same thing as what OP is asking about. IFS is used to concatenate all the elements into a single word when referenced as "${a[*]}" as a special array expansion rule but quoting itself also disables traditional uses of IFS. An unquoted array still undergoes word splitting just like an unquoted variable.

tjm3772 Over a year ago

If your array was a=( foo bar:baz ) then printf '%s\n' ${a[@]} would have different outputs with default IFS vs IFS=:.

Collectives™ on Stack Overflow

Bash IFS causes unexected results with array output

2 Answers 2

1 Comment

2 Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

1 Comment

2 Comments

Your Answer

Sign up or log in

Post as a guest

Related