Could PATH contain newlines?
It is known that a path could contain newlines in any of its components.
Should we conclude then that the environment variable $PATH
could contain newlines ?
If so, how to split the $PATH
into its elements, similar to (Bourne like):
IFS=':' ; set -f
for var in $PATH
do
echo "<$var>"
done
But if it could be done without changing IFS, even better.
shell path newlines
add a comment |
It is known that a path could contain newlines in any of its components.
Should we conclude then that the environment variable $PATH
could contain newlines ?
If so, how to split the $PATH
into its elements, similar to (Bourne like):
IFS=':' ; set -f
for var in $PATH
do
echo "<$var>"
done
But if it could be done without changing IFS, even better.
shell path newlines
Note that in the Bourne shell (contrary to POSIX shells),/bin::/usr/bin
would be split into/bin
and/usr/bin
instead of/bin
,""
and/usr/bin
.
– Stéphane Chazelas
Dec 31 '18 at 11:12
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55
add a comment |
It is known that a path could contain newlines in any of its components.
Should we conclude then that the environment variable $PATH
could contain newlines ?
If so, how to split the $PATH
into its elements, similar to (Bourne like):
IFS=':' ; set -f
for var in $PATH
do
echo "<$var>"
done
But if it could be done without changing IFS, even better.
shell path newlines
It is known that a path could contain newlines in any of its components.
Should we conclude then that the environment variable $PATH
could contain newlines ?
If so, how to split the $PATH
into its elements, similar to (Bourne like):
IFS=':' ; set -f
for var in $PATH
do
echo "<$var>"
done
But if it could be done without changing IFS, even better.
shell path newlines
shell path newlines
edited Dec 31 '18 at 10:39
ilkkachu
56.1k784155
56.1k784155
asked Dec 31 '18 at 6:34
Isaac
11.2k11649
11.2k11649
Note that in the Bourne shell (contrary to POSIX shells),/bin::/usr/bin
would be split into/bin
and/usr/bin
instead of/bin
,""
and/usr/bin
.
– Stéphane Chazelas
Dec 31 '18 at 11:12
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55
add a comment |
Note that in the Bourne shell (contrary to POSIX shells),/bin::/usr/bin
would be split into/bin
and/usr/bin
instead of/bin
,""
and/usr/bin
.
– Stéphane Chazelas
Dec 31 '18 at 11:12
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55
Note that in the Bourne shell (contrary to POSIX shells),
/bin::/usr/bin
would be split into /bin
and /usr/bin
instead of /bin
, ""
and /usr/bin
.– Stéphane Chazelas
Dec 31 '18 at 11:12
Note that in the Bourne shell (contrary to POSIX shells),
/bin::/usr/bin
would be split into /bin
and /usr/bin
instead of /bin
, ""
and /usr/bin
.– Stéphane Chazelas
Dec 31 '18 at 11:12
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55
add a comment |
2 Answers
2
active
oldest
votes
In POSIX shells, $IFS
is a field delimiter, not separator, so a $PATH
value like /bin:/usr/bin:
would be split into /bin
and /usr/bin
instead of /bin
, /usr/bin
and the empty string (meaning the current directory). You need:
IFS=:; set -o noglob
for var in $PATH""; do
printf '<%s>n' "$var"
done
To avoid modifying global settings, you can use a shell with explicit splitting operators like zsh
:
for var in "${(s/:/@)PATH}"; do
printf '<%s>n' "$var"
done
Though in that case, zsh
already has the $path
array tied to $PATH
like in csh
/tcsh
, so:
for var in "$path[@]"; do
printf '<%s>n' "$var"
done
In any case, yes, in theory $PATH
like any variable could contain newline characters, the newline character is not special in any way when it comes to file path resolution. I don't expect anyone sensible would put a directory with newline (or wildcards) in their $PATH
or name a command with newline in its name. It's also hard to imagine a scenario where someone could exploit a script that makes the assumption that $PATH
won't contain newline characters.
Note thatset -f
is more portable thanset -o noglob
. (Yes, less legible, but what can we do?).
– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's-f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way inzsh
unless you're in sh emulation...
– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants onset -f
we must useset -o noglob
to be compatible with zsh?
– Isaac
2 days ago
I'm saying thatset -o noglob
is more portable among the shells that can run that code if we want to considerzsh -o shwordsplit -o globsubst
in that list.set -f
is more portable among ancient Bourne-like shells, but those shells that don't supportset -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features fromcsh
(...)
– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions.zsh
fixed that bug, soset -o noglob
is not needed there unless in sh emulation (whereset -f
works to disable it) or the globsubst option is enabled.
– Stéphane Chazelas
yesterday
add a comment |
Yes, PATH
can contain newlines (even on ancient Unix system).
As to splitting any string in shell, the only way you can do it portably is with IFS
. You can use IFS=:; set -f; set -- $PATH
or pass it to a function instead of looping with for
, though.
With bash
you can also "read" a string into an array:
xtra=$'somenothernplacenn'; PATH="$PATH:$xtra"
mapfile -td: path < <(printf %s "$PATH")
printf '<%s>n' "${path[@]}"
But using arrays is usually not a good idea, because they can't be stored transparently in environment variables or passed as a single argument to external commands.
Notice that IFS
will terminate fields, not separate them (kind of like n
at the end of the file won't be treated like an empty line by programs reading the file line-by-line); if that's not what's expected, and you really want to create an extra empty field at the end when splitting a string that ends in a character from IFS
, you should join an empty string after the variable that is subject to word splitting:
(P=/bin:; IFS=:; printf '<%s>n' $P"")
</bin>
<>
The word splitting algorithm will also ignore white space characters at the beginning of the string, if those whitespace characters are part of IFS
. If you want an extra field for the leading whitespace, you should also join an empty string before the variable:
(P=' foo : bar '; IFS=': '; set -f; set -- $P; printf '<%s>n' "$@")
<foo>
<bar>
(P=' foo : bar '; IFS=': '; set -f; set -- ""$P""; printf '<%s>n' "$@")
<>
<foo>
<bar>
<>
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need theset -f
to take effect before the$PATH
expansion. So it should beset -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.
– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
|
show 6 more comments
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491706%2fcould-path-contain-newlines%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
In POSIX shells, $IFS
is a field delimiter, not separator, so a $PATH
value like /bin:/usr/bin:
would be split into /bin
and /usr/bin
instead of /bin
, /usr/bin
and the empty string (meaning the current directory). You need:
IFS=:; set -o noglob
for var in $PATH""; do
printf '<%s>n' "$var"
done
To avoid modifying global settings, you can use a shell with explicit splitting operators like zsh
:
for var in "${(s/:/@)PATH}"; do
printf '<%s>n' "$var"
done
Though in that case, zsh
already has the $path
array tied to $PATH
like in csh
/tcsh
, so:
for var in "$path[@]"; do
printf '<%s>n' "$var"
done
In any case, yes, in theory $PATH
like any variable could contain newline characters, the newline character is not special in any way when it comes to file path resolution. I don't expect anyone sensible would put a directory with newline (or wildcards) in their $PATH
or name a command with newline in its name. It's also hard to imagine a scenario where someone could exploit a script that makes the assumption that $PATH
won't contain newline characters.
Note thatset -f
is more portable thanset -o noglob
. (Yes, less legible, but what can we do?).
– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's-f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way inzsh
unless you're in sh emulation...
– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants onset -f
we must useset -o noglob
to be compatible with zsh?
– Isaac
2 days ago
I'm saying thatset -o noglob
is more portable among the shells that can run that code if we want to considerzsh -o shwordsplit -o globsubst
in that list.set -f
is more portable among ancient Bourne-like shells, but those shells that don't supportset -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features fromcsh
(...)
– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions.zsh
fixed that bug, soset -o noglob
is not needed there unless in sh emulation (whereset -f
works to disable it) or the globsubst option is enabled.
– Stéphane Chazelas
yesterday
add a comment |
In POSIX shells, $IFS
is a field delimiter, not separator, so a $PATH
value like /bin:/usr/bin:
would be split into /bin
and /usr/bin
instead of /bin
, /usr/bin
and the empty string (meaning the current directory). You need:
IFS=:; set -o noglob
for var in $PATH""; do
printf '<%s>n' "$var"
done
To avoid modifying global settings, you can use a shell with explicit splitting operators like zsh
:
for var in "${(s/:/@)PATH}"; do
printf '<%s>n' "$var"
done
Though in that case, zsh
already has the $path
array tied to $PATH
like in csh
/tcsh
, so:
for var in "$path[@]"; do
printf '<%s>n' "$var"
done
In any case, yes, in theory $PATH
like any variable could contain newline characters, the newline character is not special in any way when it comes to file path resolution. I don't expect anyone sensible would put a directory with newline (or wildcards) in their $PATH
or name a command with newline in its name. It's also hard to imagine a scenario where someone could exploit a script that makes the assumption that $PATH
won't contain newline characters.
Note thatset -f
is more portable thanset -o noglob
. (Yes, less legible, but what can we do?).
– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's-f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way inzsh
unless you're in sh emulation...
– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants onset -f
we must useset -o noglob
to be compatible with zsh?
– Isaac
2 days ago
I'm saying thatset -o noglob
is more portable among the shells that can run that code if we want to considerzsh -o shwordsplit -o globsubst
in that list.set -f
is more portable among ancient Bourne-like shells, but those shells that don't supportset -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features fromcsh
(...)
– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions.zsh
fixed that bug, soset -o noglob
is not needed there unless in sh emulation (whereset -f
works to disable it) or the globsubst option is enabled.
– Stéphane Chazelas
yesterday
add a comment |
In POSIX shells, $IFS
is a field delimiter, not separator, so a $PATH
value like /bin:/usr/bin:
would be split into /bin
and /usr/bin
instead of /bin
, /usr/bin
and the empty string (meaning the current directory). You need:
IFS=:; set -o noglob
for var in $PATH""; do
printf '<%s>n' "$var"
done
To avoid modifying global settings, you can use a shell with explicit splitting operators like zsh
:
for var in "${(s/:/@)PATH}"; do
printf '<%s>n' "$var"
done
Though in that case, zsh
already has the $path
array tied to $PATH
like in csh
/tcsh
, so:
for var in "$path[@]"; do
printf '<%s>n' "$var"
done
In any case, yes, in theory $PATH
like any variable could contain newline characters, the newline character is not special in any way when it comes to file path resolution. I don't expect anyone sensible would put a directory with newline (or wildcards) in their $PATH
or name a command with newline in its name. It's also hard to imagine a scenario where someone could exploit a script that makes the assumption that $PATH
won't contain newline characters.
In POSIX shells, $IFS
is a field delimiter, not separator, so a $PATH
value like /bin:/usr/bin:
would be split into /bin
and /usr/bin
instead of /bin
, /usr/bin
and the empty string (meaning the current directory). You need:
IFS=:; set -o noglob
for var in $PATH""; do
printf '<%s>n' "$var"
done
To avoid modifying global settings, you can use a shell with explicit splitting operators like zsh
:
for var in "${(s/:/@)PATH}"; do
printf '<%s>n' "$var"
done
Though in that case, zsh
already has the $path
array tied to $PATH
like in csh
/tcsh
, so:
for var in "$path[@]"; do
printf '<%s>n' "$var"
done
In any case, yes, in theory $PATH
like any variable could contain newline characters, the newline character is not special in any way when it comes to file path resolution. I don't expect anyone sensible would put a directory with newline (or wildcards) in their $PATH
or name a command with newline in its name. It's also hard to imagine a scenario where someone could exploit a script that makes the assumption that $PATH
won't contain newline characters.
edited Dec 31 '18 at 11:17
answered Dec 31 '18 at 10:46
Stéphane Chazelas
300k54564913
300k54564913
Note thatset -f
is more portable thanset -o noglob
. (Yes, less legible, but what can we do?).
– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's-f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way inzsh
unless you're in sh emulation...
– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants onset -f
we must useset -o noglob
to be compatible with zsh?
– Isaac
2 days ago
I'm saying thatset -o noglob
is more portable among the shells that can run that code if we want to considerzsh -o shwordsplit -o globsubst
in that list.set -f
is more portable among ancient Bourne-like shells, but those shells that don't supportset -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features fromcsh
(...)
– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions.zsh
fixed that bug, soset -o noglob
is not needed there unless in sh emulation (whereset -f
works to disable it) or the globsubst option is enabled.
– Stéphane Chazelas
yesterday
add a comment |
Note thatset -f
is more portable thanset -o noglob
. (Yes, less legible, but what can we do?).
– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's-f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way inzsh
unless you're in sh emulation...
– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants onset -f
we must useset -o noglob
to be compatible with zsh?
– Isaac
2 days ago
I'm saying thatset -o noglob
is more portable among the shells that can run that code if we want to considerzsh -o shwordsplit -o globsubst
in that list.set -f
is more portable among ancient Bourne-like shells, but those shells that don't supportset -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features fromcsh
(...)
– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions.zsh
fixed that bug, soset -o noglob
is not needed there unless in sh emulation (whereset -f
works to disable it) or the globsubst option is enabled.
– Stéphane Chazelas
yesterday
Note that
set -f
is more portable than set -o noglob
. (Yes, less legible, but what can we do?).– Isaac
Jan 1 at 0:29
Note that
set -f
is more portable than set -o noglob
. (Yes, less legible, but what can we do?).– Isaac
Jan 1 at 0:29
@Isaac (happy new year to you to). well, that's arguable.
noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's -f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way in zsh
unless you're in sh emulation...– Stéphane Chazelas
2 days ago
@Isaac (happy new year to you to). well, that's arguable.
noglob
is POSIX and not Bourne, but the Bourne shell didn't support splitting that way, so it's just as well that an error be reported in that case, to tell users to use a POSIX shell instead to interpret that script. On the other hand, zsh's -f
option is the one from csh, not Bourne, except in sh emulation. Granted, there's no point splitting that way in zsh
unless you're in sh emulation...– Stéphane Chazelas
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants on
set -f
we must use set -o noglob
to be compatible with zsh?– Isaac
2 days ago
So, are you saying that because zsh took the rational decision of being non-compatible with all the other Bourne shell descendants on
set -f
we must use set -o noglob
to be compatible with zsh?– Isaac
2 days ago
I'm saying that
set -o noglob
is more portable among the shells that can run that code if we want to consider zsh -o shwordsplit -o globsubst
in that list. set -f
is more portable among ancient Bourne-like shells, but those shells that don't support set -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features from csh
(...)– Stéphane Chazelas
yesterday
I'm saying that
set -o noglob
is more portable among the shells that can run that code if we want to consider zsh -o shwordsplit -o globsubst
in that list. set -f
is more portable among ancient Bourne-like shells, but those shells that don't support set -o noglob
cannot run that code correctly anyway. When zsh was written in 1990, csh/tcsh were by far the most popular shells at the time. All of ksh/bash/zsh borrowed features from csh
(...)– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its
-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions. zsh
fixed that bug, so set -o noglob
is not needed there unless in sh emulation (where set -f
works to disable it) or the globsubst option is enabled.– Stéphane Chazelas
yesterday
(...) csh had the -f option (for fast start) long before the Bourne shell added its
-f
to disable glob. So if you want to blame something for breaking compatibility, blame the Bourne (SysV) shell. There would be not reason why one would want to disable glob if it weren't for that bug of the Bourne shell whereby globbing is performed upon expansions. zsh
fixed that bug, so set -o noglob
is not needed there unless in sh emulation (where set -f
works to disable it) or the globsubst option is enabled.– Stéphane Chazelas
yesterday
add a comment |
Yes, PATH
can contain newlines (even on ancient Unix system).
As to splitting any string in shell, the only way you can do it portably is with IFS
. You can use IFS=:; set -f; set -- $PATH
or pass it to a function instead of looping with for
, though.
With bash
you can also "read" a string into an array:
xtra=$'somenothernplacenn'; PATH="$PATH:$xtra"
mapfile -td: path < <(printf %s "$PATH")
printf '<%s>n' "${path[@]}"
But using arrays is usually not a good idea, because they can't be stored transparently in environment variables or passed as a single argument to external commands.
Notice that IFS
will terminate fields, not separate them (kind of like n
at the end of the file won't be treated like an empty line by programs reading the file line-by-line); if that's not what's expected, and you really want to create an extra empty field at the end when splitting a string that ends in a character from IFS
, you should join an empty string after the variable that is subject to word splitting:
(P=/bin:; IFS=:; printf '<%s>n' $P"")
</bin>
<>
The word splitting algorithm will also ignore white space characters at the beginning of the string, if those whitespace characters are part of IFS
. If you want an extra field for the leading whitespace, you should also join an empty string before the variable:
(P=' foo : bar '; IFS=': '; set -f; set -- $P; printf '<%s>n' "$@")
<foo>
<bar>
(P=' foo : bar '; IFS=': '; set -f; set -- ""$P""; printf '<%s>n' "$@")
<>
<foo>
<bar>
<>
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need theset -f
to take effect before the$PATH
expansion. So it should beset -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.
– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
|
show 6 more comments
Yes, PATH
can contain newlines (even on ancient Unix system).
As to splitting any string in shell, the only way you can do it portably is with IFS
. You can use IFS=:; set -f; set -- $PATH
or pass it to a function instead of looping with for
, though.
With bash
you can also "read" a string into an array:
xtra=$'somenothernplacenn'; PATH="$PATH:$xtra"
mapfile -td: path < <(printf %s "$PATH")
printf '<%s>n' "${path[@]}"
But using arrays is usually not a good idea, because they can't be stored transparently in environment variables or passed as a single argument to external commands.
Notice that IFS
will terminate fields, not separate them (kind of like n
at the end of the file won't be treated like an empty line by programs reading the file line-by-line); if that's not what's expected, and you really want to create an extra empty field at the end when splitting a string that ends in a character from IFS
, you should join an empty string after the variable that is subject to word splitting:
(P=/bin:; IFS=:; printf '<%s>n' $P"")
</bin>
<>
The word splitting algorithm will also ignore white space characters at the beginning of the string, if those whitespace characters are part of IFS
. If you want an extra field for the leading whitespace, you should also join an empty string before the variable:
(P=' foo : bar '; IFS=': '; set -f; set -- $P; printf '<%s>n' "$@")
<foo>
<bar>
(P=' foo : bar '; IFS=': '; set -f; set -- ""$P""; printf '<%s>n' "$@")
<>
<foo>
<bar>
<>
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need theset -f
to take effect before the$PATH
expansion. So it should beset -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.
– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
|
show 6 more comments
Yes, PATH
can contain newlines (even on ancient Unix system).
As to splitting any string in shell, the only way you can do it portably is with IFS
. You can use IFS=:; set -f; set -- $PATH
or pass it to a function instead of looping with for
, though.
With bash
you can also "read" a string into an array:
xtra=$'somenothernplacenn'; PATH="$PATH:$xtra"
mapfile -td: path < <(printf %s "$PATH")
printf '<%s>n' "${path[@]}"
But using arrays is usually not a good idea, because they can't be stored transparently in environment variables or passed as a single argument to external commands.
Notice that IFS
will terminate fields, not separate them (kind of like n
at the end of the file won't be treated like an empty line by programs reading the file line-by-line); if that's not what's expected, and you really want to create an extra empty field at the end when splitting a string that ends in a character from IFS
, you should join an empty string after the variable that is subject to word splitting:
(P=/bin:; IFS=:; printf '<%s>n' $P"")
</bin>
<>
The word splitting algorithm will also ignore white space characters at the beginning of the string, if those whitespace characters are part of IFS
. If you want an extra field for the leading whitespace, you should also join an empty string before the variable:
(P=' foo : bar '; IFS=': '; set -f; set -- $P; printf '<%s>n' "$@")
<foo>
<bar>
(P=' foo : bar '; IFS=': '; set -f; set -- ""$P""; printf '<%s>n' "$@")
<>
<foo>
<bar>
<>
Yes, PATH
can contain newlines (even on ancient Unix system).
As to splitting any string in shell, the only way you can do it portably is with IFS
. You can use IFS=:; set -f; set -- $PATH
or pass it to a function instead of looping with for
, though.
With bash
you can also "read" a string into an array:
xtra=$'somenothernplacenn'; PATH="$PATH:$xtra"
mapfile -td: path < <(printf %s "$PATH")
printf '<%s>n' "${path[@]}"
But using arrays is usually not a good idea, because they can't be stored transparently in environment variables or passed as a single argument to external commands.
Notice that IFS
will terminate fields, not separate them (kind of like n
at the end of the file won't be treated like an empty line by programs reading the file line-by-line); if that's not what's expected, and you really want to create an extra empty field at the end when splitting a string that ends in a character from IFS
, you should join an empty string after the variable that is subject to word splitting:
(P=/bin:; IFS=:; printf '<%s>n' $P"")
</bin>
<>
The word splitting algorithm will also ignore white space characters at the beginning of the string, if those whitespace characters are part of IFS
. If you want an extra field for the leading whitespace, you should also join an empty string before the variable:
(P=' foo : bar '; IFS=': '; set -f; set -- $P; printf '<%s>n' "$@")
<foo>
<bar>
(P=' foo : bar '; IFS=': '; set -f; set -- ""$P""; printf '<%s>n' "$@")
<>
<foo>
<bar>
<>
edited 2 days ago
answered Dec 31 '18 at 9:20
pizdelect
36716
36716
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need theset -f
to take effect before the$PATH
expansion. So it should beset -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.
– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
|
show 6 more comments
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need theset -f
to take effect before the$PATH
expansion. So it should beset -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.
– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
Using arrays is often an excellent idea, as a number of answers here on unix.SE show. It's almost impossible to handle lists of strings with arbitrary data without using an array. You only need lists of paths with whitespace, or a list of command arguments to get the issue. Of course you can use the positional parameters instead of an array, but those aren't any better regarding the points you mention: they can't be sanely pushed through the environment, nor passed as a single argument to external commands.
– ilkkachu
Dec 31 '18 at 10:32
No, you need the
set -f
to take effect before the $PATH
expansion. So it should be set -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
No, you need the
set -f
to take effect before the $PATH
expansion. So it should be set -o noglob; set -- $PATH""
– Stéphane Chazelas
Dec 31 '18 at 10:41
@ilkkachu fwiw, quoting the here-string variable is not needed:
x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.– pizdelect
Dec 31 '18 at 12:14
@ilkkachu fwiw, quoting the here-string variable is not needed:
x='a b'; mapfile -td: <<< $x y; printf '<%s>n' "$y"
; but the added trailing newline is a problem, really.– pizdelect
Dec 31 '18 at 12:14
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@ilkkachu and that's documented in the bash manual, under "Here Strings": "Pathname expansion and word splitting are not performed"
– pizdelect
Dec 31 '18 at 12:25
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
@StéphaneChazelas thanks, I've changed it to use a process substitution instead.
– pizdelect
Dec 31 '18 at 12:36
|
show 6 more comments
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f491706%2fcould-path-contain-newlines%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Note that in the Bourne shell (contrary to POSIX shells),
/bin::/usr/bin
would be split into/bin
and/usr/bin
instead of/bin
,""
and/usr/bin
.– Stéphane Chazelas
Dec 31 '18 at 11:12
I can't reproduce. That preserving of empty elements for non-whitespace was introduced by the Korn shell, that's a very well known difference with the Bourne shell.
– Stéphane Chazelas
Dec 31 '18 at 16:55