Why does POSIX require certain shell built-ins to have an external implementation?












7















From this question about whether printf is a built-in for yash, comes this answer that quotes the POSIX standard.



The answer points out that the POSIX search sequence is to find an external implementation of the desired command, and then, if the shell has implemented it as a built-in, run the built-in. (For built-ins that aren't special built-ins.)



Why does POSIX have this requirement for an external implementation to exist before allowing an internal implementation to be run?



It seems... arbitrary, so I am curious.










share|improve this question























  • I believe that is a way to enable/disable builtins if desired/required.

    – Isaac
    8 hours ago






  • 2





    Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

    – studog
    8 hours ago











  • @studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

    – Wildcard
    7 hours ago











  • @Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

    – Kusalananda
    7 hours ago













  • @Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

    – Wildcard
    7 hours ago
















7















From this question about whether printf is a built-in for yash, comes this answer that quotes the POSIX standard.



The answer points out that the POSIX search sequence is to find an external implementation of the desired command, and then, if the shell has implemented it as a built-in, run the built-in. (For built-ins that aren't special built-ins.)



Why does POSIX have this requirement for an external implementation to exist before allowing an internal implementation to be run?



It seems... arbitrary, so I am curious.










share|improve this question























  • I believe that is a way to enable/disable builtins if desired/required.

    – Isaac
    8 hours ago






  • 2





    Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

    – studog
    8 hours ago











  • @studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

    – Wildcard
    7 hours ago











  • @Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

    – Kusalananda
    7 hours ago













  • @Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

    – Wildcard
    7 hours ago














7












7








7


1






From this question about whether printf is a built-in for yash, comes this answer that quotes the POSIX standard.



The answer points out that the POSIX search sequence is to find an external implementation of the desired command, and then, if the shell has implemented it as a built-in, run the built-in. (For built-ins that aren't special built-ins.)



Why does POSIX have this requirement for an external implementation to exist before allowing an internal implementation to be run?



It seems... arbitrary, so I am curious.










share|improve this question














From this question about whether printf is a built-in for yash, comes this answer that quotes the POSIX standard.



The answer points out that the POSIX search sequence is to find an external implementation of the desired command, and then, if the shell has implemented it as a built-in, run the built-in. (For built-ins that aren't special built-ins.)



Why does POSIX have this requirement for an external implementation to exist before allowing an internal implementation to be run?



It seems... arbitrary, so I am curious.







shell posix shell-builtin






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked 8 hours ago









studogstudog

30017




30017













  • I believe that is a way to enable/disable builtins if desired/required.

    – Isaac
    8 hours ago






  • 2





    Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

    – studog
    8 hours ago











  • @studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

    – Wildcard
    7 hours ago











  • @Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

    – Kusalananda
    7 hours ago













  • @Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

    – Wildcard
    7 hours ago



















  • I believe that is a way to enable/disable builtins if desired/required.

    – Isaac
    8 hours ago






  • 2





    Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

    – studog
    8 hours ago











  • @studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

    – Wildcard
    7 hours ago











  • @Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

    – Kusalananda
    7 hours ago













  • @Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

    – Wildcard
    7 hours ago

















I believe that is a way to enable/disable builtins if desired/required.

– Isaac
8 hours ago





I believe that is a way to enable/disable builtins if desired/required.

– Isaac
8 hours ago




2




2





Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

– studog
8 hours ago





Disabling the built-in by removing the external implementation? Now there are no commands of name printf available.

– studog
8 hours ago













@studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

– Wildcard
7 hours ago





@studog, so create an empty file with the same name as the built-in, turn on the execute bit, and put it in a directory in your PATH. :P

– Wildcard
7 hours ago













@Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

– Kusalananda
7 hours ago







@Wildcard A strictly compliant shell would then see the name while searching the PATH and then call the built-in utility, not the external script. What if you'd want to call the external script in your path? Hmm... This seems to call for a table describing the different possibilities. There is one here, but it doesn't make sense to me.

– Kusalananda
7 hours ago















@Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

– Wildcard
7 hours ago





@Kusalananda, re your first sentence, that was my point. Hence why I said to create an empty file.

– Wildcard
7 hours ago










2 Answers
2






active

oldest

votes


















3














That's quite absurd and that's why no shell is implementing it in its default mode.



That's probably up to shell & standard lawyers to determine, but my interpretation is that it was a botched attempt to have the regular built-ins associated with a path, and let the user override eg. printf by having their own printf implementation in /foo/bin and then setting PATH=/foo/bin:$PATH. The rationale and its illustrating example seem to confirm that.



However, the standard did not end up requiring that, but something completely different (and also useless and unexpected).



You can read more about it in this bug report. FWIW, I wasn't able to find any shell implementing the requirements from the final accepted text there, either.






share|improve this answer

































    2














    This is an "as if" rule.



    Simply put: The behaviour of the shell as users see it should not change if an implementation decides to make a standard external command also available as shell built-in.



    The contrast that I showed at https://unix.stackexchange.com/a/496291/5132 between the behaviours of (on the one hand) the PD Korn, MirBSD Korn, and Heirloom Bourne shells; (on the other hand) the Z, 93 Korn, Bourne Again, and Debian Almquist shells; and (on the gripping hand) the Watanabe shell highlights this.



    For the shells that do not have printf as a built-in, removing /usr/bin from PATH makes an invocation of printf stop working. The POSIX conformant behaviour, exhibited by the Watanabe shell in its conformant mode, causes the same result. The behaviour of the shell that has a printf built-in is as if it were invoking an external command.



    Whereas the behaviour of all of the non-conformant shells does not alter if /usr/bin is removed from PATH, and they do not behave as if they were invoking an external command.



    What the standard is trying to guarantee to you is that shells can build-in all sorts of normally external commands (or implement them as its own shell functions), and you'll still get the same behaviour from the built-ins as you did with the external commands if you adjust PATH to stop the commands from being found. PATH remains your tool for selecting and controlling what commands you can invoke.



    (As explained at https://unix.stackexchange.com/a/448799/5132, years ago people chose the personality of their Unix by changing what was on PATH.)



    One might opine that making the command always work irrespective of whether it can be found on PATH is in fact the point of making normally external commands built-in. (It's why my nosh toolset just gained a built-in printenv command in version 1.38, in fact. Although this is not a shell.)



    But the standard is giving you the guarantee that you'll see the same behaviour for regular external commands that are not on PATH from the shell as you will see from other non-shell programs invoking the execvpe() function, and the shell will not magically be able to run (apparently) ordinary external commands that other programs cannot find with the same PATH. Everything works self-consistently from the user's perspective, and PATH is the tool for controlling how it works.



    Further reading




    • Why are POSIX mandatory utilities not built into the shell?

    • Why is echo a shell built in command?

    • How does bash execute commands






    share|improve this answer























      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "106"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: false,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: null,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f496341%2fwhy-does-posix-require-certain-shell-built-ins-to-have-an-external-implementatio%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      3














      That's quite absurd and that's why no shell is implementing it in its default mode.



      That's probably up to shell & standard lawyers to determine, but my interpretation is that it was a botched attempt to have the regular built-ins associated with a path, and let the user override eg. printf by having their own printf implementation in /foo/bin and then setting PATH=/foo/bin:$PATH. The rationale and its illustrating example seem to confirm that.



      However, the standard did not end up requiring that, but something completely different (and also useless and unexpected).



      You can read more about it in this bug report. FWIW, I wasn't able to find any shell implementing the requirements from the final accepted text there, either.






      share|improve this answer






























        3














        That's quite absurd and that's why no shell is implementing it in its default mode.



        That's probably up to shell & standard lawyers to determine, but my interpretation is that it was a botched attempt to have the regular built-ins associated with a path, and let the user override eg. printf by having their own printf implementation in /foo/bin and then setting PATH=/foo/bin:$PATH. The rationale and its illustrating example seem to confirm that.



        However, the standard did not end up requiring that, but something completely different (and also useless and unexpected).



        You can read more about it in this bug report. FWIW, I wasn't able to find any shell implementing the requirements from the final accepted text there, either.






        share|improve this answer




























          3












          3








          3







          That's quite absurd and that's why no shell is implementing it in its default mode.



          That's probably up to shell & standard lawyers to determine, but my interpretation is that it was a botched attempt to have the regular built-ins associated with a path, and let the user override eg. printf by having their own printf implementation in /foo/bin and then setting PATH=/foo/bin:$PATH. The rationale and its illustrating example seem to confirm that.



          However, the standard did not end up requiring that, but something completely different (and also useless and unexpected).



          You can read more about it in this bug report. FWIW, I wasn't able to find any shell implementing the requirements from the final accepted text there, either.






          share|improve this answer















          That's quite absurd and that's why no shell is implementing it in its default mode.



          That's probably up to shell & standard lawyers to determine, but my interpretation is that it was a botched attempt to have the regular built-ins associated with a path, and let the user override eg. printf by having their own printf implementation in /foo/bin and then setting PATH=/foo/bin:$PATH. The rationale and its illustrating example seem to confirm that.



          However, the standard did not end up requiring that, but something completely different (and also useless and unexpected).



          You can read more about it in this bug report. FWIW, I wasn't able to find any shell implementing the requirements from the final accepted text there, either.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited 6 hours ago

























          answered 7 hours ago









          mosvymosvy

          6,3561426




          6,3561426

























              2














              This is an "as if" rule.



              Simply put: The behaviour of the shell as users see it should not change if an implementation decides to make a standard external command also available as shell built-in.



              The contrast that I showed at https://unix.stackexchange.com/a/496291/5132 between the behaviours of (on the one hand) the PD Korn, MirBSD Korn, and Heirloom Bourne shells; (on the other hand) the Z, 93 Korn, Bourne Again, and Debian Almquist shells; and (on the gripping hand) the Watanabe shell highlights this.



              For the shells that do not have printf as a built-in, removing /usr/bin from PATH makes an invocation of printf stop working. The POSIX conformant behaviour, exhibited by the Watanabe shell in its conformant mode, causes the same result. The behaviour of the shell that has a printf built-in is as if it were invoking an external command.



              Whereas the behaviour of all of the non-conformant shells does not alter if /usr/bin is removed from PATH, and they do not behave as if they were invoking an external command.



              What the standard is trying to guarantee to you is that shells can build-in all sorts of normally external commands (or implement them as its own shell functions), and you'll still get the same behaviour from the built-ins as you did with the external commands if you adjust PATH to stop the commands from being found. PATH remains your tool for selecting and controlling what commands you can invoke.



              (As explained at https://unix.stackexchange.com/a/448799/5132, years ago people chose the personality of their Unix by changing what was on PATH.)



              One might opine that making the command always work irrespective of whether it can be found on PATH is in fact the point of making normally external commands built-in. (It's why my nosh toolset just gained a built-in printenv command in version 1.38, in fact. Although this is not a shell.)



              But the standard is giving you the guarantee that you'll see the same behaviour for regular external commands that are not on PATH from the shell as you will see from other non-shell programs invoking the execvpe() function, and the shell will not magically be able to run (apparently) ordinary external commands that other programs cannot find with the same PATH. Everything works self-consistently from the user's perspective, and PATH is the tool for controlling how it works.



              Further reading




              • Why are POSIX mandatory utilities not built into the shell?

              • Why is echo a shell built in command?

              • How does bash execute commands






              share|improve this answer




























                2














                This is an "as if" rule.



                Simply put: The behaviour of the shell as users see it should not change if an implementation decides to make a standard external command also available as shell built-in.



                The contrast that I showed at https://unix.stackexchange.com/a/496291/5132 between the behaviours of (on the one hand) the PD Korn, MirBSD Korn, and Heirloom Bourne shells; (on the other hand) the Z, 93 Korn, Bourne Again, and Debian Almquist shells; and (on the gripping hand) the Watanabe shell highlights this.



                For the shells that do not have printf as a built-in, removing /usr/bin from PATH makes an invocation of printf stop working. The POSIX conformant behaviour, exhibited by the Watanabe shell in its conformant mode, causes the same result. The behaviour of the shell that has a printf built-in is as if it were invoking an external command.



                Whereas the behaviour of all of the non-conformant shells does not alter if /usr/bin is removed from PATH, and they do not behave as if they were invoking an external command.



                What the standard is trying to guarantee to you is that shells can build-in all sorts of normally external commands (or implement them as its own shell functions), and you'll still get the same behaviour from the built-ins as you did with the external commands if you adjust PATH to stop the commands from being found. PATH remains your tool for selecting and controlling what commands you can invoke.



                (As explained at https://unix.stackexchange.com/a/448799/5132, years ago people chose the personality of their Unix by changing what was on PATH.)



                One might opine that making the command always work irrespective of whether it can be found on PATH is in fact the point of making normally external commands built-in. (It's why my nosh toolset just gained a built-in printenv command in version 1.38, in fact. Although this is not a shell.)



                But the standard is giving you the guarantee that you'll see the same behaviour for regular external commands that are not on PATH from the shell as you will see from other non-shell programs invoking the execvpe() function, and the shell will not magically be able to run (apparently) ordinary external commands that other programs cannot find with the same PATH. Everything works self-consistently from the user's perspective, and PATH is the tool for controlling how it works.



                Further reading




                • Why are POSIX mandatory utilities not built into the shell?

                • Why is echo a shell built in command?

                • How does bash execute commands






                share|improve this answer


























                  2












                  2








                  2







                  This is an "as if" rule.



                  Simply put: The behaviour of the shell as users see it should not change if an implementation decides to make a standard external command also available as shell built-in.



                  The contrast that I showed at https://unix.stackexchange.com/a/496291/5132 between the behaviours of (on the one hand) the PD Korn, MirBSD Korn, and Heirloom Bourne shells; (on the other hand) the Z, 93 Korn, Bourne Again, and Debian Almquist shells; and (on the gripping hand) the Watanabe shell highlights this.



                  For the shells that do not have printf as a built-in, removing /usr/bin from PATH makes an invocation of printf stop working. The POSIX conformant behaviour, exhibited by the Watanabe shell in its conformant mode, causes the same result. The behaviour of the shell that has a printf built-in is as if it were invoking an external command.



                  Whereas the behaviour of all of the non-conformant shells does not alter if /usr/bin is removed from PATH, and they do not behave as if they were invoking an external command.



                  What the standard is trying to guarantee to you is that shells can build-in all sorts of normally external commands (or implement them as its own shell functions), and you'll still get the same behaviour from the built-ins as you did with the external commands if you adjust PATH to stop the commands from being found. PATH remains your tool for selecting and controlling what commands you can invoke.



                  (As explained at https://unix.stackexchange.com/a/448799/5132, years ago people chose the personality of their Unix by changing what was on PATH.)



                  One might opine that making the command always work irrespective of whether it can be found on PATH is in fact the point of making normally external commands built-in. (It's why my nosh toolset just gained a built-in printenv command in version 1.38, in fact. Although this is not a shell.)



                  But the standard is giving you the guarantee that you'll see the same behaviour for regular external commands that are not on PATH from the shell as you will see from other non-shell programs invoking the execvpe() function, and the shell will not magically be able to run (apparently) ordinary external commands that other programs cannot find with the same PATH. Everything works self-consistently from the user's perspective, and PATH is the tool for controlling how it works.



                  Further reading




                  • Why are POSIX mandatory utilities not built into the shell?

                  • Why is echo a shell built in command?

                  • How does bash execute commands






                  share|improve this answer













                  This is an "as if" rule.



                  Simply put: The behaviour of the shell as users see it should not change if an implementation decides to make a standard external command also available as shell built-in.



                  The contrast that I showed at https://unix.stackexchange.com/a/496291/5132 between the behaviours of (on the one hand) the PD Korn, MirBSD Korn, and Heirloom Bourne shells; (on the other hand) the Z, 93 Korn, Bourne Again, and Debian Almquist shells; and (on the gripping hand) the Watanabe shell highlights this.



                  For the shells that do not have printf as a built-in, removing /usr/bin from PATH makes an invocation of printf stop working. The POSIX conformant behaviour, exhibited by the Watanabe shell in its conformant mode, causes the same result. The behaviour of the shell that has a printf built-in is as if it were invoking an external command.



                  Whereas the behaviour of all of the non-conformant shells does not alter if /usr/bin is removed from PATH, and they do not behave as if they were invoking an external command.



                  What the standard is trying to guarantee to you is that shells can build-in all sorts of normally external commands (or implement them as its own shell functions), and you'll still get the same behaviour from the built-ins as you did with the external commands if you adjust PATH to stop the commands from being found. PATH remains your tool for selecting and controlling what commands you can invoke.



                  (As explained at https://unix.stackexchange.com/a/448799/5132, years ago people chose the personality of their Unix by changing what was on PATH.)



                  One might opine that making the command always work irrespective of whether it can be found on PATH is in fact the point of making normally external commands built-in. (It's why my nosh toolset just gained a built-in printenv command in version 1.38, in fact. Although this is not a shell.)



                  But the standard is giving you the guarantee that you'll see the same behaviour for regular external commands that are not on PATH from the shell as you will see from other non-shell programs invoking the execvpe() function, and the shell will not magically be able to run (apparently) ordinary external commands that other programs cannot find with the same PATH. Everything works self-consistently from the user's perspective, and PATH is the tool for controlling how it works.



                  Further reading




                  • Why are POSIX mandatory utilities not built into the shell?

                  • Why is echo a shell built in command?

                  • How does bash execute commands







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered 2 hours ago









                  JdeBPJdeBP

                  33.9k469160




                  33.9k469160






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Unix & Linux Stack Exchange!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f496341%2fwhy-does-posix-require-certain-shell-built-ins-to-have-an-external-implementatio%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Сан-Квентин

                      Алькесар

                      Josef Freinademetz