Can SSDs execute copy-paste locally, on SSD, with minimal data transfer to CPU?












1















When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?



For example, the following python code:



from shutil import copyfile

src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)


I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...










share|improve this question























  • So use a COW filesystem and tools that support it.

    – Daniel B
    Feb 2 at 20:10
















1















When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?



For example, the following python code:



from shutil import copyfile

src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)


I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...










share|improve this question























  • So use a COW filesystem and tools that support it.

    – Daniel B
    Feb 2 at 20:10














1












1








1








When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?



For example, the following python code:



from shutil import copyfile

src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)


I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...










share|improve this question














When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?



For example, the following python code:



from shutil import copyfile

src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)


I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...







ssd copy-paste data-transfer






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Feb 1 at 15:45









SergiuSergiu

61




61













  • So use a COW filesystem and tools that support it.

    – Daniel B
    Feb 2 at 20:10



















  • So use a COW filesystem and tools that support it.

    – Daniel B
    Feb 2 at 20:10

















So use a COW filesystem and tools that support it.

– Daniel B
Feb 2 at 20:10





So use a COW filesystem and tools that support it.

– Daniel B
Feb 2 at 20:10










2 Answers
2






active

oldest

votes


















4














SSDs cannot do that - not, at least, with current firmware, and not with current OSs.



SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".



There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.



So - the drive just doesn't "know" how to do that.



A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.



I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.



Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.






share|improve this answer

































    0














    In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.



    Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).



    In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).






    share|improve this answer
























      Your Answer








      StackExchange.ready(function() {
      var channelOptions = {
      tags: "".split(" "),
      id: "3"
      };
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function() {
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled) {
      StackExchange.using("snippets", function() {
      createEditor();
      });
      }
      else {
      createEditor();
      }
      });

      function createEditor() {
      StackExchange.prepareEditor({
      heartbeatType: 'answer',
      autoActivateHeartbeat: false,
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader: {
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      },
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      });


      }
      });














      draft saved

      draft discarded


















      StackExchange.ready(
      function () {
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1401035%2fcan-ssds-execute-copy-paste-locally-on-ssd-with-minimal-data-transfer-to-cpu%23new-answer', 'question_page');
      }
      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes









      4














      SSDs cannot do that - not, at least, with current firmware, and not with current OSs.



      SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".



      There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.



      So - the drive just doesn't "know" how to do that.



      A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.



      I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.



      Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.






      share|improve this answer






























        4














        SSDs cannot do that - not, at least, with current firmware, and not with current OSs.



        SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".



        There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.



        So - the drive just doesn't "know" how to do that.



        A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.



        I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.



        Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.






        share|improve this answer




























          4












          4








          4







          SSDs cannot do that - not, at least, with current firmware, and not with current OSs.



          SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".



          There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.



          So - the drive just doesn't "know" how to do that.



          A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.



          I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.



          Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.






          share|improve this answer















          SSDs cannot do that - not, at least, with current firmware, and not with current OSs.



          SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".



          There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.



          So - the drive just doesn't "know" how to do that.



          A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.



          I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.



          Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.







          share|improve this answer














          share|improve this answer



          share|improve this answer








          edited Feb 4 at 13:45

























          answered Feb 1 at 15:54









          Jamie HanrahanJamie Hanrahan

          18.9k34280




          18.9k34280

























              0














              In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.



              Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).



              In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).






              share|improve this answer




























                0














                In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.



                Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).



                In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).






                share|improve this answer


























                  0












                  0








                  0







                  In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.



                  Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).



                  In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).






                  share|improve this answer













                  In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.



                  Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).



                  In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Feb 1 at 17:23









                  MisterSmithMisterSmith

                  2945




                  2945






























                      draft saved

                      draft discarded




















































                      Thanks for contributing an answer to Super User!


                      • Please be sure to answer the question. Provide details and share your research!

                      But avoid



                      • Asking for help, clarification, or responding to other answers.

                      • Making statements based on opinion; back them up with references or personal experience.


                      To learn more, see our tips on writing great answers.




                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function () {
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1401035%2fcan-ssds-execute-copy-paste-locally-on-ssd-with-minimal-data-transfer-to-cpu%23new-answer', 'question_page');
                      }
                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Terni

                      A new problem with tex4ht and tikz

                      Sun Ra