Can SSDs execute copy-paste locally, on SSD, with minimal data transfer to CPU?
When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?
For example, the following python code:
from shutil import copyfile
src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)
I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...
ssd copy-paste data-transfer
add a comment |
When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?
For example, the following python code:
from shutil import copyfile
src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)
I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...
ssd copy-paste data-transfer
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10
add a comment |
When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?
For example, the following python code:
from shutil import copyfile
src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)
I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...
ssd copy-paste data-transfer
When duplicating a file (copy-paste in OS, or in python), for example, an image, is all the raw data read from the SSD and passed via the CPU and then back to the SSD, or is only some metadata read to the CPU and the rest handled by the local SSD controller?
Are there SSDs capable of local copy-paste, i.e. file raw data not leaving the SSD?
For example, the following python code:
from shutil import copyfile
src = "src.JPG"
dst = "dst.JPG"
copyfile(src, dst)
I would like to reduce the energy consumption of data migration and make use of the high internal SSD bandwidth...
ssd copy-paste data-transfer
ssd copy-paste data-transfer
asked Feb 1 at 15:45
SergiuSergiu
61
61
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10
add a comment |
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10
add a comment |
2 Answers
2
active
oldest
votes
SSDs cannot do that - not, at least, with current firmware, and not with current OSs.
SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".
There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.
So - the drive just doesn't "know" how to do that.
A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.
I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.
Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.
add a comment |
In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.
Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).
In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1401035%2fcan-ssds-execute-copy-paste-locally-on-ssd-with-minimal-data-transfer-to-cpu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
SSDs cannot do that - not, at least, with current firmware, and not with current OSs.
SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".
There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.
So - the drive just doesn't "know" how to do that.
A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.
I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.
Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.
add a comment |
SSDs cannot do that - not, at least, with current firmware, and not with current OSs.
SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".
There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.
So - the drive just doesn't "know" how to do that.
A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.
I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.
Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.
add a comment |
SSDs cannot do that - not, at least, with current firmware, and not with current OSs.
SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".
There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.
So - the drive just doesn't "know" how to do that.
A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.
I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.
Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.
SSDs cannot do that - not, at least, with current firmware, and not with current OSs.
SSDs implement the same command set (with a few additions like TRIM) as traditional SATA hard drives. There is simply no command you can send the drive for "copy the contents of block x to block y".
There are commands for reading from the drive, i.e. copying the contents of specified blocks on the drive into system RAM. And there are commands for writing, i.e. copying from system RAM to specified blocks on the drive. And there are a LOT of other commands that are not read or write commands. But there are no "copy within the drive" commands.
So - the drive just doesn't "know" how to do that.
A "copy" command could be implemented in the drive firmware. But the job wouldn't be done there. The OS developers would have to modify the various OS's file system and disk drivers to support it, and would have to provide an API for it. And each OS's file management utilities would have to be modified to use it. But today, since no SSDs (or hard drives) implement an onboard "copy" command, the OSs don't have support for such a thing either.
I know you asked about files, but drives don't know anything about files. Or directories ("folders"), or even partitions for that matter. The OS's file system and other drivers implement all that stuff. The read and write commands sent to the drive always work in terms of block numbers. The OS file system driver knows which blocks make up each file. For this question, though, this doesn't matter. Drives don't implement any sort of "copy" function, whether in terms of blocks or in terms of files.
Btw, if you're curious about what is in the ATA command set, you can find a freely available and relatively recent version of the spec here.
edited Feb 4 at 13:45
answered Feb 1 at 15:54
Jamie HanrahanJamie Hanrahan
18.9k34280
18.9k34280
add a comment |
add a comment |
In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.
Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).
In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).
add a comment |
In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.
Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).
In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).
add a comment |
In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.
Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).
In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).
In a word No. Filenames are a fairly high-level concept, and they abstract away a great many complexities making a hardware copy-paste impossible.
Your example code or using a file manager both require a filesystem to operate on which is a service provided by the operating system. A raw disk as the disk controller presents it, is just a collection of 1's and 0's. The Operating systems filesystem determines how the OS writes data to the disk, and maintains various data structures on the disk to describe it. The Operating System "decides" what data to return when an application opens and reads a given filename. When writing data, its the OS/filesystem that records physically where on the disk the binary data for a given filename is stored, with raw data often being split up to best use the available unused blocks of disk (aka fragmentation).
In terms of energy consumption, i dont think you will have a lot of control here beyond choosing efficient hardware (high PSU efficency and low wattage CPU/mobo). If you can operate on raw disks (ie clone the entire disk at a binary level - ie the filesystem and all its contents) you could use a hardware disk copier which would likely be quicker and much lower wattage (but probably not what your looking for).
answered Feb 1 at 17:23
MisterSmithMisterSmith
2945
2945
add a comment |
add a comment |
Thanks for contributing an answer to Super User!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1401035%2fcan-ssds-execute-copy-paste-locally-on-ssd-with-minimal-data-transfer-to-cpu%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
So use a COW filesystem and tools that support it.
– Daniel B
Feb 2 at 20:10