Video conversion bottleneck with ffmpeg












0















I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.



The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it



Here are my specs:



Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"










share|improve this question























  • Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

    – shawn
    Jan 29 at 20:36











  • @shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

    – J M
    Jan 29 at 20:48













  • Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

    – shawn
    Jan 29 at 21:09











  • @shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

    – J M
    Jan 30 at 0:11








  • 1





    Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

    – shawn
    Jan 30 at 0:53
















0















I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.



The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it



Here are my specs:



Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"










share|improve this question























  • Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

    – shawn
    Jan 29 at 20:36











  • @shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

    – J M
    Jan 29 at 20:48













  • Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

    – shawn
    Jan 29 at 21:09











  • @shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

    – J M
    Jan 30 at 0:11








  • 1





    Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

    – shawn
    Jan 30 at 0:53














0












0








0








I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.



The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it



Here are my specs:



Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"










share|improve this question














I'm curious to know which part of my setup is the main bottleneck in converting my video files to H.265 codecs. I'd like to avoid the debate about whether it is worth it or not, and let's assume I've done enough thought to decide it is worth my CPU time and effort to do this. I'd like to diagnose the bottleneck areas that are the most cost-effective or worthwhile to consider upgrading.



The video files reside on a 4-drive external array connected to the computer via USB2. I'm running ffmpeg via a script which is basically batch processing the entire library. I do not transfer any files off the external drive during processing. ffmpeg is called from the main OS drive. If there's other relevant info I've left out, please let me know and I can provide it



Here are my specs:



Operating System: Windows 8.1 64-bit (6.3, Build 9600) (9600.winblue_r4.141028-1500)
Computer: Dell XPS 8700
Processor: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz (8 CPUs), ~3.4GHz
Memory: 8192MB RAM
Card name: NVIDIA GeForce GT 635
Main OS HDD: WD Red 1TB NAS Hard Drive - 5400 RPM Class, SATA 6 Gb/s, 64 MB Cache, 3.5"







hard-drive memory ffmpeg cpu






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Jan 29 at 17:11









J MJ M

1183




1183













  • Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

    – shawn
    Jan 29 at 20:36











  • @shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

    – J M
    Jan 29 at 20:48













  • Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

    – shawn
    Jan 29 at 21:09











  • @shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

    – J M
    Jan 30 at 0:11








  • 1





    Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

    – shawn
    Jan 30 at 0:53



















  • Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

    – shawn
    Jan 29 at 20:36











  • @shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

    – J M
    Jan 29 at 20:48













  • Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

    – shawn
    Jan 29 at 21:09











  • @shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

    – J M
    Jan 30 at 0:11








  • 1





    Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

    – shawn
    Jan 30 at 0:53

















Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

– shawn
Jan 29 at 20:36





Original file sizes and encoding? I suspect the bottleneck is the USB2 transfer. Consider rewriting your script to precache the file on the local drive, then run ffmpeg on the local copy and write it back out. Or temporarily plug the WD Red directly into a spare computer SATA connection.

– shawn
Jan 29 at 20:36













@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

– J M
Jan 29 at 20:48







@shawn Original file sizes are a whole variety of things anywhere from couple-hundred MB up to maybe 10-15 GB. The local drive (WD Red) is a 1TB drive that has almost nothing on it, so I would have room to precache things. Can you explain (or write as an answer) a bit more about the plugging the local drive into a spare SATA connection?

– J M
Jan 29 at 20:48















Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

– shawn
Jan 29 at 21:09





Remove the drive from its external enclosure and plug it in directly within your computer using a spare SATA cable (such as using the one from your DVD drive). Run your script and see if the performance improves (it will). USB2 has a theoretical performance of 480mbps, but usually gets closer to 40-45% of that and the number is impacted by all other USB operations on the same controller. SATA connections are typically 10-20x faster than USB2.

– shawn
Jan 29 at 21:09













@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

– J M
Jan 30 at 0:11







@shawn I see. Maybe I’m missing something then: the local drive is the Red (not the external). It is directly housed in the desktop computer. The external drive is a raid array of 4 drives, so I don’t think I can take those out and connect individually? Did you still mean the Red, since it is not connected through USB2 correct(?) it is mounted into the computer itself. Thanks for your continued clarifications. However, would it be worth exploring a new enclosure (computer has USB 3.0 ports) instead?

– J M
Jan 30 at 0:11






1




1





Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

– shawn
Jan 30 at 0:53





Sorry, the way I read your sysinfo it looked like you were describing the WD Red as being in the external. eSATA should make a huge difference.

– shawn
Jan 30 at 0:53










0






active

oldest

votes











Your Answer








StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "3"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});

function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});


}
});














draft saved

draft discarded


















StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1399735%2fvideo-conversion-bottleneck-with-ffmpeg%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown

























0






active

oldest

votes








0






active

oldest

votes









active

oldest

votes






active

oldest

votes
















draft saved

draft discarded




















































Thanks for contributing an answer to Super User!


  • Please be sure to answer the question. Provide details and share your research!

But avoid



  • Asking for help, clarification, or responding to other answers.

  • Making statements based on opinion; back them up with references or personal experience.


To learn more, see our tips on writing great answers.




draft saved


draft discarded














StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fsuperuser.com%2fquestions%2f1399735%2fvideo-conversion-bottleneck-with-ffmpeg%23new-answer', 'question_page');
}
);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

Сан-Квентин

Алькесар

Josef Freinademetz