How do all of these “Save video from YouTube” services work?
I mean, how do they work, generally? How do they receive the link to a video stream itself (not just the page containing a Flash player)?
I did a search on the web but couldn't find anything useful (all links point to such services, but none of them tell how they are actually implemented).
youtube
add a comment |
I mean, how do they work, generally? How do they receive the link to a video stream itself (not just the page containing a Flash player)?
I did a search on the web but couldn't find anything useful (all links point to such services, but none of them tell how they are actually implemented).
youtube
add a comment |
I mean, how do they work, generally? How do they receive the link to a video stream itself (not just the page containing a Flash player)?
I did a search on the web but couldn't find anything useful (all links point to such services, but none of them tell how they are actually implemented).
youtube
I mean, how do they work, generally? How do they receive the link to a video stream itself (not just the page containing a Flash player)?
I did a search on the web but couldn't find anything useful (all links point to such services, but none of them tell how they are actually implemented).
youtube
youtube
edited Jul 2 '14 at 10:00
Der Hochstapler
67.7k49230284
67.7k49230284
asked Jun 26 '14 at 12:44
PaulDPaulD
416156
416156
add a comment |
add a comment |
3 Answers
3
active
oldest
votes
There is a very popular open source command-line downloader called youtube-dl
, which does exactly that. It grabs the actual video and audio file links from a given YouTube link – or any other popular web video site like Vimeo, Yahoo! Video, uStream, etc.
To see how that's done, look into the YouTube extractor. That's just too much to show here. Other extractors exist for simpler sites. Steven Penny has a simple JavaScript downloader for YouTube too, which is a little more straightforward.
But basically, for a Flash video player, it must be initialized and configured through some JavaScript. Simply speaking, the Flash object's player will receive a URL of a video stream to load.
In order to find the video stream, you'd have to parse the HTML and JS code of the video page to find the relevant initialization code, and then from there try to find the link to the actual MP4 file. It might be there in plaintext, but it could also be generated on the fly with some specific download tokens. Often, the JavaScript is obfuscated to make it harder to re-engineer it. Or the video information might be contained in an XML file that's loaded asynchronously by JS.
For HTML5 progressive download video, the actual source file is usually mentioned directly in the source
child of the video
tag, so if you'd search the page for mp4
or similar. For example on German news show Tagesschau 100, you'll find:
<source src="http://media.tagesschau.de/video/2014/0626/TV-20140626-1649-5801.webl.h264.mp4" type="video/mp4">
For more advanced playback technologies like MPEG DASH or Apple's HTTP Live Streaming (HLS), you have to parse a meta-information file to get the actual video stream. The meta file (.mpd
for example in DASH, and .m3u8
for HLS) will contain links to segments of video and audio, which you'd later have to combine to get a playable file.
There's no general solution for this. It requires careful inspection and debugging of the target site.
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
|
show 7 more comments
YouTube Bookmarklet
This is how I did it with JavaScript
Start with ytplayer.config.args
object. This contains all URLs for the video.
It is broken up into
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects".
Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL
will be encoded so you will need
to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are
protected with SWF Verification Type 2. This has been around
since 2011
and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You
will typcially see these on VEVO videos and the like. They start with a
signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more
difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of
Same-origin policy. Essentially,
you cannot download this file from www.youtube.com
because they are
different domains. A workaround of this problem is
CORS. With CORS,
s.ytimg.com
could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com
. Of course
they do not do this. A workaround for this workaround is to use a CORS proxy.
This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble
the signature, you can use that in the querystring to download a video.
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
Can you give an example of ans
type video andRTMPE
type video?
– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
add a comment |
My answer:
from 22 January 2019, using these methods can get caught if you try to bypass without linking your user information as well.
Why?
since I'm a new user to this platform, I cannot comment for rule specified by @Daniel-B. According to new ToS (in German as I am in Germany; please translate) for YouTube under $6.1 G$:
You agree any automated system (including – but not limited to – any robot, spider or offline reader) to use that on the website accesses in such a way that more requests to the server within a specified time YouTube directed being able to reasonably produce as a human within the same time period using a publicly available, unmodified standard web browser;
Now they can find out the time duration for each request and can track if you are violating. How is it possible now, given this scenario and your external IP address will be known even if you use a VPN to protect yourself without linking details of user to any service.
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
add a comment |
protected by Ramhound Dec 27 '18 at 22:31
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
There is a very popular open source command-line downloader called youtube-dl
, which does exactly that. It grabs the actual video and audio file links from a given YouTube link – or any other popular web video site like Vimeo, Yahoo! Video, uStream, etc.
To see how that's done, look into the YouTube extractor. That's just too much to show here. Other extractors exist for simpler sites. Steven Penny has a simple JavaScript downloader for YouTube too, which is a little more straightforward.
But basically, for a Flash video player, it must be initialized and configured through some JavaScript. Simply speaking, the Flash object's player will receive a URL of a video stream to load.
In order to find the video stream, you'd have to parse the HTML and JS code of the video page to find the relevant initialization code, and then from there try to find the link to the actual MP4 file. It might be there in plaintext, but it could also be generated on the fly with some specific download tokens. Often, the JavaScript is obfuscated to make it harder to re-engineer it. Or the video information might be contained in an XML file that's loaded asynchronously by JS.
For HTML5 progressive download video, the actual source file is usually mentioned directly in the source
child of the video
tag, so if you'd search the page for mp4
or similar. For example on German news show Tagesschau 100, you'll find:
<source src="http://media.tagesschau.de/video/2014/0626/TV-20140626-1649-5801.webl.h264.mp4" type="video/mp4">
For more advanced playback technologies like MPEG DASH or Apple's HTTP Live Streaming (HLS), you have to parse a meta-information file to get the actual video stream. The meta file (.mpd
for example in DASH, and .m3u8
for HLS) will contain links to segments of video and audio, which you'd later have to combine to get a playable file.
There's no general solution for this. It requires careful inspection and debugging of the target site.
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
|
show 7 more comments
There is a very popular open source command-line downloader called youtube-dl
, which does exactly that. It grabs the actual video and audio file links from a given YouTube link – or any other popular web video site like Vimeo, Yahoo! Video, uStream, etc.
To see how that's done, look into the YouTube extractor. That's just too much to show here. Other extractors exist for simpler sites. Steven Penny has a simple JavaScript downloader for YouTube too, which is a little more straightforward.
But basically, for a Flash video player, it must be initialized and configured through some JavaScript. Simply speaking, the Flash object's player will receive a URL of a video stream to load.
In order to find the video stream, you'd have to parse the HTML and JS code of the video page to find the relevant initialization code, and then from there try to find the link to the actual MP4 file. It might be there in plaintext, but it could also be generated on the fly with some specific download tokens. Often, the JavaScript is obfuscated to make it harder to re-engineer it. Or the video information might be contained in an XML file that's loaded asynchronously by JS.
For HTML5 progressive download video, the actual source file is usually mentioned directly in the source
child of the video
tag, so if you'd search the page for mp4
or similar. For example on German news show Tagesschau 100, you'll find:
<source src="http://media.tagesschau.de/video/2014/0626/TV-20140626-1649-5801.webl.h264.mp4" type="video/mp4">
For more advanced playback technologies like MPEG DASH or Apple's HTTP Live Streaming (HLS), you have to parse a meta-information file to get the actual video stream. The meta file (.mpd
for example in DASH, and .m3u8
for HLS) will contain links to segments of video and audio, which you'd later have to combine to get a playable file.
There's no general solution for this. It requires careful inspection and debugging of the target site.
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
|
show 7 more comments
There is a very popular open source command-line downloader called youtube-dl
, which does exactly that. It grabs the actual video and audio file links from a given YouTube link – or any other popular web video site like Vimeo, Yahoo! Video, uStream, etc.
To see how that's done, look into the YouTube extractor. That's just too much to show here. Other extractors exist for simpler sites. Steven Penny has a simple JavaScript downloader for YouTube too, which is a little more straightforward.
But basically, for a Flash video player, it must be initialized and configured through some JavaScript. Simply speaking, the Flash object's player will receive a URL of a video stream to load.
In order to find the video stream, you'd have to parse the HTML and JS code of the video page to find the relevant initialization code, and then from there try to find the link to the actual MP4 file. It might be there in plaintext, but it could also be generated on the fly with some specific download tokens. Often, the JavaScript is obfuscated to make it harder to re-engineer it. Or the video information might be contained in an XML file that's loaded asynchronously by JS.
For HTML5 progressive download video, the actual source file is usually mentioned directly in the source
child of the video
tag, so if you'd search the page for mp4
or similar. For example on German news show Tagesschau 100, you'll find:
<source src="http://media.tagesschau.de/video/2014/0626/TV-20140626-1649-5801.webl.h264.mp4" type="video/mp4">
For more advanced playback technologies like MPEG DASH or Apple's HTTP Live Streaming (HLS), you have to parse a meta-information file to get the actual video stream. The meta file (.mpd
for example in DASH, and .m3u8
for HLS) will contain links to segments of video and audio, which you'd later have to combine to get a playable file.
There's no general solution for this. It requires careful inspection and debugging of the target site.
There is a very popular open source command-line downloader called youtube-dl
, which does exactly that. It grabs the actual video and audio file links from a given YouTube link – or any other popular web video site like Vimeo, Yahoo! Video, uStream, etc.
To see how that's done, look into the YouTube extractor. That's just too much to show here. Other extractors exist for simpler sites. Steven Penny has a simple JavaScript downloader for YouTube too, which is a little more straightforward.
But basically, for a Flash video player, it must be initialized and configured through some JavaScript. Simply speaking, the Flash object's player will receive a URL of a video stream to load.
In order to find the video stream, you'd have to parse the HTML and JS code of the video page to find the relevant initialization code, and then from there try to find the link to the actual MP4 file. It might be there in plaintext, but it could also be generated on the fly with some specific download tokens. Often, the JavaScript is obfuscated to make it harder to re-engineer it. Or the video information might be contained in an XML file that's loaded asynchronously by JS.
For HTML5 progressive download video, the actual source file is usually mentioned directly in the source
child of the video
tag, so if you'd search the page for mp4
or similar. For example on German news show Tagesschau 100, you'll find:
<source src="http://media.tagesschau.de/video/2014/0626/TV-20140626-1649-5801.webl.h264.mp4" type="video/mp4">
For more advanced playback technologies like MPEG DASH or Apple's HTTP Live Streaming (HLS), you have to parse a meta-information file to get the actual video stream. The meta file (.mpd
for example in DASH, and .m3u8
for HLS) will contain links to segments of video and audio, which you'd later have to combine to get a playable file.
There's no general solution for this. It requires careful inspection and debugging of the target site.
edited Jun 26 '14 at 16:47
answered Jun 26 '14 at 12:57
slhckslhck
160k47445467
160k47445467
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
|
show 7 more comments
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
3
3
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
One question, what is Youtube/Google's policy on this? Are they ok with this, or not so much?
– JMK
Jun 26 '14 at 15:53
31
31
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
The YouTube Terms of Service in §5.1.L disallow consumption of their content through any other means than streaming, so theoretically it's not allowed. In practice, they won't be able to enforce that though. Any downloader can more or less simulate that it's just streaming.
– slhck
Jun 26 '14 at 16:23
2
2
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
@StevenPenny do you have any non minified version of that?
– TankorSmash
Jun 26 '14 at 18:08
5
5
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
@slhck Flash could also make HTTP requests by itself. Instead, it uses the browser’s HTTP engine. If Flash itself made the requests, they wouldn’t be “visible” to the browser. Sure would be great for advertisers. ;)
– Daniel B
Jun 26 '14 at 19:49
3
3
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
@slhck they can't enforce it programmatically, but if they got their lawyer team out could they enforce it legally?
– Cruncher
Jun 27 '14 at 15:45
|
show 7 more comments
YouTube Bookmarklet
This is how I did it with JavaScript
Start with ytplayer.config.args
object. This contains all URLs for the video.
It is broken up into
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects".
Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL
will be encoded so you will need
to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are
protected with SWF Verification Type 2. This has been around
since 2011
and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You
will typcially see these on VEVO videos and the like. They start with a
signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more
difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of
Same-origin policy. Essentially,
you cannot download this file from www.youtube.com
because they are
different domains. A workaround of this problem is
CORS. With CORS,
s.ytimg.com
could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com
. Of course
they do not do this. A workaround for this workaround is to use a CORS proxy.
This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble
the signature, you can use that in the querystring to download a video.
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
Can you give an example of ans
type video andRTMPE
type video?
– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
add a comment |
YouTube Bookmarklet
This is how I did it with JavaScript
Start with ytplayer.config.args
object. This contains all URLs for the video.
It is broken up into
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects".
Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL
will be encoded so you will need
to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are
protected with SWF Verification Type 2. This has been around
since 2011
and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You
will typcially see these on VEVO videos and the like. They start with a
signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more
difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of
Same-origin policy. Essentially,
you cannot download this file from www.youtube.com
because they are
different domains. A workaround of this problem is
CORS. With CORS,
s.ytimg.com
could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com
. Of course
they do not do this. A workaround for this workaround is to use a CORS proxy.
This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble
the signature, you can use that in the querystring to download a video.
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
Can you give an example of ans
type video andRTMPE
type video?
– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
add a comment |
YouTube Bookmarklet
This is how I did it with JavaScript
Start with ytplayer.config.args
object. This contains all URLs for the video.
It is broken up into
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects".
Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL
will be encoded so you will need
to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are
protected with SWF Verification Type 2. This has been around
since 2011
and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You
will typcially see these on VEVO videos and the like. They start with a
signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more
difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of
Same-origin policy. Essentially,
you cannot download this file from www.youtube.com
because they are
different domains. A workaround of this problem is
CORS. With CORS,
s.ytimg.com
could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com
. Of course
they do not do this. A workaround for this workaround is to use a CORS proxy.
This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble
the signature, you can use that in the querystring to download a video.
YouTube Bookmarklet
This is how I did it with JavaScript
Start with ytplayer.config.args
object. This contains all URLs for the video.
It is broken up into
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects".
Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL
will be encoded so you will need
to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are
protected with SWF Verification Type 2. This has been around
since 2011
and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You
will typcially see these on VEVO videos and the like. They start with a
signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more
difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of
Same-origin policy. Essentially,
you cannot download this file from www.youtube.com
because they are
different domains. A workaround of this problem is
CORS. With CORS,
s.ytimg.com
could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com
. Of course
they do not do this. A workaround for this workaround is to use a CORS proxy.
This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble
the signature, you can use that in the querystring to download a video.
edited Jul 19 '18 at 15:55
answered Jun 27 '14 at 5:09
Steven PennySteven Penny
1
1
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
Can you give an example of ans
type video andRTMPE
type video?
– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
add a comment |
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
Can you give an example of ans
type video andRTMPE
type video?
– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
1
1
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
Do you happen to know (for DASH playback) how the YouTube player determines the byte ranges to be requested from the media representation? The MPD file only lists segments.
– slhck
Jul 9 '14 at 12:51
2
2
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
Not exactly.. There is a single file for video and audio, but YouTube requests them by byte ranges, in several chunks. If you switch to another quality, it'll change the byte range too. Just wondering how the player knows which second corresponds to which byte offset.
– slhck
Jul 9 '14 at 16:25
1
1
Can you give an example of an
s
type video and RTMPE
type video?– Chloe
May 3 '16 at 5:10
Can you give an example of an
s
type video and RTMPE
type video?– Chloe
May 3 '16 at 5:10
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
@SurajJain here is new page - I will be rewriting this answer as I use a different method now github.com/svnpenn/umber/blob/master/bmklet/youtube/download.js
– Steven Penny
Feb 19 '18 at 12:39
add a comment |
My answer:
from 22 January 2019, using these methods can get caught if you try to bypass without linking your user information as well.
Why?
since I'm a new user to this platform, I cannot comment for rule specified by @Daniel-B. According to new ToS (in German as I am in Germany; please translate) for YouTube under $6.1 G$:
You agree any automated system (including – but not limited to – any robot, spider or offline reader) to use that on the website accesses in such a way that more requests to the server within a specified time YouTube directed being able to reasonably produce as a human within the same time period using a publicly available, unmodified standard web browser;
Now they can find out the time duration for each request and can track if you are violating. How is it possible now, given this scenario and your external IP address will be known even if you use a VPN to protect yourself without linking details of user to any service.
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
add a comment |
My answer:
from 22 January 2019, using these methods can get caught if you try to bypass without linking your user information as well.
Why?
since I'm a new user to this platform, I cannot comment for rule specified by @Daniel-B. According to new ToS (in German as I am in Germany; please translate) for YouTube under $6.1 G$:
You agree any automated system (including – but not limited to – any robot, spider or offline reader) to use that on the website accesses in such a way that more requests to the server within a specified time YouTube directed being able to reasonably produce as a human within the same time period using a publicly available, unmodified standard web browser;
Now they can find out the time duration for each request and can track if you are violating. How is it possible now, given this scenario and your external IP address will be known even if you use a VPN to protect yourself without linking details of user to any service.
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
add a comment |
My answer:
from 22 January 2019, using these methods can get caught if you try to bypass without linking your user information as well.
Why?
since I'm a new user to this platform, I cannot comment for rule specified by @Daniel-B. According to new ToS (in German as I am in Germany; please translate) for YouTube under $6.1 G$:
You agree any automated system (including – but not limited to – any robot, spider or offline reader) to use that on the website accesses in such a way that more requests to the server within a specified time YouTube directed being able to reasonably produce as a human within the same time period using a publicly available, unmodified standard web browser;
Now they can find out the time duration for each request and can track if you are violating. How is it possible now, given this scenario and your external IP address will be known even if you use a VPN to protect yourself without linking details of user to any service.
My answer:
from 22 January 2019, using these methods can get caught if you try to bypass without linking your user information as well.
Why?
since I'm a new user to this platform, I cannot comment for rule specified by @Daniel-B. According to new ToS (in German as I am in Germany; please translate) for YouTube under $6.1 G$:
You agree any automated system (including – but not limited to – any robot, spider or offline reader) to use that on the website accesses in such a way that more requests to the server within a specified time YouTube directed being able to reasonably produce as a human within the same time period using a publicly available, unmodified standard web browser;
Now they can find out the time duration for each request and can track if you are violating. How is it possible now, given this scenario and your external IP address will be known even if you use a VPN to protect yourself without linking details of user to any service.
edited Dec 27 '18 at 22:52
Scott
15.7k113990
15.7k113990
answered Dec 27 '18 at 22:06
user103720user103720
91
91
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
add a comment |
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
1
1
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
Welcome traveler from the future...It isn’t clear how this answers the author’s question
– Ramhound
Dec 27 '18 at 22:31
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
If you wanted to warn the author, with regards to a specific answer, you should have submitted a comment and downvoted any answer you didn't think was helpful due to the possible legal issues that could follow. This answer reads more like what you would find on a discussion forum post, and Super User, isn't a discussion forum
– Ramhound
Jan 13 at 0:34
add a comment |
protected by Ramhound Dec 27 '18 at 22:31
Thank you for your interest in this question.
Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).
Would you like to answer one of these unanswered questions instead?