I have created the method in class named A as shown below which functionality is to send the mails using java mail api , now the bellow method accepts the parameters now my query is that how should i add recipients in an array as the recipients must be of array type that is passes please advise
class A
{
public boolean postMail(String recipients[], String subject, String message, String from) throws Exception {
boolean debug = true;
boolean result = false;
// create a message
try {
// set the from and to address
InternetAddress addressFrom = new InternetAddress(from);
this.mimeMessage.setFrom(addressFrom);
InternetAddress[] addressTo = new InternetAddress[recipients.length];
for (int index = 0; index < recipients.length; index++) {
addressTo[index] = new InternetAddress(recipients[index]);
}
this.mimeMessage.setRecipients(Message.RecipientType.BCC, addressTo);
this.mimeMessage.addHeader("MyHeaderName", "myHeaderValue");
// Setting the Subject and Content Type
this.mimeMessage.setSubject(subject);
this.mimeMessage.setContent(message, "text/html");
Transport.send(this.mimeMessage);
result = true;
} catch (MessagingException messagingException) {
result = false;
messagingException.printStackTrace();
// throw new CASSException(messagingException);
}
return result;
}
}
now my query is that i am trying to invoke this method from within the class shown below '
class A
{
String Subject = "testmailsubject";
String message = "simple test message";
String from ="test#abc.com";
A a1 = new A ();
a1.postMail
//***********method *************
public boolean postMail(String recipients[], String subject, String message, String from) throws Exception {
boolean debug = true;
boolean result = false;
// create a message
try {
// set the from and to address
InternetAddress addressFrom = new InternetAddress(from);
this.mimeMessage.setFrom(addressFrom);
InternetAddress[] addressTo = new InternetAddress[recipients.length];
for (int index = 0; index < recipients.length; index++) {
addressTo[index] = new InternetAddress(recipients[index]);
}
this.mimeMessage.setRecipients(Message.RecipientType.BCC, addressTo);
this.mimeMessage.addHeader("MyHeaderName", "myHeaderValue");
// Setting the Subject and Content Type
this.mimeMessage.setSubject(subject);
this.mimeMessage.setContent(message, "text/html");
Transport.send(this.mimeMessage);
result = true;
} catch (MessagingException messagingException) {
result = false;
messagingException.printStackTrace();
// throw new CASSException(messagingException);
}
return result;
}
}
Add Receipeints like this
Address[] TO = new Address[] {InternetAddress.parse("karthik#cherukuri.com"),
InternetAddress.parse("karthik2#cherukuri.com"),
InternetAddress.parse("karthik3#cherukuri.com")};
message.addRecipients(Message.RecipientType.TO, TO);
else if you want to send recipient array directly to method just initialize it email list you wanted.
String Subject = "testmailsubject";
String message = "simple test message";
String from ="test#abc.com";
String[] recipient = new String[]{"karthik#cherukuri.com","karthik2#cherukuri.com", "karthik3#cherukuri.com"};
a1.postmail(recipient,subject,from,message);
I am getting the following issues after scanning the code using fortify....
1>path manipulation issue:
private MimeMessage prepareMessage(EmailMessage req) throws EmailProviderException {
long start=System.currentTimeMillis(),finish=0;
try {
MimeMessage message = emailSender.createMimeMessage();
// Create a multipart message
MimeMessageHelper helper = new MimeMessageHelper(message, true);
// Set email addresses
helper.setFrom(convertAddress(req.getFromAddress()));
helper.setTo(convertAddress(req.getToAddress()));
helper.setCc(convertAddress(req.getCcAddress()));
helper.setBcc(convertAddress(req.getBccAddress()));
// Set subject and body
helper.setSubject(req.getEmailSubject());
String emailBody = req.getEmailBody();
String emailMime = req.getEmailMimeType();
MimeBodyPart messagePart = new MimeBodyPart();
DataSource bodyDataSource = new ByteArrayDataSource(emailBody, emailMime);
messagePart.setDataHandler(new DataHandler(bodyDataSource));
helper.getMimeMultipart().addBodyPart(messagePart);
// Add attachments
List<EmailAttachment> lAttach = req.getEmailAttachment();
if (lAttach != null) {
for (EmailAttachment attachMnt: lAttach) {
DataSource dSource = new ByteArrayDataSource(attachMnt
.getContent(), attachMnt
.getMimeType());
helper.addAttachment(attachMnt.getFileName(), dSource);
}
}
finish=System.currentTimeMillis();
statsLogger.info(new FedExLogEntry("prepareMessage took {0}ms",new Object[]{finish-start}));
return message;
} catch (Exception e) {
// Covers MessagingException, IllegalStateException, IOException, MailException
String emsg = new StringBuilder("Unable to prepare smtp message.")
.append("\n").append(req.toString()).toString();
logger.warn(emsg, e);
throw new EmailProviderException(emsg, e);
}
}
Null dereference issues
issue 1.
public byte[] toByteArray(Notification nd) throws EmailProviderException {
String message = null;
try {
JAXBContext jc = JAXBContext.newInstance(nd.getClass());
if (jc != null) {
Marshaller m = jc.createMarshaller();
StringWriter sw = new StringWriter();
m.marshal(nd, sw);
message = sw.toString();
}
} catch (JAXBException e) {
throw new EmailProviderException("Unable to convert NDS notification to byte array.", e);
}
return message.getBytes();
}
null dereference issue 2..
private void addLastUpdatedHours(
List<LocationHoursForADate> masterHours, List<LocationHoursWithSearchedDate> hoursToAdd, Map<String,String> scheduleTypeIncludesMap){
String prevScheduleTypeCode = null;
String prevHourTypeCode = null;
Date searchedDate = null;
// Build map of locationHours to searchDates
List<LocationHours> locationHours = null;
Map<Date, List<LocationHours>> locationHoursSearchDatesMap = new HashMap<Date, List<LocationHours>>();
for(LocationHoursWithSearchedDate locationHoursWithSearchedDate : hoursToAdd) {
if(scheduleTypeIncludesMap.containsKey(locationHoursWithSearchedDate.getLocationHoursPK().getScheduleTypeCd())) {
searchedDate = locationHoursWithSearchedDate.getLocationHoursPK().getSearchedDate();
locationHours = locationHoursSearchDatesMap.get(searchedDate);
if(locationHours==null) {
locationHours = new ArrayList<LocationHours>();
locationHoursSearchDatesMap.put(searchedDate,locationHours);
}
locationHours.add(locationHoursWithSearchedDate.createLocationHours());
}
}
for(Map.Entry<Date,List<LocationHours>> entry : locationHoursSearchDatesMap.entrySet()) {
prevHourTypeCode = null;
prevScheduleTypeCode = null;
searchedDate = entry.getKey();
for(LocationHours hour: entry.getValue()){
// new ST & new 01, add it
if((prevScheduleTypeCode == null) && (prevHourTypeCode == null)){
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
else{
//same ST
if(prevScheduleTypeCode.equals(hour.getLocationHoursPK().getScheduleTypeCd())){
// same 01, skip this schedule
if(prevHourTypeCode.equals(hour.getHoursType().getHoursTypeCd())){
continue;
}
else { //new 01, add it
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
}
else{ //new ST, add it
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
}
}
}
}
}
Well, the second case (null deference 1) is easy to spot.
The problem is that if there is an exception in the try-catch block then the object message will be null, so the final instruction return message.getBytes() will raise a NullPointerException.
One way to avoid it is changing the return statement like this:
return message != null ? message.getBytes() : null;
But maybe, you will prefer to raise the exception...
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
how to download videos from youtube on Java?
need class(or piece of code) which describes how to do that.
Thank you.
import java.io.BufferedReader;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.Reader;
import java.io.StringWriter;
import java.io.UnsupportedEncodingException;
import java.io.Writer;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.ArrayList;
import java.util.List;
import java.util.Scanner;
import java.util.logging.Formatter;
import java.util.logging.Handler;
import java.util.logging.Level;
import java.util.logging.LogRecord;
import java.util.logging.Logger;
import java.util.regex.Pattern;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.CookieStore;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.ClientContext;
import org.apache.http.client.utils.URIUtils;
import org.apache.http.client.utils.URLEncodedUtils;
import org.apache.http.impl.client.BasicCookieStore;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
import org.apache.http.protocol.BasicHttpContext;
import org.apache.http.protocol.HttpContext;
public class JavaYoutubeDownloader {
public static String newline = System.getProperty("line.separator");
private static final Logger log = Logger.getLogger(JavaYoutubeDownloader.class.getCanonicalName());
private static final Level defaultLogLevelSelf = Level.FINER;
private static final Level defaultLogLevel = Level.WARNING;
private static final Logger rootlog = Logger.getLogger("");
private static final String scheme = "http";
private static final String host = "www.youtube.com";
private static final Pattern commaPattern = Pattern.compile(",");
private static final Pattern pipePattern = Pattern.compile("\\|");
private static final char[] ILLEGAL_FILENAME_CHARACTERS = { '/', '\n', '\r', '\t', '\0', '\f', '`', '?', '*', '\\', '<', '>', '|', '\"', ':' };
private static void usage(String error) {
if (error != null) {
System.err.println("Error: " + error);
}
System.err.println("usage: JavaYoutubeDownload VIDEO_ID DESTINATION_DIRECTORY");
System.exit(-1);
}
public static void main(String[] args) {
if (args == null || args.length == 0) {
usage("Missing video id. Extract from http://www.youtube.com/watch?v=VIDEO_ID");
}
try {
setupLogging();
log.fine("Starting");
String videoId = null;
String outdir = ".";
// TODO Ghetto command line parsing
if (args.length == 1) {
videoId = args[0];
} else if (args.length == 2) {
videoId = args[0];
outdir = args[1];
}
int format = 18; // http://en.wikipedia.org/wiki/YouTube#Quality_and_codecs
String encoding = "UTF-8";
String userAgent = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13";
File outputDir = new File(outdir);
String extension = getExtension(format);
play(videoId, format, encoding, userAgent, outputDir, extension);
} catch (Throwable t) {
t.printStackTrace();
}
log.fine("Finished");
}
private static String getExtension(int format) {
// TODO
return "mp4";
}
private static void play(String videoId, int format, String encoding, String userAgent, File outputdir, String extension) throws Throwable {
log.fine("Retrieving " + videoId);
List<NameValuePair> qparams = new ArrayList<NameValuePair>();
qparams.add(new BasicNameValuePair("video_id", videoId));
qparams.add(new BasicNameValuePair("fmt", "" + format));
URI uri = getUri("get_video_info", qparams);
CookieStore cookieStore = new BasicCookieStore();
HttpContext localContext = new BasicHttpContext();
localContext.setAttribute(ClientContext.COOKIE_STORE, cookieStore);
HttpClient httpclient = new DefaultHttpClient();
HttpGet httpget = new HttpGet(uri);
httpget.setHeader("User-Agent", userAgent);
log.finer("Executing " + uri);
HttpResponse response = httpclient.execute(httpget, localContext);
HttpEntity entity = response.getEntity();
if (entity != null && response.getStatusLine().getStatusCode() == 200) {
InputStream instream = entity.getContent();
String videoInfo = getStringFromInputStream(encoding, instream);
if (videoInfo != null && videoInfo.length() > 0) {
List<NameValuePair> infoMap = new ArrayList<NameValuePair>();
URLEncodedUtils.parse(infoMap, new Scanner(videoInfo), encoding);
String token = null;
String downloadUrl = null;
String filename = videoId;
for (NameValuePair pair : infoMap) {
String key = pair.getName();
String val = pair.getValue();
log.finest(key + "=" + val);
if (key.equals("token")) {
token = val;
} else if (key.equals("title")) {
filename = val;
} else if (key.equals("fmt_url_map")) {
String[] formats = commaPattern.split(val);
for (String fmt : formats) {
String[] fmtPieces = pipePattern.split(fmt);
if (fmtPieces.length == 2) {
// in the end, download somethin!
downloadUrl = fmtPieces[1];
int pieceFormat = Integer.parseInt(fmtPieces[0]);
if (pieceFormat == format) {
// found what we want
downloadUrl = fmtPieces[1];
break;
}
}
}
}
}
filename = cleanFilename(filename);
if (filename.length() == 0) {
filename = videoId;
} else {
filename += "_" + videoId;
}
filename += "." + extension;
File outputfile = new File(outputdir, filename);
if (downloadUrl != null) {
downloadWithHttpClient(userAgent, downloadUrl, outputfile);
}
}
}
}
private static void downloadWithHttpClient(String userAgent, String downloadUrl, File outputfile) throws Throwable {
HttpGet httpget2 = new HttpGet(downloadUrl);
httpget2.setHeader("User-Agent", userAgent);
log.finer("Executing " + httpget2.getURI());
HttpClient httpclient2 = new DefaultHttpClient();
HttpResponse response2 = httpclient2.execute(httpget2);
HttpEntity entity2 = response2.getEntity();
if (entity2 != null && response2.getStatusLine().getStatusCode() == 200) {
long length = entity2.getContentLength();
InputStream instream2 = entity2.getContent();
log.finer("Writing " + length + " bytes to " + outputfile);
if (outputfile.exists()) {
outputfile.delete();
}
FileOutputStream outstream = new FileOutputStream(outputfile);
try {
byte[] buffer = new byte[2048];
int count = -1;
while ((count = instream2.read(buffer)) != -1) {
outstream.write(buffer, 0, count);
}
outstream.flush();
} finally {
outstream.close();
}
}
}
private static String cleanFilename(String filename) {
for (char c : ILLEGAL_FILENAME_CHARACTERS) {
filename = filename.replace(c, '_');
}
return filename;
}
private static URI getUri(String path, List<NameValuePair> qparams) throws URISyntaxException {
URI uri = URIUtils.createURI(scheme, host, -1, "/" + path, URLEncodedUtils.format(qparams, "UTF-8"), null);
return uri;
}
private static void setupLogging() {
changeFormatter(new Formatter() {
#Override
public String format(LogRecord arg0) {
return arg0.getMessage() + newline;
}
});
explicitlySetAllLogging(Level.FINER);
}
private static void changeFormatter(Formatter formatter) {
Handler[] handlers = rootlog.getHandlers();
for (Handler handler : handlers) {
handler.setFormatter(formatter);
}
}
private static void explicitlySetAllLogging(Level level) {
rootlog.setLevel(Level.ALL);
for (Handler handler : rootlog.getHandlers()) {
handler.setLevel(defaultLogLevelSelf);
}
log.setLevel(level);
rootlog.setLevel(defaultLogLevel);
}
private static String getStringFromInputStream(String encoding, InputStream instream) throws UnsupportedEncodingException, IOException {
Writer writer = new StringWriter();
char[] buffer = new char[1024];
try {
Reader reader = new BufferedReader(new InputStreamReader(instream, encoding));
int n;
while ((n = reader.read(buffer)) != -1) {
writer.write(buffer, 0, n);
}
} finally {
instream.close();
}
String result = writer.toString();
return result;
}
}
/**
* <pre>
* Exploded results from get_video_info:
*
* fexp=90...
* allow_embed=1
* fmt_stream_map=35|http://v9.lscache8...
* fmt_url_map=35|http://v9.lscache8...
* allow_ratings=1
* keywords=Stefan Molyneux,Luke Bessey,anarchy,stateless society,giant stone cow,the story of our unenslavement,market anarchy,voluntaryism,anarcho capitalism
* track_embed=0
* fmt_list=35/854x480/9/0/115,34/640x360/9/0/115,18/640x360/9/0/115,5/320x240/7/0/0
* author=lukebessey
* muted=0
* length_seconds=390
* plid=AA...
* ftoken=null
* status=ok
* watermark=http://s.ytimg.com/yt/swf/logo-vfl_bP6ud.swf,http://s.ytimg.com/yt/swf/hdlogo-vfloR6wva.swf
* timestamp=12...
* has_cc=False
* fmt_map=35/854x480/9/0/115,34/640x360/9/0/115,18/640x360/9/0/115,5/320x240/7/0/0
* leanback_module=http://s.ytimg.com/yt/swfbin/leanback_module-vflJYyeZN.swf
* hl=en_US
* endscreen_module=http://s.ytimg.com/yt/swfbin/endscreen-vflk19iTq.swf
* vq=auto
* avg_rating=5.0
* video_id=S6IZP3yRJ9I
* token=vPpcFNh...
* thumbnail_url=http://i4.ytimg.com/vi/S6IZP3yRJ9I/default.jpg
* title=The Story of Our Unenslavement - Animated
* </pre>
*/
I know i am answering late. But this code may useful for some one. So i am attaching it here.
Use the following java code to download the videos from YouTube.
package com.mycompany.ytd;
import java.io.File;
import java.net.URL;
import com.github.axet.vget.VGet;
/**
*
* #author Manindar
*/
public class YTD {
public static void main(String[] args) {
try {
String url = "https://www.youtube.com/watch?v=s10ARdfQUOY";
String path = "D:\\Manindar\\YTD\\";
VGet v = new VGet(new URL(url), new File(path));
v.download();
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
Add the below Dependency in your POM.XML file
<dependency>
<groupId>com.github.axet</groupId>
<artifactId>vget</artifactId>
<version>1.1.33</version>
</dependency>
Hope this will be useful.
Regarding the format (mp4 or flv) decide which URL you want to use. Then use this tutorial to download the video and save it into a local directory.
Ref :Youtube Video Download (Android/Java)
Edit 3
You can use the Lib : https://github.com/HaarigerHarald/android-youtubeExtractor
Ex :
String youtubeLink = "http://youtube.com/watch?v=xxxx";
new YouTubeExtractor(this) {
#Override
public void onExtractionComplete(SparseArray<YtFile> ytFiles, VideoMeta vMeta) {
if (ytFiles != null) {
int itag = 22;
String downloadUrl = ytFiles.get(itag).getUrl();
}
}
}.extract(youtubeLink, true, true);
They decipherSignature using :
private boolean decipherSignature(final SparseArray<String> encSignatures) throws IOException {
// Assume the functions don't change that much
if (decipherFunctionName == null || decipherFunctions == null) {
String decipherFunctUrl = "https://s.ytimg.com/yts/jsbin/" + decipherJsFileName;
BufferedReader reader = null;
String javascriptFile;
URL url = new URL(decipherFunctUrl);
HttpURLConnection urlConnection = (HttpURLConnection) url.openConnection();
urlConnection.setRequestProperty("User-Agent", USER_AGENT);
try {
reader = new BufferedReader(new InputStreamReader(urlConnection.getInputStream()));
StringBuilder sb = new StringBuilder("");
String line;
while ((line = reader.readLine()) != null) {
sb.append(line);
sb.append(" ");
}
javascriptFile = sb.toString();
} finally {
if (reader != null)
reader.close();
urlConnection.disconnect();
}
if (LOGGING)
Log.d(LOG_TAG, "Decipher FunctURL: " + decipherFunctUrl);
Matcher mat = patSignatureDecFunction.matcher(javascriptFile);
if (mat.find()) {
decipherFunctionName = mat.group(1);
if (LOGGING)
Log.d(LOG_TAG, "Decipher Functname: " + decipherFunctionName);
Pattern patMainVariable = Pattern.compile("(var |\\s|,|;)" + decipherFunctionName.replace("$", "\\$") +
"(=function\\((.{1,3})\\)\\{)");
String mainDecipherFunct;
mat = patMainVariable.matcher(javascriptFile);
if (mat.find()) {
mainDecipherFunct = "var " + decipherFunctionName + mat.group(2);
} else {
Pattern patMainFunction = Pattern.compile("function " + decipherFunctionName.replace("$", "\\$") +
"(\\((.{1,3})\\)\\{)");
mat = patMainFunction.matcher(javascriptFile);
if (!mat.find())
return false;
mainDecipherFunct = "function " + decipherFunctionName + mat.group(2);
}
int startIndex = mat.end();
for (int braces = 1, i = startIndex; i < javascriptFile.length(); i++) {
if (braces == 0 && startIndex + 5 < i) {
mainDecipherFunct += javascriptFile.substring(startIndex, i) + ";";
break;
}
if (javascriptFile.charAt(i) == '{')
braces++;
else if (javascriptFile.charAt(i) == '}')
braces--;
}
decipherFunctions = mainDecipherFunct;
// Search the main function for extra functions and variables
// needed for deciphering
// Search for variables
mat = patVariableFunction.matcher(mainDecipherFunct);
while (mat.find()) {
String variableDef = "var " + mat.group(2) + "={";
if (decipherFunctions.contains(variableDef)) {
continue;
}
startIndex = javascriptFile.indexOf(variableDef) + variableDef.length();
for (int braces = 1, i = startIndex; i < javascriptFile.length(); i++) {
if (braces == 0) {
decipherFunctions += variableDef + javascriptFile.substring(startIndex, i) + ";";
break;
}
if (javascriptFile.charAt(i) == '{')
braces++;
else if (javascriptFile.charAt(i) == '}')
braces--;
}
}
// Search for functions
mat = patFunction.matcher(mainDecipherFunct);
while (mat.find()) {
String functionDef = "function " + mat.group(2) + "(";
if (decipherFunctions.contains(functionDef)) {
continue;
}
startIndex = javascriptFile.indexOf(functionDef) + functionDef.length();
for (int braces = 0, i = startIndex; i < javascriptFile.length(); i++) {
if (braces == 0 && startIndex + 5 < i) {
decipherFunctions += functionDef + javascriptFile.substring(startIndex, i) + ";";
break;
}
if (javascriptFile.charAt(i) == '{')
braces++;
else if (javascriptFile.charAt(i) == '}')
braces--;
}
}
if (LOGGING)
Log.d(LOG_TAG, "Decipher Function: " + decipherFunctions);
decipherViaWebView(encSignatures);
if (CACHING) {
writeDeciperFunctToChache();
}
} else {
return false;
}
} else {
decipherViaWebView(encSignatures);
}
return true;
}
Now with use of this library High Quality Videos Lossing Audio so i use the MediaMuxer for Murging Audio and Video for Final Output
Edit 1
https://stackoverflow.com/a/15240012/9909365
Why the previous answer not worked
Pattern p2 = Pattern.compile("sig=(.*?)[&]");
Matcher m2 = p2.matcher(url);
String sig = null;
if (m2.find()) {
sig = m2.group(1);
}
As of November 2016, this is a little rough around the edges, but
displays the basic principle. The url_encoded_fmt_stream_map today
does not have a space after the colon (better make this optional) and
"sig" has been changed to "signature"
and while i am debuging the code i found the new keyword its
signature&s in many video's URL
here edited answer
private static final HashMap<String, Meta> typeMap = new HashMap<String, Meta>();
initTypeMap(); call first
class Meta {
public String num;
public String type;
public String ext;
Meta(String num, String ext, String type) {
this.num = num;
this.ext = ext;
this.type = type;
}
}
class Video {
public String ext = "";
public String type = "";
public String url = "";
Video(String ext, String type, String url) {
this.ext = ext;
this.type = type;
this.url = url;
}
}
public ArrayList<Video> getStreamingUrisFromYouTubePage(String ytUrl)
throws IOException {
if (ytUrl == null) {
return null;
}
// Remove any query params in query string after the watch?v=<vid> in
// e.g.
// http://www.youtube.com/watch?v=0RUPACpf8Vs&feature=youtube_gdata_player
int andIdx = ytUrl.indexOf('&');
if (andIdx >= 0) {
ytUrl = ytUrl.substring(0, andIdx);
}
// Get the HTML response
/* String userAgent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:8.0.1)";*/
/* HttpClient client = new DefaultHttpClient();
client.getParams().setParameter(CoreProtocolPNames.USER_AGENT,
userAgent);
HttpGet request = new HttpGet(ytUrl);
HttpResponse response = client.execute(request);*/
String html = "";
HttpsURLConnection c = (HttpsURLConnection) new URL(ytUrl).openConnection();
c.setRequestMethod("GET");
c.setDoOutput(true);
c.connect();
InputStream in = c.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in));
StringBuilder str = new StringBuilder();
String line = null;
while ((line = reader.readLine()) != null) {
str.append(line.replace("\\u0026", "&"));
}
in.close();
html = str.toString();
// Parse the HTML response and extract the streaming URIs
if (html.contains("verify-age-thumb")) {
Log.e("Downloader", "YouTube is asking for age verification. We can't handle that sorry.");
return null;
}
if (html.contains("das_captcha")) {
Log.e("Downloader", "Captcha found, please try with different IP address.");
return null;
}
Pattern p = Pattern.compile("stream_map\":\"(.*?)?\"");
// Pattern p = Pattern.compile("/stream_map=(.[^&]*?)\"/");
Matcher m = p.matcher(html);
List<String> matches = new ArrayList<String>();
while (m.find()) {
matches.add(m.group());
}
if (matches.size() != 1) {
Log.e("Downloader", "Found zero or too many stream maps.");
return null;
}
String urls[] = matches.get(0).split(",");
HashMap<String, String> foundArray = new HashMap<String, String>();
for (String ppUrl : urls) {
String url = URLDecoder.decode(ppUrl, "UTF-8");
Log.e("URL","URL : "+url);
Pattern p1 = Pattern.compile("itag=([0-9]+?)[&]");
Matcher m1 = p1.matcher(url);
String itag = null;
if (m1.find()) {
itag = m1.group(1);
}
Pattern p2 = Pattern.compile("signature=(.*?)[&]");
Matcher m2 = p2.matcher(url);
String sig = null;
if (m2.find()) {
sig = m2.group(1);
} else {
Pattern p23 = Pattern.compile("signature&s=(.*?)[&]");
Matcher m23 = p23.matcher(url);
if (m23.find()) {
sig = m23.group(1);
}
}
Pattern p3 = Pattern.compile("url=(.*?)[&]");
Matcher m3 = p3.matcher(ppUrl);
String um = null;
if (m3.find()) {
um = m3.group(1);
}
if (itag != null && sig != null && um != null) {
Log.e("foundArray","Adding Value");
foundArray.put(itag, URLDecoder.decode(um, "UTF-8") + "&"
+ "signature=" + sig);
}
}
Log.e("foundArray","Size : "+foundArray.size());
if (foundArray.size() == 0) {
Log.e("Downloader", "Couldn't find any URLs and corresponding signatures");
return null;
}
ArrayList<Video> videos = new ArrayList<Video>();
for (String format : typeMap.keySet()) {
Meta meta = typeMap.get(format);
if (foundArray.containsKey(format)) {
Video newVideo = new Video(meta.ext, meta.type,
foundArray.get(format));
videos.add(newVideo);
Log.d("Downloader", "YouTube Video streaming details: ext:" + newVideo.ext
+ ", type:" + newVideo.type + ", url:" + newVideo.url);
}
}
return videos;
}
private class YouTubePageStreamUriGetter extends AsyncTask<String, String, ArrayList<Video>> {
ProgressDialog progressDialog;
#Override
protected void onPreExecute() {
super.onPreExecute();
progressDialog = ProgressDialog.show(webViewActivity.this, "",
"Connecting to YouTube...", true);
}
#Override
protected ArrayList<Video> doInBackground(String... params) {
ArrayList<Video> fVideos = new ArrayList<>();
String url = params[0];
try {
ArrayList<Video> videos = getStreamingUrisFromYouTubePage(url);
/* Log.e("Downloader","Size of Video : "+videos.size());*/
if (videos != null && !videos.isEmpty()) {
for (Video video : videos)
{
Log.e("Downloader", "ext : " + video.ext);
if (video.ext.toLowerCase().contains("mp4") || video.ext.toLowerCase().contains("3gp") || video.ext.toLowerCase().contains("flv") || video.ext.toLowerCase().contains("webm")) {
ext = video.ext.toLowerCase();
fVideos.add(new Video(video.ext,video.type,video.url));
}
}
return fVideos;
}
} catch (Exception e) {
e.printStackTrace();
Log.e("Downloader", "Couldn't get YouTube streaming URL", e);
}
Log.e("Downloader", "Couldn't get stream URI for " + url);
return null;
}
#Override
protected void onPostExecute(ArrayList<Video> streamingUrl) {
super.onPostExecute(streamingUrl);
progressDialog.dismiss();
if (streamingUrl != null) {
if (!streamingUrl.isEmpty()) {
//Log.e("Steaming Url", "Value : " + streamingUrl);
for (int i = 0; i < streamingUrl.size(); i++) {
Video fX = streamingUrl.get(i);
Log.e("Founded Video", "URL : " + fX.url);
Log.e("Founded Video", "TYPE : " + fX.type);
Log.e("Founded Video", "EXT : " + fX.ext);
}
//new ProgressBack().execute(new String[]{streamingUrl, filename + "." + ext});
}
}
}
}
public void initTypeMap()
{
typeMap.put("13", new Meta("13", "3GP", "Low Quality - 176x144"));
typeMap.put("17", new Meta("17", "3GP", "Medium Quality - 176x144"));
typeMap.put("36", new Meta("36", "3GP", "High Quality - 320x240"));
typeMap.put("5", new Meta("5", "FLV", "Low Quality - 400x226"));
typeMap.put("6", new Meta("6", "FLV", "Medium Quality - 640x360"));
typeMap.put("34", new Meta("34", "FLV", "Medium Quality - 640x360"));
typeMap.put("35", new Meta("35", "FLV", "High Quality - 854x480"));
typeMap.put("43", new Meta("43", "WEBM", "Low Quality - 640x360"));
typeMap.put("44", new Meta("44", "WEBM", "Medium Quality - 854x480"));
typeMap.put("45", new Meta("45", "WEBM", "High Quality - 1280x720"));
typeMap.put("18", new Meta("18", "MP4", "Medium Quality - 480x360"));
typeMap.put("22", new Meta("22", "MP4", "High Quality - 1280x720"));
typeMap.put("37", new Meta("37", "MP4", "High Quality - 1920x1080"));
typeMap.put("33", new Meta("38", "MP4", "High Quality - 4096x230"));
}
Edit 2:
Some time This Code Not worked proper
Same-origin policy
https://en.wikipedia.org/wiki/Same-origin_policy
https://en.wikipedia.org/wiki/Cross-origin_resource_sharing
problem of Same-origin policy. Essentially, you cannot download this file from www.youtube.com because they are different domains. A workaround of this problem is [CORS][1].
Ref : https://superuser.com/questions/773719/how-do-all-of-these-save-video-from-youtube-services-work/773998#773998
url_encoded_fmt_stream_map // traditional: contains video and audio stream
adaptive_fmts // DASH: contains video or audio stream
Each of these is a comma separated array of what I would call "stream objects". Each "stream object" will contain values like this
url // direct HTTP link to a video
itag // code specifying the quality
s // signature, security measure to counter downloading
Each URL will be encoded so you will need to decode them. Now the tricky part.
YouTube has at least 3 security levels for their videos
unsecured // as expected, you can download these with just the unencoded URL
s // see below
RTMPE // uses "rtmpe://" protocol, no known method for these
The RTMPE videos are typically used on official full length movies, and are protected with SWF Verification Type 2. This has been around since 2011 and has yet to be reverse engineered.
The type "s" videos are the most difficult that can actually be downloaded. You will typcially see these on VEVO videos and the like. They start with a signature such as
AA5D05FA7771AD4868BA4C977C3DEAAC620DE020E.0F421820F42978A1F8EAFCDAC4EF507DB5
Then the signature is scrambled with a function like this
function mo(a) {
a = a.split("");
a = lo.rw(a, 1);
a = lo.rw(a, 32);
a = lo.IC(a, 1);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 77);
a = lo.IC(a, 3);
a = lo.wS(a, 44);
return a.join("")
}
This function is dynamic, it typically changes every day. To make it more difficult the function is hosted at a URL such as
http://s.ytimg.com/yts/jsbin/html5player-en_US-vflycBCEX.js
this introduces the problem of Same-origin policy. Essentially, you cannot download this file from www.youtube.com because they are different domains. A workaround of this problem is CORS. With CORS, s.ytimg.com could add this header
Access-Control-Allow-Origin: http://www.youtube.com
and it would allow the JavaScript to download from www.youtube.com. Of course they do not do this. A workaround for this workaround is to use a CORS proxy. This is a proxy that responds with the following header to all requests
Access-Control-Allow-Origin: *
So, now that you have proxied your JS file, and used the function to scramble the signature, you can use that in the querystring to download a video.
ytd2 is a fully functional YouTube video downloader. Check out its source code if you want to see how it's done.
Alternatively, you can also call an external process like youtube-dl to do the job. This is probably the easiest solution but it isn't in "pure" Java.