Very fast URL check - java

So I creating an android app where I need to check if a file exist on a remote server really fast because I have to test ~1000 links before the app become usable.
I currently call a function that return the URL if it's valid and null if not.
public String CheckUrl(String url) {
try {
URL urll = new URL(url);
HttpURLConnection huc = (HttpURLConnection) urll.openConnection();
huc.setRequestMethod("GET"); //OR huc.setRequestMethod ("HEAD");
huc.connect();
int code = huc.getResponseCode();
System.out.println(code);
if (code == 200) {
return url;
} else {
return null;
}
} catch (Exception e) {
return null;
}
}
and I use it like this:
for (Element episode : episodes) {
globalEpisodeCounter++;
localEpisodeCounter++;
MLP_Episode currentEpisode = new MLP_Episode();
Elements links = episode.getElementsByTag("a");
Element linkObj = links.get(0);
Element thumObj = linkObj.getElementsByTag("img").get(0);
Element titleObj = linkObj.getElementsByTag("b").get(0);
int notRealsead = episode.getElementsByClass("btn btn-sm btn-error").size();
Boolean epReleased = false;
if (notRealsead == 0) {
epReleased = true;
}
currentEpisode.url = "https://www.newlunarrepublic.fr" + linkObj.attributes().get("href");
currentEpisode.thumbUrl = "https://www.newlunarrepublic.fr" + thumObj.attributes().get("src");
currentEpisode.title = titleObj.text();
currentEpisode.released = epReleased;
currentEpisode.id_local = localEpisodeCounter;
currentEpisode.id_global = globalEpisodeCounter;
currentEpisode.in_season_num = seasonCounter;
if (epReleased) {
currentEpisode.url_vo_1080p = CheckUrl(
"---------/NLR-1080p-" + addZero(seasonCounter) + "x" + addZero(localEpisodeCounter) + ".webm");
}
epList.add(currentEpisode);
}
At the and end of the search the search thread call a function to update UI
But the down side of the function is that it's very slow 1-2 link/sec which ranslate in 15min waiting before the app is usable

So the answer was to run the check in a separate thread:
Thread thread = new Thread() {
#Override
public void run() {
try {
currentEpisode.url_vo_1080p = CheckUrl("------------/NLR-1080p-"+addZero(seasonCounter2)+"x"+addZero(localEpisodeCounter2)+".webm");
}
catch (Exception e) {}
}
};
thread.start();

Related

What‘s the different of entry.softTtl and entry.ttl in volley?

In class HttpHeaderParser:
public static Cache.Entry parseCacheHeaders(NetworkResponse response) {
long now = System.currentTimeMillis();
Map<String, String> headers = response.headers;
long serverDate = 0;
long serverExpires = 0;
long softExpire = 0;
long maxAge = 0;
boolean hasCacheControl = false;
String serverEtag = null;
String headerValue;
headerValue = headers.get("Date");
if (headerValue != null) {
serverDate = parseDateAsEpoch(headerValue);
}
headerValue = headers.get("Cache-Control");
if (headerValue != null) {
hasCacheControl = true;
String[] tokens = headerValue.split(",");
for (int i = 0; i < tokens.length; i++) {
String token = tokens[i].trim();
if (token.equals("no-cache") || token.equals("no-store")) {
return null;
} else if (token.startsWith("max-age=")) {
try {
maxAge = Long.parseLong(token.substring(8));
} catch (Exception e) {
}
} else if (token.equals("must-revalidate") || token.equals("proxy-revalidate")) {
maxAge = 0;
}
}
}
headerValue = headers.get("Expires");
if (headerValue != null) {
serverExpires = parseDateAsEpoch(headerValue);
}
serverEtag = headers.get("ETag");
// Cache-Control takes precedence over an Expires header, even if both exist and Expires
// is more restrictive.
if (hasCacheControl) {
softExpire = now + maxAge * 1000;
} else if (serverDate > 0 && serverExpires >= serverDate) {
// Default semantic for Expire header in HTTP specification is softExpire.
softExpire = now + (serverExpires - serverDate);
}
Cache.Entry entry = new Cache.Entry();
entry.data = response.data;
entry.etag = serverEtag;
entry.softTtl = softExpire;
entry.ttl = entry.softTtl;
entry.serverDate = serverDate;
entry.responseHeaders = headers;
return entry;
}
entry.softTtl = softExpire;
entry.ttl = entry.softTtl;
This two variables has the same value, so why?
In class CacheDispatcher
#Override
public void run() {
...
...
// If it is completely expired, just send it to the network.
if (entry.isExpired()) {
request.addMarker("cache-hit-expired");
request.setCacheEntry(entry);
mNetworkQueue.put(request);
continue;
}
...
if (!entry.refreshNeeded()) {
// Completely unexpired cache hit. Just deliver the response.
mDelivery.postResponse(request, response);
} else {
...
// Post the intermediate response back to the user and have
// the delivery then forward the request along to the network.
mDelivery.postResponse(request, response, new Runnable() {
#Override
public void run() {
try {
mNetworkQueue.put(request);
} catch (InterruptedException e) {
// Not much we can do about this.
}
}
});
}
...
}
How can I differentiate between the cast of entry.isExpired() and entry.refreshNeeded() as the values are the same?
Have a look at this post : Group Google : Volley Users - soft and hard ttl on cache
refreshNeeded() and isExpired() are not exactly the same. One compares to the ttl value and the other to softTtl. This can be used to implement request semantics where you will return a response from cache even if it is "soft" expired, but will then go to the network and refresh, returning a new response if the data has changed.

Grabbing HTML source, how can I grab the HTML in the state it would be in after the page loads?

I have been working on grabbing html source for a client I made, so I decided that I would like to get the status of the minecraft login and session servers. There is already a website that does this, so I figured I could just grab the HTML and search through it with Java. After a test run, I noticed the output was not what I am looking for.
I'm hoping to be able to achieve an output of something more like on this site.
MC Status
public class Status {
private static Minecraft mc = Minecraft.getMinecraft();
public static void toggle(String s) {
if (s.contentEquals("-status")) {
Variables.status = !Variables.status;
if (Variables.status) {
enable();
} else {
disable();
}
} else {
update();
}
}
public static void enable() {
mc.thePlayer.addChatMessage(Variables.ChatTeal + "Minecraft Status enabled.");
}
public static void disable() {
mc.thePlayer.addChatMessage(Variables.ChatTeal + "Minecraft Status disabled.");
}
#SuppressWarnings("unchecked")
public static void update() {
BufferedReader br;
String line;
String s;
int i = 1;
try {
URL url = new URL("http://xpaw.ru/mcstatus/");
HttpURLConnection httpcon = (HttpURLConnection) url.openConnection();
httpcon.addRequestProperty("User-Agent", "Mozilla/4.76");
httpcon.connect();
br = new BufferedReader(new InputStreamReader((InputStream) httpcon.getContent()));
Variables.statusList.clear();
while ((line = br.readLine()) != null) {
if (i == 65 || i == 72 || i == 79 || i == 86 || i == 93) {
s = line.replaceAll("<div class=\"name\">", "").trim();
line = s.replaceAll("</div>", "").trim();
Variables.statusNameList.add(line);
System.out.println(line);
}
if (i == 66 || i == 73 || i == 80 || i == 87 || i == 94) {
s = line.replaceAll("<h2 class=\"status\">", "").trim();
line = s.replaceAll("…</h2>", "").trim();
Variables.statusList.add(line);
System.out.println(line);
}
i++;
}
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
The page is using JavaScript to load the statuses of Mojang's services. Additionally, the page is behind cloudflare which can occasionally do browser checks. You would need to use a Java browser emulator, such as HtmlUnit. Why don't you just ping Mojang's servers yourself and see if they're online or not? Surely that's much easier than parsing this page.
I managed to greatly simplify this and ended up settling for a simple text output. Seems to work fine for my needs.
public class Status {
private static Minecraft mc = Minecraft.getMinecraft();
public static void getStatus() {
String status = IOHandler.getHtmlSource("http://status.mojang.com/check");
String[] statusArray = status.split(",");
String a = "";
for (int i = 0; i < statusArray.length; i++) {
a = statusArray[i].replaceAll("\\[", "").replaceAll("\\]", "")
.replaceAll("\\{", "").replaceAll("\"", " ")
.replaceAll("\\:", "-").replaceAll("\\}", "")
.trim();
if (a.contains("green")) {
a = a.replaceAll(" - green", "");
mc.thePlayer.addChatMessage(Variables.ChatBGreen + a);
} else if (a.contains("yellow")) {
a = a.replaceAll(" - yellow", "");
mc.thePlayer.addChatMessage(Variables.ChatYellow + a);
} else if (a.contains("red")) {
a = a.replaceAll(" - red", "");
mc.thePlayer.addChatMessage(Variables.ChatDRed + a);
} else {
mc.thePlayer.addChatMessage(Variables.ChatDRed + "ERROR");
}
}
}
}

ElasticSearch index exists not working / reliable

I am writing a simple Java wrapper around ElasticSearch's admin client. To test it I have a main method that first checks if an index exists (IndicesExistsRequest), if so deletes it (DeleteIndexRequest), and creates the index again. See code below. Yet I consistently get an IndexAlreadyExistsException.
By the way I am trying to get a client for the node that you start from the command prompt (by simply typing "elastic search"). I have tried every combination of methods on nodeBuilder's fluent interface, but I can't seem to get one.
public static void main(String[] args) {
ElasticSearchJavaClient esjc = new ElasticSearchJavaClient("nda");
if (esjc.indexExists()) {
esjc.deleteIndex();
}
esjc.createIndex();
URL url = SchemaCreator.class.getResource("/elasticsearch/specimen.type.json");
String mappings = FileUtil.getContents(url);
esjc.createType("specimen", mappings);
}
final Client esClient;
final IndicesAdminClient adminClient;
final String indexName;
public ElasticSearchJavaClient(String indexName) {
this.indexName = indexName;
esClient = nodeBuilder().clusterName("elasticsearch").client(true).node().client();
adminClient = esClient.admin().indices();
}
public boolean deleteIndex() {
logger.info("Deleting index " + indexName);
DeleteIndexRequest request = new DeleteIndexRequest(indexName);
try {
DeleteIndexResponse response = adminClient.delete(request).actionGet();
if (!response.isAcknowledged()) {
throw new Exception("Failed to delete index " + indexName);
}
logger.info("Index deleted");
return true;
} catch (IndexMissingException e) {
logger.info("No such index: " + indexName);
return false;
}
}
public boolean indexExists() {
logger.info(String.format("Verifying existence of index \"%s\"", indexName));
IndicesExistsRequest request = new IndicesExistsRequest(indexName);
IndicesExistsResponse response = adminClient.exists(request).actionGet();
if (response.isExists()) {
logger.info("Index exists");
return true;
}
logger.info("No such index");
return false;
}
public void createIndex() {
logger.info("Creating index " + indexName);
CreateIndexRequest request = new CreateIndexRequest(indexName);
IndicesAdminClient iac = esClient.admin().indices();
CreateIndexResponse response = iac.create(request).actionGet();
if (!response.isAcknowledged()) {
throw new Exception("Failed to create index " + indexName);
}
logger.info("Index created");
}
You can also execute a synchronous request like this:
boolean exists = client.admin().indices()
.prepareExists(INDEX_NAME)
.execute().actionGet().isExists();
Here is my solution when using RestHighLevelClient client;
Here a code-snippet: :
public boolean checkIfIndexExists(String indexName) throws IOException {
Response response = client.getLowLevelClient().performRequest("HEAD", "/" + indexName);
int statusCode = response.getStatusLine().getStatusCode();
return (statusCode != 404);
}
A contribution for someone else !
The skgemini's answer is ok if you want to check if index is available by the actual index name or any of its aliases.
If you however want to check only by the index name, here is how.
public boolean checkIfIndexExists(String index) {
IndexMetaData indexMetaData = client.admin().cluster()
.state(Requests.clusterStateRequest())
.actionGet()
.getState()
.getMetaData()
.index(index);
return (indexMetaData != null);
}
OK, I figured out a solution. Since the java client's calls are done asynchronously you have to use the variant which takes an action listener. The solution still gets a bit contrived though:
// Inner class because it's just used to be thrown out of
// the action listener implementation to signal that the
// index exists
private class ExistsException extends RuntimeException {
}
public boolean exists() {
logger.info(String.format("Verifying existence of index \"%s\"", indexName));
IndicesExistsRequest request = new IndicesExistsRequest(indexName);
try {
adminClient.exists(request, new ActionListener<IndicesExistsResponse>() {
public void onResponse(IndicesExistsResponse response) {
if (response.isExists()) {
throw new ExistsException();
}
}
public void onFailure(Throwable e) {
ExceptionUtil.smash(e);
}
});
}
catch (ExistsException e) {
return true;
}
return false;
}
I had the same issue but i didn't like the solution which uses an ActionListener. ElasticSearch also offers a Future variant (at least at version 6.1.0).
Here a code-snippet:
public boolean doesIndexExists(String indexName, TransportClient client) {
IndicesExistsRequest request = new IndicesExistsRequest(indexName);
ActionFuture<IndicesExistsResponse> future = client.admin().indices().exists(request);
try {
IndicesExistsResponse response = future.get();
boolean result = response.isExists();
logger.info("Existence of index '" + indexName + "' result is " + result);
return result;
} catch (InterruptedException | ExecutionException e) {
logger.error("Exception at waiting for IndicesExistsResponse", e);
return false;//do some clever exception handling
}
}
May be this helps someone else too. Cheers!
This works on Elasticsearch 7.x:
public boolean indexExists(String indexName) throws IOException {
return client.indices().exists(new org.elasticsearch.client.indices.GetIndexRequest(indexName), RequestOptions.DEFAULT);
}

HTTP Post request on BlackBerry

I am trying to send a json string , from my BlackBerry OS < 7.X application to my Server. I am trying to use an HTTP Post request. What i have done so far is :
String httpURL = "http://ip_of_my_server/phpServer/receiver2.php/" + jsonString;
try {
HttpConnection httpConn;
httpConn = (HttpConnection) Connector.open(httpURL + getConnectionString());
httpConn.setRequestMethod(HttpConnection.POST);
httpConn.setRequestProperty("Content-Type", "application/json; charset=utf-8");
DataOutputStream _outStream = new DataOutputStream(httpConn.openDataOutputStream());
byte[] request_body = httpURL.getBytes();
for (int i = 0; i < request_body.length; i++) {
_outStream.writeByte(request_body[i]);
}
DataInputStream _inputStream = new DataInputStream(httpConn.openInputStream());
StringBuffer _responseMessage = new StringBuffer();
int ch;
while ((ch = _inputStream.read()) != -1) {
_responseMessage.append((char) ch);
}
String res = (_responseMessage.toString());
String response = res.trim();
System.out.println("!!!!!!!!!!!!!! Response is: " + response);
httpConn.close();
} catch (Exception e) {
Dialog.alert("Error - " + e.toString());
}
The code works in a way that i dont fully understand. The author of the above code suggested to use as an httpURL the URL of the server + my json string. The final result is that on my server instead of arriving the json string , is arriving a string like that :
http://ip_of_my_server/phpServer/receiver2.php/ + jsonString
I am not familiar with java. I have previously done this for WP7 and iOS and in the httpUrl i put my servers URL and then with a command i was "appending" my json string to the http request and everything worked as expected.
How can i append the json string to the above HttpRequest , instead of adding it to the URL so that in the server arrives the JSON String only?
EDIT (providing the rest of the code that was used)
//used to specify connection type ( wifi - 3g - etc )
public static String getConnectionString() {
String connectionString = null;
// Wifi is the preferred transmission method
if (WLANInfo.getWLANState() == WLANInfo.WLAN_STATE_CONNECTED) {
connectionString = ";interface=wifi";
}
// Is the carrier network the only way to connect?
else if ((CoverageInfo.getCoverageStatus() & CoverageInfo.COVERAGE_DIRECT) == CoverageInfo.COVERAGE_DIRECT) {
String carrierUid = getCarrierBIBSUid();
if (carrierUid == null) {
// Has carrier coverage, but not BIBS. So use the carrier's TCP network
connectionString = ";deviceside=true";
} else {
// otherwise, use the Uid to construct a valid carrier BIBS request
connectionString = ";deviceside=false;connectionUID="+carrierUid + ";ConnectionType=mds-public";
}
}
// Check for an MDS connection instead (BlackBerry Enterprise Server)
else if ((CoverageInfo.getCoverageStatus() & CoverageInfo.COVERAGE_MDS) == CoverageInfo.COVERAGE_MDS) {
connectionString = ";deviceside=false";
}
// If there is no connection available abort to avoid hassling the user unnecessarily
else if (CoverageInfo.getCoverageStatus() == CoverageInfo.COVERAGE_NONE) {
connectionString = "none";
}
// In theory, all bases are covered by now so this shouldn't be reachable.But hey, just in case ...
else {
connectionString = ";deviceside=true";
}
System.out.println("!!!!!!!!!!!!!! Connection type is: " + connectionString);
return connectionString;
}
/**
* Looks through the phone's service book for a carrier provided BIBS
* network
*
* #return The uid used to connect to that network.
*/
private synchronized static String getCarrierBIBSUid() {
ServiceRecord[] records = ServiceBook.getSB().getRecords();
int currentRecord;
for (currentRecord = 0; currentRecord < records.length; currentRecord++) {
if (records[currentRecord].getCid().toLowerCase().equals("ippp")) {
if (records[currentRecord].getName().toLowerCase()
.indexOf("bibs") >= 0) {
return records[currentRecord].getUid();
}
}
}
return null;
}
The the first line should be simply your URL
String httpURL = "http://ip_of_my_server/phpServer/receiver2.php";
And you should only send the json string to the server as request.
instead of byte[] request_body = httpURL.getBytes();
use byte[] request_body = jsonString.getBytes();
Here is the method for OS 5.0 and above
public static HttpConnection getHttpConnection(String url, byte[] postData) {
HttpConnection conn = null;
OutputStream out = null;
try {
conn = (HttpConnection) new ConnectionFactory().getConnection(url).getConnection();
if (conn != null) {
if (postData == null) {
conn.setRequestMethod(HttpConnection.GET);
conn.setRequestProperty("User-Agent", "Profile/MIDP-2.0 Configuration/CLDC-1.0");
} else {
conn.setRequestMethod(HttpConnection.POST);
conn.setRequestProperty("Content-Length", String.valueOf(postData.length));
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("User-Agent", "Profile/MIDP-2.0 Configuration/CLDC-1.0");
out = conn.openOutputStream();
out.write(postData);
out.flush();
}
if (conn.getResponseCode() != 0) {
return conn;
}
}
} catch (Exception e) {
} finally {
try {
out.close();
} catch (Exception e2) {
}
}
//Only if exception occurs, we close the connection.
//Otherwise the caller should close the connection himself.
try {
conn.close();
} catch (Exception e1) {
}
return null;
}
Here is the complete class if you want it to work with OS 4.2 and above. You may need to replace the constant COVERAGE_DIRECT by its value 1, if you want to compile it with < 4.5.
public class ConnectionHelper {
/**
* Returns the working connection type. The connection types can be BIS, BES, TCP, WAP2, TCPIP
*/
public static HttpConnection getHttpConnection(String url, byte[] postData) {
int[] preferredOrder = new int[] { CONNECTION_WIFI, CONNECTION_BIS, CONNECTION_BES, CONNECTION_UNITE, CONNECTION_WAP2, CONNECTION_TCPIP, };
for (int i = 0; i < preferredOrder.length; i++) {
int type = preferredOrder[i];
if (isPresent(type)) {
HttpConnection conn = null;
OutputStream out = null;
try {
conn = (HttpConnection) Connector.open(convertURL(type, url));
if (conn != null) {
if (postData == null) {
conn.setRequestMethod(HttpConnection.GET);
conn.setRequestProperty("User-Agent", "Profile/MIDP-2.0 Configuration/CLDC-1.0");
} else {
conn.setRequestMethod(HttpConnection.POST);
conn.setRequestProperty(HttpProtocolConstants.HEADER_CONTENT_LENGTH, String.valueOf(postData.length));
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("User-Agent", "Profile/MIDP-2.0 Configuration/CLDC-1.0");
out = conn.openOutputStream();
out.write(postData);
out.flush();
}
if (conn.getResponseCode() != 0) {
return conn;
}
}
} catch (Exception e) {
} finally {
try {
out.close();
} catch (Exception e2) {
}
}
}
}
// Only if exception occurs, we close the connection.
// Otherwise the caller should close the connection himself.
try {
conn.close();
} catch (Exception e1) {
}
return null;
}
/** Stores transport ServiceBooks if found. Otherwise, null */
private static ServiceRecord srMDS, srWiFi, srBIS, srWAP2, srUnite;
private static final int CONNECTION_DEFAULT = 0;
private static final int CONNECTION_BIS = 1;
private static final int CONNECTION_BES = 2;
private static final int CONNECTION_TCPIP = 3;
private static final int CONNECTION_WIFI = 4;
private static final int CONNECTION_WAP2 = 5;
private static final int CONNECTION_UNITE = 6;
private static final int CONFIG_TYPE_BES = 1;
private static final String UNITE_NAME = "Unite";
private static void checkTransportAvailability() {
initializeTransportAvailability();
}
/**
* Initializes the ServiceRecord instances for each transport (if available). Otherwise leaves it null. Also determines if sufficient coverage is available for each transport
* and sets coverage* flags.
*/
private static void initializeTransportAvailability() {
ServiceBook sb = ServiceBook.getSB();
ServiceRecord[] records = sb.getRecords();
for (int i = 0; i < records.length; i++) {
ServiceRecord myRecord = records[i];
String cid, uid;
if (myRecord.isValid() && !myRecord.isDisabled()) {
cid = myRecord.getCid().toLowerCase();
uid = myRecord.getUid().toLowerCase();
// BIS
if (cid.indexOf("ippp") != -1 && uid.indexOf("gpmds") != -1) {
srBIS = myRecord;
}
// BES
if (cid.indexOf("ippp") != -1 && uid.indexOf("gpmds") == -1) {
srMDS = myRecord;
}
// WiFi
if (cid.indexOf("wptcp") != -1 && uid.indexOf("wifi") != -1) {
srWiFi = myRecord;
}
// Wap2.0
if (cid.indexOf("wptcp") != -1 && uid.indexOf("wifi") == -1 && uid.indexOf("mms") == -1) {
srWAP2 = myRecord;
}
// Unite
if (getConfigType(myRecord) == CONFIG_TYPE_BES && myRecord.getName().equals(UNITE_NAME)) {
srUnite = myRecord;
}
}
}
}
/**
* Gets the config type of a ServiceRecord using getDataInt below
*
* #param record
* A ServiceRecord
* #return configType of the ServiceRecord
*/
private static int getConfigType(ServiceRecord record) {
return getDataInt(record, 12);
}
/**
* Gets the config type of a ServiceRecord. Passing 12 as type returns the configType.
*
* #param record
* A ServiceRecord
* #param type
* dataType
* #return configType
*/
private static int getDataInt(ServiceRecord record, int type) {
DataBuffer buffer = null;
buffer = getDataBuffer(record, type);
if (buffer != null) {
try {
return ConverterUtilities.readInt(buffer);
} catch (EOFException e) {
return -1;
}
}
return -1;
}
/**
* Utility Method for getDataInt()
*/
private static DataBuffer getDataBuffer(ServiceRecord record, int type) {
byte[] data = record.getApplicationData();
if (data != null) {
DataBuffer buffer = new DataBuffer(data, 0, data.length, true);
try {
buffer.readByte();
} catch (EOFException e1) {
return null;
}
if (ConverterUtilities.findType(buffer, type)) {
return buffer;
}
}
return null;
}
private static String convertURL(int connectionType, String url) {
switch (connectionType) {
case CONNECTION_BES:
url += ";deviceside=false";
break;
case CONNECTION_BIS:
url += ";deviceside=false" + ";ConnectionType=mds-public";
break;
case CONNECTION_TCPIP:
url += ";deviceside=true";
break;
case CONNECTION_WIFI:
url += ";interface=wifi";
break;
case CONNECTION_WAP2:
url += ";deviceside=true;ConnectionUID=" + srWAP2.getUid();
break;
case CONNECTION_UNITE:
url += ";deviceside=false;ConnectionUID=" + srUnite.getUid();
break;
}
return url;
}
private static boolean isPresent(int connectionType) {
checkTransportAvailability();
switch (connectionType) {
case CONNECTION_BIS:
return (srBIS != null && CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_BIS_B));
case CONNECTION_BES:
return (srMDS != null && CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_MDS));
case CONNECTION_WIFI:
return (srWiFi != null && CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_DIRECT, RadioInfo.WAF_WLAN, false));
case CONNECTION_TCPIP:
return (CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_DIRECT));
case CONNECTION_WAP2:
return (srWAP2 != null && CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_DIRECT));
case CONNECTION_UNITE:
return (srUnite != null && CoverageInfo.isCoverageSufficient(CoverageInfo.COVERAGE_MDS));
case CONNECTION_DEFAULT:
return true;
}
return false;
}
}
And finally post your data.
public void sendJson(String jsonString) {
String httpURL = "http://ip_of_my_server/phpServer/receiver2.php";
HttpConnection httpConn = null;
try {
httpConn = getHttpConnection(httpURL, jsonString.getBytes());
if(httpConn.getResponseCode() == 200) {
//If you need the output, then read it. Otherwise comment it.
byte[] data = IOUtilities.streamToBytes(httpConn.openInputStream());
String response = new String(data);
System.out.println("!!!!!!!!!!!!!! Response is: " + response);
}
} catch (Exception e) {
}
finally {
try {
httpConn.close();
} catch (Exception e2) {
}
}
}

Getting metadata from SHOUTcast using IcyStreamMeta

I am writing an app for Android that grabs meta data from SHOUTcast mp3 streams. I am using a pretty nifty class I found online that I slightly modified, but I am still having 2 problems.
1) I have to continuously ping the server to update the metadata using a TimerTask. I am not fond of this approach but it was all I could think of.
2) There is a metric tonne of garbage collection while my app is running. Removing the TimerTask got rid of the garbage collection issue so I am not sure if I am just doing it wrong or if this is normal.
Here is the class I am using:
public class IcyStreamMeta {
protected URL streamUrl;
private Map<String, String> metadata;
private boolean isError;
public IcyStreamMeta(URL streamUrl) {
setStreamUrl(streamUrl);
isError = false;
}
/**
* Get artist using stream's title
*
* #return String
* #throws IOException
*/
public String getArtist() throws IOException {
Map<String, String> data = getMetadata();
if (!data.containsKey("StreamTitle"))
return "";
try {
String streamTitle = data.get("StreamTitle");
String title = streamTitle.substring(0, streamTitle.indexOf("-"));
return title.trim();
}catch (StringIndexOutOfBoundsException e) {
return "";
}
}
/**
* Get title using stream's title
*
* #return String
* #throws IOException
*/
public String getTitle() throws IOException {
Map<String, String> data = getMetadata();
if (!data.containsKey("StreamTitle"))
return "";
try {
String streamTitle = data.get("StreamTitle");
String artist = streamTitle.substring(streamTitle.indexOf("-")+1);
return artist.trim();
} catch (StringIndexOutOfBoundsException e) {
return "";
}
}
public Map<String, String> getMetadata() throws IOException {
if (metadata == null) {
refreshMeta();
}
return metadata;
}
public void refreshMeta() throws IOException {
retreiveMetadata();
}
private void retreiveMetadata() throws IOException {
URLConnection con = streamUrl.openConnection();
con.setRequestProperty("Icy-MetaData", "1");
con.setRequestProperty("Connection", "close");
//con.setRequestProperty("Accept", null);
con.connect();
int metaDataOffset = 0;
Map<String, List<String>> headers = con.getHeaderFields();
InputStream stream = con.getInputStream();
if (headers.containsKey("icy-metaint")) {
// Headers are sent via HTTP
metaDataOffset = Integer.parseInt(headers.get("icy-metaint").get(0));
} else {
// Headers are sent within a stream
StringBuilder strHeaders = new StringBuilder();
char c;
while ((c = (char)stream.read()) != -1) {
strHeaders.append(c);
if (strHeaders.length() > 5 && (strHeaders.substring((strHeaders.length() - 4), strHeaders.length()).equals("\r\n\r\n"))) {
// end of headers
break;
}
}
// Match headers to get metadata offset within a stream
Pattern p = Pattern.compile("\\r\\n(icy-metaint):\\s*(.*)\\r\\n");
Matcher m = p.matcher(strHeaders.toString());
if (m.find()) {
metaDataOffset = Integer.parseInt(m.group(2));
}
}
// In case no data was sent
if (metaDataOffset == 0) {
isError = true;
return;
}
// Read metadata
int b;
int count = 0;
int metaDataLength = 4080; // 4080 is the max length
boolean inData = false;
StringBuilder metaData = new StringBuilder();
// Stream position should be either at the beginning or right after headers
while ((b = stream.read()) != -1) {
count++;
// Length of the metadata
if (count == metaDataOffset + 1) {
metaDataLength = b * 16;
}
if (count > metaDataOffset + 1 && count < (metaDataOffset + metaDataLength)) {
inData = true;
} else {
inData = false;
}
if (inData) {
if (b != 0) {
metaData.append((char)b);
}
}
if (count > (metaDataOffset + metaDataLength)) {
break;
}
}
// Set the data
metadata = IcyStreamMeta.parseMetadata(metaData.toString());
// Close
stream.close();
}
public boolean isError() {
return isError;
}
public URL getStreamUrl() {
return streamUrl;
}
public void setStreamUrl(URL streamUrl) {
this.metadata = null;
this.streamUrl = streamUrl;
this.isError = false;
}
public static Map<String, String> parseMetadata(String metaString) {
Map<String, String> metadata = new HashMap<String, String>();
String[] metaParts = metaString.split(";");
Pattern p = Pattern.compile("^([a-zA-Z]+)=\\'([^\\']*)\\'$");
Matcher m;
for (int i = 0; i < metaParts.length; i++) {
m = p.matcher(metaParts[i]);
if (m.find()) {
metadata.put((String)m.group(1), (String)m.group(2));
}
}
return metadata;
}
}
And here is my timer:
private void getMeta() {
timer.schedule(new TimerTask() {
public void run() {
try {
icy = new IcyStreamMeta(new URL(stationUrl));
runOnUiThread(new Runnable() {
public void run() {
try {
artist.setText(icy.getArtist());
title.setText(icy.getTitle());
} catch (IOException e) {
e.printStackTrace();
} catch (StringIndexOutOfBoundsException e) {
e.printStackTrace();
}
}
});
} catch (MalformedURLException e) {
e.printStackTrace();
}
}
},0,5000);
}
Much appreciation for any assistance!
I've replaced the IcyStreamMeta class in my program and am getting the meta data from the 7.html file that is a part of the SHOUTcast spec. Far less data usage and all that so I feel it is a better option.
I am still using the TimerTask, which is acceptable. There is practically no GC any more and I am happy with using 7.html and a little regex. :)

Categories

Resources