Im trying to use Jsoup to make http Get request Discord Api, to get users information like this, what am I doing wrong ?
thread {
val id = "MY_ID:"
val token = "MY_TOKEN_BOT"
val source = Jsoup.connect("https://discord.com/api/v9/users/${id}")
.header("Authorization","Bot $token")
.ignoreContentType(true)
.method(Connection.Method.GET)
.execute()
.body()
runOnUiThread {
binding.txv.text = JSONObject(source).toString()
}
}
Response code GET 403
org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=[https://discord.com/api/v9/users/264097054047862794]
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:890)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:829)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:366)
at com.example.jsouptester.MainActivity$testing$1.invoke(MainActivity.kt:80)
at com.example.jsouptester.MainActivity$testing$1.invoke(MainActivity.kt:62)
at kotlin.concurrent.ThreadsKt$thread$thread$1.run(Thread.kt:30)
Related
I am trying to read data from reddit using java. I am using JRAW.
Here is my code:
public class Main {
public static void main(String args[]) {
System.out.println('a');
String username = "dummyName";
UserAgent userAgent = new UserAgent("crawl", "com.example.crawl", "v0.1", username);
Credentials credentials = Credentials.script(username, <password>,<clientID>, <client-secret>);
NetworkAdapter adapter = new OkHttpNetworkAdapter(userAgent);
RedditClient reddit = OAuthHelper.automatic(adapter, credentials);
Account me = reddit.me().about();
System.out.println(me.getName());
SubmissionReference submission = reddit.submission("https://www.reddit.com/r/diabetes/comments/9rlkdm/shady_insurance_work_around_to_pay_for_my_dexcom/");
RootCommentNode rcn = submission.comments();
System.out.println(rcn.getDepth());
System.out.println();
// Submission submission1 = submission.inspect();
// System.out.println(submission1.getSelfText());
// System.out.println(submission1.getUrl());
// System.out.println(submission1.getTitle());
// System.out.println(submission1.getAuthor());
// System.out.println(submission1.getCreated());
System.out.println("-----------------------------------------------------------------");
}
}
I am making two requests as of now, first one is reddit.me().about(); and the second is reddit.submission("https://www.reddit.com/r/diabetes/comments/9rlkdm/ shady_insurance_work_around_to_pay_for_my_dexcom/");
The output is:
a
[1 ->] GET https://oauth.reddit.com/api/v1/me?raw_json=1
[<- 1] 200 application/json: '{"is_employee": false, "seen_layout_switch": true, "has_visited_new_profile": false, "pref_no_profanity": true, "has_external_account": false, "pref_geopopular": "GL(...)
dummyName
[2 ->] GET https://oauth.reddit.com/comments/https%3A%2F%2Fwww.reddit.com%2Fr%2Fdiabetes%2Fcomments%2F9rlkdm%2Fshady_insurance_work_around_to_pay_for_my_dexcom%2F?sort=confidence&sr_detail=false&(...)
[<- 2] 400 application/json: '{"message": "Bad Request", "error": 400}'
Exception in thread "main" net.dean.jraw.ApiException: API returned error: 400 (Bad Request), relevant parameters: []
at net.dean.jraw.models.internal.ObjectBasedApiExceptionStub.create(ObjectBasedApiExceptionStub.java:57)
at net.dean.jraw.models.internal.ObjectBasedApiExceptionStub.create(ObjectBasedApiExceptionStub.java:33)
at net.dean.jraw.RedditClient.request(RedditClient.kt:186)
at net.dean.jraw.RedditClient.request(RedditClient.kt:219)
at net.dean.jraw.RedditClient.request(RedditClient.kt:255)
at net.dean.jraw.references.SubmissionReference.comments(SubmissionReference.kt:50)
at net.dean.jraw.references.SubmissionReference.comments(SubmissionReference.kt:28)
at Main.main(Main.java:36)
Caused by: net.dean.jraw.http.NetworkException: HTTP request created unsuccessful response: GET https://oauth.reddit.com/comments/https%3A%2F%2Fwww.reddit.com%2Fr%2Fdiabetes%2Fcomments%2F9rlkdm%2Fshady_insurance_work_around_to_pay_for_my_dexcom%2F?sort=confidence&sr_detail=false&raw_json=1 -> 400
... 6 more
As it can been that my first request gives me a response of my username but in the second response i am getting a bad request 400 error.
To check whether my client ID and client secret were working correctly I did the same request using python PRAW library.
import praw
from praw.models import MoreComments
reddit = praw.Reddit(client_id=<same-as-in-java>, client_secret=<same-as-in-java>,
password=<same-as-in-java>, user_agent='crawl',
username="dummyName")
submission = reddit.submission(
url='https://www.reddit.com/r/redditdev/comments/1x70wl/how_to_get_all_replies_to_a_comment/')
print(submission.selftext)
print(submission.url)
print(submission.title)
print(submission.author)
print(submission.created_utc)
print('-----------------------------------------------------------------')
This gives the desired result without any errors so the client secret details must be working.
The only doubt I have is in the user agent creation in java UserAgent userAgent = new UserAgent("crawl", "com.example.crawl", "v0.1", username);.
I followed the following link.
What exactly does the target platform, the unique ID or the version mean. I tried to keep the same format as in the link. Also using the same username as in other places. On the other hand the user_agent in python was a string crawl.
Please tell me if I am missing anything and what could be the issue.
Thank you
P.S. I want to do this in java. not python.
Since your first query is working the credentials are correct. In JRAW don't give the whole URL but only the id in the submission function.
Change this
SubmissionReference submission = reddit.submission("https://www.reddit.com/r/diabetes/comments/9rlkdm/shady_insurance_work_around_to_pay_for_my_dexcom/");
to this
SubmissionReference submission = reddit.submission("9rlkdm");
where the id is the random string after /comment/ in the URL.
Hope this helps.
I am trying to make an app in Android and have some problem. I have searched the web but I haven't found an obvious solution.
In my Android app I send a Post request for a login task using Retrofit.
#FormUrlEncoded
#POST("/login")
Call<Boolean> loginUser(#Field("username") String userName, #Field("password") String password);
My server is in Node.js with express and I don't know how to extract the username and password parameters.
my app.js file looks like this:
var http = require('http');
var express = require('express');
var bodyparser = require('body-parser');
var connection = require('./dbConnection');
var app = express();
require('./models')(app);
require('./controllers')(app);
require('./routes')(app);
app.use(bodyparser.urlencoded({extended: true}));
app.use(bodyparser.json());
connection.init();
var server = app.listen(3000, function() {
console.log('Server listening on port ' + server.address().port);
});
And my routes file for login looks like this:
var loginM = require('../models/login');
var loginC = require('../controllers/login');
module.exports = function(app) {
app.post('/login/', function(req, res, next) {
loginM.attemptLogin(req.body, res);
});
}
req.body does not seem to give me the #Field variables, I have tried req.headers and req.params as well with no success. Can someone explain how to extract them?
Much appreciated
On NodeJs side, or better: expressjs app side, you need to use a body-parser middleware, such as https://github.com/expressjs/body-parser
npm install body-parser --save
Since body-parser supports JSON as well as URL encoded input (which retrofit #Field generates) you need to add appropriate middleware functions to app:
var bodyParser = require('body-parser');
// parse JSON inputs
app.use(bodyParser.json());
// Also, parse URL encoded inputs
app.use(bodyParser.urlencoded());
// handle requests
app.post('/login/', function(req, res, next) {
loginM.attemptLogin(req.body, res);
});
Also, remember that you add body-parser middleware Before adding routes/controllers to the app. Because parser middleware should be executed before so that input is parsed by the time request handling logic is executed.
var app = express();
// first
app.use(bodyparser.urlencoded({extended: true}));
app.use(bodyparser.json());
// after
require('./models')(app);
require('./controllers')(app);
require('./routes')(app);
I've been trying to place labels over newly created nodes using Neo4J restful api. The following is the CURL request I've tried.
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://localhost:7474/db/data/");
String propertyUri_labels = "http://localhost:7474/db/data/node/9/labels";
Response response = target
.path(propertyUri_labels)
.request(MediaType.APPLICATION_JSON)
.header("application/xml", "true")
.accept(MediaType.APPLICATION_JSON)
.put(Entity.entity("\"" + "Artist" + "\"", MediaType.APPLICATION_JSON_TYPE));
System.out.println( String.format( "PUT to [%s], status code [%d]",
propertyUri_labels, response.getStatus() ) );
However, I have not been successful and got an status code [400]. Is there something wrong with my URL "http://localhost:7474/db/data/node/9/labels" ?
Pls help.
Thanks!
Try a POST instead of PUT.
Check the documentation => http://neo4j.com/docs/rest-docs/current/#rest-api-node-labels
I have built an app for Shopify Shops and it was working fine until yesterday. Link to the documentation followed to create a Shopify app: https://docs.shopify.com/api/authentication/oauth
But now there seems to be an error while generating an access token. I have been getting the error in response:
{"error":"795: unexpected token at 'code=<my-code>\u0026client_secret=<my-secret>\u0026client_id=<my-client-id>'"}
Here is the gist of code written in Google App Script.
function doPost(e)
{
try
{
var client_id = e.parameter['client_id'];
var client_secret = e.parameter['client_secret'];
var code = e.parameter['code'];
var shopUrl = e.parameter['shopUrl'];
var headers = {
"Accept":"application/json",
"Content-Type":"application/json"
};
var payload = {
"client_id" : client_id,
"client_secret" : client_secret,
"code": code
};
var options =
{
"method" : "POST",
"payload" : payload,
"headers" : headers,
"muteHttpExceptions":true
};
var response = UrlFetchApp.fetch(shopUrl, options);
var data = response.getContentText().split('"')[3];
//response variable gives the following response
// {"error":"795: unexpected token at 'code=<my-code>\u0026client_secret=<my-client-secret>\u0026client_id=<my-client-id>'"}
return ContentService.createTextOutput(data).setMimeType(ContentService.MimeType.TEXT);
}//try
catch(e)
{
//log the exception
}//catch
}//doPost
Did anybody encounter the same error? Please help
Make sure that you requested to right URL, it just likes as
"https://[YOUR_SHOP_DOMAIN]/admin/oauth/access_token".
Make sure that you gave right ID&secret.
Check my PHP-code here for getting Shopify access-token:
ShopifyClient.php (scroll to the function "getAccessToken")
I'm trying to login and extract data from a fantasyfootball website.
I get the following error,
Jul 24, 2015 8:01:12 PM StatsCollector main
SEVERE: null
org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=http://fantasy.premierleague.com/
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:537)
at org.jsoup.helper.HttpConnection$Response.execute(HttpConnection.java:493)
at org.jsoup.helper.HttpConnection.execute(HttpConnection.java:205)
at StatsCollector.main(StatsCollector.java:26)
whenever I try this code. Where am I going wrong?
public class StatsCollector {
public static void main (String [] args){
try {
String url = "http://fantasy.premierleague.com/";
Connection.Response response = Jsoup.connect(url).method(Connection.Method.GET).execute();
Response res= Jsoup
.connect(url)
.data("ismEmail", "example#googlemail.com", "id_password", "examplepassword")
.method(Method.POST)
.execute();
Map<String, String> loginCookies = res.cookies();
Document doc = Jsoup.connect("http://fantasy.premierleague.com/transfers")
.cookies(loginCookies)
.get();
String title = doc.title();
System.out.println(title);
}
catch (IOException ex) {
Logger.getLogger(StatsCollector.class.getName()).log(Level.SEVERE,null,ex);
}
}
}
Response res= Jsoup
.connect(url)
.data("ismEmail", "example#googlemail.com", "id_password", "examplepassword")
.method(Method.POST)
.execute();
Are you trying to execute this actual code? This seems to be an example code with placeholders instead of login credentials. This would explain the error you received, HTTP 403.
Edit 1
My bad. I took a look at the login form on that site, and it seems to me that you confused the id of the input elements ("ismEmail" and "id_password" with the name which gets sent with the form ("email", "password"). Is this working for you?
Response res= Jsoup
.connect(url)
.data("email", "example#googlemail.com", "password", "examplepassword")
.method(Method.POST)
.execute();
Edit 2
Okay, this was stuck in my head, beacause signing into a website with JSoup should not be that hard. I created an account there and tried for myself. Code first:
String url = "https://users.premierleague.com/PremierUser/j_spring_security_check";
Response res = Jsoup
.connect(url)
.followRedirects(false)
.timeout(2_000)
.data("j_username", "<USER>")
.data("j_password", "<PASSWORD>")
.method(Method.POST)
.execute();
Map<String, String> loginCookies = res.cookies();
Document doc = Jsoup.connect("http://fantasy.premierleague.com/squad-selection/")
.cookies(loginCookies)
.get();
So what is happening here? First I realized, that the target of the login form was wrong. The page seems to be built on spring, so the form attributes and target use spring defaults j_spring_security_check, j_username and j_password. Then a read timeout occurred to me, until I set the flag followRedirects(false). I can only guess why this helped, but maybe this is a protection against crawlers?
In the end i try to connect to the squad selection page, and the parsed response contains my personal view and data. This code seems to work for me, would you give it a try?