Well, I'm kinda new to mobile app development, but I have a fair amount of experience with Java. I stumbled upon CodeNameOne and thought I'd try to make an app based on "It's Learning". Unfortunately this site won't let me get straight to the logon page without going through the main portal first. So here's my question:
Is there any way (in CodeNameOne) to automate navigation on the web in the background and get information from the site, such as login and the newsfeed, then send data from the app and to the web page?
(Something like the way HtmlUnit works is what I'm looking for)
You can do some of that work using ConnectionRequest to simulate user interaction but that would probably be fragile as Andrew mentioned. The main issue is that they might change the site structure and your HtmlUnit code would break.
What we do when faced with such a situation is create a server that abstracts the "scraping" behind a set of clearly defined REST API's. That way if the site design changes we just update the server to match the change and all the clients out there will work as usual without requiring an update.
Naturally there is always a possibility of the site blocking you or even blocking your server IP address/range so Andrews advice of actually talking to the guys is sound advice.
Related
I want to create an app for a website that I don't own, like youtube or reddit. Every time I search for help, all I keep getting is how to create a webview. I've done that but that's not what I want. I want to take the website's content and create an app with it, like create my own buttons and such. I want to convert a website into an app.
I was thinking of somehow loading the website in the background and creating activities/buttons/intents that basically do what the webpage does but I don't know how to go about that or if there's a better way.
Thank you in advance.
Please let me know if clarification is needed.
Alright i got -3 votes for some reason and it has been 12 hours since i posted this question, i'm guessing the question has been consumed by the abyss of questions by now.
I made a reddit post and got some responses:
one user:
The term you’re looking for is “web scraping“ or “site scraping”. I think that’s probably what you need to research before you’re ready to handle how to make that into an app.
That’s a good starting point and once you have the content from scraping, making the app is no different than any other app.
second user:
If the website has a public (or private) API, you can use your buttons and networking calls to their API to build your client app. You are building a "YouTube client" in this case using the YouTube API.
If they dont have an API and you don't want to display their website, what you are doing is trying to reverse engineer their website and hijack the UI which is not a good practice and may even be illegal depending on the site and terms.
third user:
#second user is right on the money, but one thing I'll like to add is, the goal is to get data off of the website so that you can use in your app. An API is one (convenient) way of doing that but others are,
RSS or similar feed content (prefer this if there website supports)
HTML scraping
The above two will require you store the data somewhere else (and possible expose an API instead) because of technical reasons: you don't have access to older entries in RSS feeds and it's atrocious to keep on scraping everytime you need to access the website data. They can be done on the client (mobile app) but are best done on a server. This means you might need to get your hands dirty writing server code but you don't have to.
/thread
I'm going to do research on website scrapping and APIs to see which route i should go.
Hopefully this helped someone else.
I'm having a small problem with a personal project, I'm trying to create an auction site in React - it's an amazing learning experience - with a Java backend.
One trouble I've stumbled upon is how to "sync" data effectively, when new data is inserted into the database from a different user, such as if a user would bid on an item, how would other users update their data?
The only way I can see it happening is with a push model, or constantly polling for data from the server. Both seem pretty out of reach, to be honest, although the push-model seems the more likely of the two.
I could somehow set up an observer pattern with users "registered" when viewing an item, although if there is a better method, I would love to hear it.
I already tried searching for answers, but either none exist (I doubt it), or I'm simply terrible at using Google and don't know the keywords to search for to get the specific results I need. (Likely)
Thanks in advance!
You are looking for a kind of server push technology. Try researching more into Websockets or Server-Sent-Events.
A start could be: WebSockets vs. Server-Sent events/EventSource
I am going to develop uber-like application.Here I have to send latitude and longitude to web and mobile devices continuously with my service,What I have do to get this.
Can anyone please give some idea.
You should start by designing how the application is to be used, seen from all the different users perspectives.
For instance is this a web app, or a native app, or both?
Then from that knowledge, you need to define a communication protocol.
You should be able to determine if the client will be polling for data, or if you need to push it from the server onto the clients.
This also goes for the data that travels the other way.
From here you choose a language for programming, and then start doing some proof of concept tests.
The choice will depend on the chosen underlying technologies
(web / native / os / available libraries)
After some test work you may have something that works, then you need to review or add security to the communication, cause we do not want everyone collecting location data from everyone that has the app installed.
Then run beta trials and eliminate the worst bugs, and then release the app.
You'll want some sort of asynchronous task which can get new data from your server and refresh the mobile and web content to reflect the content of the server. You'll also want to notify the server whenever you make local changes to content and want to reflect those changes. Android provides the SyncAdapter pattern as a way to easily solve this pattern. You'll need to register user accounts, and then Android will perform lots of magic for you, and allow you to automatically sync. Here's a good tutorial: http://www.c99.org/2010/01/23/writing-an-android-sync-provider-part-1/
I have done some researching but I haven't find a clean explanation of how it all work.
I would like to know how does applicationa like Instagram, Wordcrack (online game), AirBnb and many other app have tier "Login with Facebook" done?
I currently work on the Java REST services that backups up an iOS app.
If some one already has done a similar things in their app/server I would really appreciate some help
I'm not looking for ready to use code answer but more of a general idea of the architecture to achieve this.
thank you!
I've done it in JavaScript, but the principle will surely be the same.
You have an API, provided by facebook. There are some information there :
https://developers.facebook.com/docs/apis-and-sdks#third-party-sdks
And, more than that, you have declare your own app on facebook. This has, for me, 2 goals :
provide the url of your server, so when you'll call facebook with the
login information, he will know where to redirect
specify the access your app need.
When i've begun to watch this, i've discovered satellizer. I think that's a good entry point to understand how it works with an example :
https://github.com/sahat/satellizer
Hope this help.
Supporting login through their website (Facebook,Twitter,Instagram etc) allows a user to use his already active profile to signup to a portal or an app. This way a user experience is enhanced and having to have separate login credentials for different apps and sites are reduced.
A developer utilizes the APIs or SDKs that these social networks expose to let you integrate it in your app and provide an easy sign up process to your users.
I need to add a functionality to my java-based web application that will allow users to click on a link and the application will automatically call the user and another party and connect them in a phone call.
Does anybody know what would this entail?
Thanks
It can be done with Twilio, and their new, easy Conferencing API. Trust me, it's really really simple. Another option might be CloudVox, but I haven't (formally) tried their service yet.
The World-Wide Web Consortium has an integrated set of speech interaction standards that you'll find interesting. There's a markup language called VoiceXML that is analogous to HTML in that web applications generate it. It differs from HTML in that it's specialized for temporally-based speech interactions instead of visual interactions. So instead of looking at a screen you listen to audio prompts and computer-generated speech. Instead of typing and mousing, you speak back and what you say is processed by a speech recognizer or recorded.
There are many companies using VoiceXML to automate voice response systems, and they handle billions of calls per year. You've probably talked to them many times without realizing it. One of the best companies in this space is Voxeo, and they have a developer site at http://evolution.voxeo.com/ that you can play with. Evolution lets you call your web application over an ordinary phone (or Skype). You actually talk to a VoiceXML-based web browser which will fetch a VoiceXML page from your Java application server, "play" it to you, listen to what you say, and then report that back to your app via a form submission, get the next page to render to you, etc.
Another related standard is CCXML, or Call Control XML. You use this to create teleconferences that may or may not include a voice response application.
So it sounds like in your case you want your standard web application to talk to a CCXML server and ask it establish call legs to the web user and to a customer service line. I know that Voxeo Evolution offers CCXML as well.
There are other good companies in this space too. One that comes to mind is TellMe, which was bought by Microsoft a year or two ago. These two companies (and others) offer professional services too.
So I wanted to write this up as an answer to the comment above. The Skype API provides a number of options for telephony in COM, Java and Python:
Skype4Java - https://developer.skype.com/wiki/Java_API
Skype4Py - https://developer.skype.com/wiki/Skype4Py
Skype4COM - https://developer.skype.com/Docs/Skype4COM
They provide a communication and command protocol layer for working with Skype, more info on the API here:
https://developer.skype.com/Docs/ApiDoc/Overview_of_the_Skype_API
It's kind of different for every platform, the Linux version is based on DBus or X11.
Try FreeSWITCH. I have done this before. Its pretty straight forward. Can be a bit hairy when you need to log call accounting and all those stuff. I hopefully would be able to provide you some guidelines and code samples, let me get home first. Cheers.
The good thing in using FreeSWITCH, you will be able to handle multiple calls, and quite a number of. You might need that in future.
Note: You have to use some kind of VoIP provider in order to do that. I was using Gizmo5 that time and it was pretty good.
Sorry buddy, lost the servlet code somewhere. But no worries it was a simple servlet. Fortunately, I had added my example Java code for XML-RPC, into the FreeSWITCH wiki, and actually that was the code my servlet was invoking down the road. Below is the snippet.
XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();
XmlRpcClient client = new XmlRpcClient();
try {
config.setServerURL(new URL("http://localhost:8080/RPC2"));
config.setBasicUserName("freeswitch");
config.setBasicPassword("works");
client.setConfig(config);
// For external phone calls using VoIP. We will use something like below.
// new Object[]{"originate", "sofia/gateway/gizmo1/6098989898 &bridge(sofia/gateway/gizmo9/0116054545454)"}
// gizmo1, and gizmo9 are the accounts configured under freeswitch gateway configuration.
client.execute("freeswitch.api", new Object[]{"originate", "sofia/internal/1001 &park()"});
} catch (Exception ex) {
ex.printStackTrace();
}
Moreover, you need to configure few things prior doing this. You need to set up the gateway using your VoIP provider settings.
For FreeSWITCH related help, take a look at this SO Thread.
I know of 2 API providers that does what you need:
1) twilio - can connect to 2 or more parties using TwiML (their markup). example
2) Hoiio - very easy to use with 1 line of RESTful api call. example