I wish to use OSM (or an open, OSM based service) in order to "create" routes between aleatory points and then sample those routes an unlimited amount of time. Sample as in get lan+lat and various other info if available (e.g. elevation, points of interest... etc).
I've been struggling to find similar projects, or documentation that might help me. Right now I am quite unsure even on the subject of how to download part of the OSM schema locally so that I don't have to use the API over the web and spam the OSM servers.
Most resources that I've been able to find online are, sadly, enough, sparsely documented and often unmaintained :/
If I were to split what I need to learn into 3 parts those would be:
a) Get the OSM schema for a certain "region" of the world downloaded and running locally and connect to & control it.
b) Figure out how to create an entity around the lines of "route" between two points (say addresses in a city).
c) Figure out how to query said entities for various samples along the route.
This needn't be done in OSM, if there is a product better suited for this, but I have to use something open and OSM seems to be by far the biggest most well maintained project.
(I should note I am building the app in scala, but I'm fine with documentation for other languages/language agnostic, as well as it actually explains stuff and goes into details, instead of just throwing some incomplete lines of code at you)
Re 1: Grab OSM from http://planet.osm.org/ and extract your area via osmcut, osmosis, osmconvert etc. Or use preprocessed extracts e.g. from geofabrik
Re 2: Street names are simple, everything else is complex. You'll need another preprocessing step which gives you the boundaries or some guesses of every city/village and then feed the routing engine with this.
Re 3: Once you have this stored in the routing engine this should be rather simple to query and return.
OSM seems to be by far the biggest most well maintained project.
Maybe here is a misunderstanding: OpenStreetMap is a database and so it is just data. Now it happens to be an ecosystem for various tools like tools for routing, see the tagged GraphHopper routing engine here and others here
Related
I am developing a proof of concept/prototype for an internal tool. The basic premise is, that we have a couple of units that have a GPS on them, and they report their location (with typical GPS accuracy in NMEA format) to a java api I have built in a cloud environment.
[Device GPS] -> [Sends Cellular GSM request to API] -> [API queries Google
Roads] -> [Google Roads returns snapped coordinates] -> [App maps received
coordinates]
The java API takes the reported coordinates, and essentially sends a request to the Google Maps Roads API and uses the Snap-to-road feature (this will essentially take whatever point you gave it along with an optionally traveled path of coordinates, and give you coordinates back that are essentially on the path it believes you are traveling). This is great and all, but with one setback.
IT IS EXPENSIVE.
For two devices reporting their location (similar to the way Uber or Lyft may do it) every few seconds, the cost of running this application essentially makes it a sunk cost. 24,000 queries will cost you $300-500. For one device.
We have taken a few efforts to only accept requests to the API during work hours - but that is still barely scratching the surface.
I am fairly capable at writing whatever I would need (I believe at least), or can find libraries to do what I think needs to be done - but I have no idea what recourse to take. I am thinking of something along the lines of caching requests in another table and querying that first, but I am not certain how reliable that will be.
Has anyone implemented such a solution or something similar? I just need to defray the amount of requests I send to Google Maps' API before I bankrupt this project.
If you have not already I would think you would want to look at the Google Maps API Pricing Sheet ( https://cloud.google.com/maps-platform/pricing/sheet/? ) and talk to Google. They should be able to explain to you the correct architectural approach (caching parts of maps locally, local path processing etc.).
At https://en.wikipedia.org/wiki/Google_Maps#Google_Maps_API they explain the pricing changes and other useful details, but the section also mentions alternatives such as OpenLayers and local map hosting if your use case would allow for that.
It may helpful to better understand the scope of the devices' travel. Are the devices to be put on vehicles that only travel around a factory, or on a personal device that a user can take anywhere in the world?
What i'm trying to do is to integrate anomaly detection module in the existing java application to allow user to chose from different algorithms and forecasting models
Egads library looks quite optimistic, but I'm not sure whether it fits my purposes, in case new data coming in should I store and update the existing model or to pass the whole data once again. Also what if I would like to forecast only 15 min time window, by passing only 15 min data in the results won't be precise for sure.
Might be there're any other useful techniques and someone could share his experience of similar tasks. Unfortunately can't find any other java libs for that purposes.
What I found out is that we can't store the model trained initially and apply it against any incoming data, as soon as the initial time series is changed the exception is thrown. That's why the only possible option here to to train the model every time new data comes in, fortunately it doesn't have great performance impact on our system yet.
Library itself looks fine and could be used as a base in building anomaly detection systems, but its still not that flexible as it Python competitors, however it's open sourced and can be modified anytime depending on your needs.
I am planing to write a Java application to identify certain types of trees using google maps . I have knoweldge in google maps api. my problem is how to recognize trees from map. I found some libs like jjil. But i don't know whether it is useful or not. Can any one give some inputs for this project ?
The general idea would probably be something like this:
Collect a large number of positive and negative samples (i.e. images from Google Maps that you want to recognize and ones that you don't want to recognize). It's hard to say how many you'll really need; depending on how similar they are and how many features you need, 100 might be enough or 10000 might be too few.
Find a set of texture features. Nobody can tell you what features are optimal without seeing the sample images (1) first.
Train a machine learning algorithm (e.g. SVM, neural network). Split the sample set into a training set and a test set to judge the discrimination quality.
Step 1 is just work, and I doubt anyone on Stackoverflow will do it for you. If you have the samples and post them, we might be able to help with steps 2/3, though.
I'm trying to implement part of the facebook ads api, the auto complete function ads.getAutoCompleteData
Basically, Facebook supplies this 39MB file which updated weekly, and which contains targeting ads data including colleges, college majors, workplaces, locales, countries, regions and cities.
Our application needs to access all of those objects and supply auto completion using this file's data.
I'm thinking of preferred ways to solved this. I was thinking about one of the following options:
Loading it to memory using Trie (Patricia-trie), the disadvantage of course that it will take too much memory on the server.
Using a dedicated search platform such as Solr on a different machine, the disadvantage is perhaps over-engineering (Though the file size will probably increase largely in the future).
(Fill here cool, easy and speed of light option) ?
Well, what do you think?
I would stick with a service oriented architecture (especially if the product is supposed to handle high volumes) and go with Solr. That being said, 39 MB is not a lot of hold in memory if it's going to be a singleton. With indexes and all this will get up to what? 400MB? This of course depends on what your product does and what kind of hardware you wish to run it on.
I would go with Solr or write your own service that reads the file into a fast DB like MySQL's MyISAM table (or even in-memory table) and use mysql's text search feature to serve up results. Barring that I would try to use Solr as a service.
The benefit of writing my own service is that I know what is going on, the down side is that it'll be no where as powerful as Solr. However I suspect writing my own service will take less time to implement.
Consider writing your own service that serves up request in a async manner (if your product is a website then using ajax). The trouble with Solr or Lucene is that if you get stuck, there is not a lot of help out there.
Just my 2 cents.
Looking for an all in one solution that I can distribute within my Java application to store a small amount of data as records. This data can change over time and will be updated frequently. It stores book information - author, genre, title, rating etc.
I would like to have something that will allow me to look into full text searching on a particular table, searching is something that will be crucial.
The only real requirement outside of the searching abilities is minimal setup, I dont expect MySQL etc to be running as a service, so the solution needs minimal attention form the end user.
So far I have been looking at SQLLITE.
As for the searching its totally new to me, one of the reasons I am doing this project is so I can learn about it.
Any feedback is appreciated.
You can look at HSQLDB that is the closest thing to sqlite, and is very simple (there are three storage: in memory, in a local file, in a distant server).
It can works with apache Lucene to get a full text search.