Can I open several Mongo connections simultaneously from the same thread, or is it preferable to open just one connection and use it for all my calls?
Java MongoDB Driver maintains an internal connection pool, you should get a connection from MongoClient and then close it once you are done with it. MongoClient is thread safe, so you can use it easily in multi-threaded environment.
Quick sample code shown below.
MongoClient mongo = new MongoClient("localhost", 27017);
DB db = mongo.getDB("test");
DBCollection col = db.getCollection("users");
//Some operations
//close resources
mongo.close();
}
If you look at the documentation, you'll see that you'll be using a connection pool whether you want it or not, with a default maximum size of 100.
So to answer your question: use a single MongoClient across all threads, but bear in mind that this does not equate to a single MongoDB connection.
Related
I want to create a "X" number of connections to mongodb on my application startup (i.e before my application starts taking traffic).
MongoDB Version: 4.0.11
Mongo Java Driver Version (maven): 3.4.1
I have tried setting the "minConnectionsPerHost" to the required number, but when i execute the code it barely open 1 or 2 connections. But, when i put load on my application the connection count is slowly going up to accommodate the traffic. I want to create those connections before my application starts taking traffic.
ServerAddress address = new ServerAddress("localhost", 27017);
List<ServerAddress> serverAddresses = Arrays.asList(address);
MongoCredential credential =
MongoCredential.createCredential("XXXX", "XXXX",
"XXXX".toCharArray());
List<MongoCredential> mongoCredentials =
Arrays.asList(credential);
MongoClientOptions clientOptions =
MongoClientOptions.builder().connectionsPerHost(100).
minConnectionsPerHost(50).build();
MongoClient mongoClient = new MongoClient(serverAddresses,
mongoCredentials, clientOptions);
Is there a way to achieve this using the mongo java driver?
You can set a minConnectionsPerHost() in the options builder, and then use a warmup script to create many connections. The connection pool will keep the minConnectionsPerHost connections alive without closing.
The warmup script can have a program which spawns 2*minConnectionsPerHost number of threads, which will connect and do may be a dummy read operation. This way connections will be opened, minimum connections will be kept alive.
This seems to be dirty solution :-) But might work!
I'm working with the java Mongo driver 3.2 and I'm trying to figure out exactly what I should have multiple instances of and what should be persisted across the app.
So, for example, MongoClient says that there should be only one instance for the app. It also says that it establishes a connection pool. Which is great. When are those connections actually made? I want to make sure I'm effectively using the pool.
So is the connection to the DB made when I do MongoClient#getDataBase? MongoDatabase#getCollection? Or some time when I perform an action on the collection itself, such as .find?
What should be saved off? Should I have multiple instances of the Database? the Collection? or is it just the client?
Mainly I'm trying to make sure that I don't limit myself to one connection but that I also don't spam the database unnecessarily.
Once instance should be fine per application. MongoDB Driver Quick Tour - Java seems clear enough. EDIT: My understanding of the writing is that when you call MongoClient.getDB(); is when you are connecting, to be sure parsing through their source code if available to see the exact moment. EDIT 2: Added links to MongoDB Java Driver which is here MongoDB Driver and particularly the class your checking for is MongoClient which extends Mongo.class which is where the .getDB(); is located; in this method they check the DB Cache and if not they create a new DB.class Instance. After running through the code it appears in the Mongo.class constructor so when you call new MongoClient(); they call connector.start()
public Mongo( ServerAddress addr , MongoOptions options )
throws MongoException {
_addr = addr;
_addrs = null;
_options = options;
_applyMongoOptions();
_connector = new DBTCPConnector( this , _addr );
_connector.start();
_cleaner = new DBCleanerThread();
_cleaner.start();
}
From their page
// To directly connect to a single MongoDB server (note that this will not auto-discover the primary even
// if it's a member of a replica set:
MongoClient mongoClient = new MongoClient();
// or
MongoClient mongoClient = new MongoClient( "localhost" );
// or
MongoClient mongoClient = new MongoClient( "localhost" , 27017 );
// or, to connect to a replica set, with auto-discovery of the primary, supply a seed list of members
MongoClient mongoClient = new MongoClient(Arrays.asList(new ServerAddress("localhost", 27017),
new ServerAddress("localhost", 27018),
new ServerAddress("localhost", 27019)));
DB db = mongoClient.getDB( "mydb" );
At this point, the db object will be a connection to a MongoDB server
for the specified database. With it, you can do further operations.
The MongoClient class is designed to be thread safe and shared among
threads. Typically you create only 1 instance for a given database
cluster and use it across your application.
important
When creating many MongoClient instances:
All resource usage limits (max connections, etc) apply per MongoClient instance
To dispose of an instance, make sure you call MongoClient.close() to clean up resources
Hey first time using mongo replica sets, and I've configured a set of 3 servers in Java. I'm trying to determine which secondary is being read from by my service (using secondary preferred). Does the mongo driver expose access to this info within Java?
As I'm struggling to find a reliable way to track it. I'm also aware once a connection is made to a secondary it is unlikely to switch to read from the other secondary unless there is failure.
You can use the getAdress method like below, see the documentation for more details.
MongoClient mongoClient = new MongoClient("localhost", 27017);
ServerAddress serverAdress = mongoClient.getAddress()
I run a java program which is a thread executor program that inserts thousands of documents to a table in mongodb. I get the following error
Exception in thread "pool-1-thread-301" com.mongodb.MongoWaitQueueFullException: Too many threads are already waiting for a connection. Max number of threads (maxWaitQueueSize) of 500 has been exceeded.
at com.mongodb.PooledConnectionProvider.get(PooledConnectionProvider.java:70)
at com.mongodb.DefaultServer.getConnection(DefaultServer.java:73)
at com.mongodb.BaseCluster$WrappedServer.getConnection(BaseCluster.java:221)
at com.mongodb.DBTCPConnector$MyPort.getConnection(DBTCPConnector.java:508)
at com.mongodb.DBTCPConnector$MyPort.get(DBTCPConnector.java:456)
at com.mongodb.DBTCPConnector.getPrimaryPort(DBTCPConnector.java:414)
at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:176)
at com.mongodb.DBCollectionImpl.insert(DBCollectionImpl.java:159)
at com.mongodb.DBCollection.insert(DBCollection.java:93)
at com.mongodb.DBCollection.insert(DBCollection.java:78)
at com.mongodb.DBCollection.insert(DBCollection.java:120)
at ScrapResults103$MyRunnable.run(MyProgram.java:368)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
at java.lang.Thread.run(Thread.java:695)
How can I resolve this? Please help me.
You need to check what is the connections per host value which you have given while setting up connection (looking at the exception I think you would have set it to 500).
MongoClientOptions.Builder builder = new MongoClientOptions.Builder();
builder.connectionsPerHost(200);
MongoClientOptions options = builder.build();
mongoClient = new MongoClient(URI, connectionOptions);
An ideal way of setting the connections per host would be by trial and error but you need to make sure that the value which you set should not exceed the number of connections you can have by opening the mongo shell and executing:
db.serverStatus().connections.available
you are in maxWaitQueueSize limit , so increase multiplier ;)
MongoClientOptions options = MongoClientOptions.builder()
.threadsAllowedToBlockForConnectionMultiplier(10)
.build();
MongoClient mongo = new MongoClient("127.0.0.1:27017", options);
//run 2000 threads and use database ;)
waitQueueMultiple is the product of maxConnectionPoolSize and threadsAllowedToBlockForConnectionMultiplier hence you can modify one of these three options to tune your app in MongoClientOptions with corresponding values and consume it to your MongoClient as an argument how it was done above here (marked as an answer) https://stackoverflow.com/a/25347310/2852528
BUT
I strongly recommend analysing your code first (where it communicates to the DB), and if no optimization is available (e.g. caching, using aggregation, paging etc.) then go ahead and change the options
I have created a mysqlDatasource connection using the following code:
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
If Im running my application and the connections are disconnected because mysql restarted or for some other reason, the remaining operations will fail even if the mysql server instance is running.
In that case I want to recreate a connection? Is this possible? How do I go about doing this?
In most cases when you're creating multiple connections and juggling them, it's better to use a connection pool, where the Connection objects are just that, objects, and are multiplexed to actual socket connections handled by an underlying implementation. This means that connections are created automatically when you need them and you don't need to worry about reclaiming resources and creating an appropriate number of connections.
Two prominent examples are BoneCP and C3P0.
The other option, in addition to Mr Mao's, is to define the url explicitly using setURL to allow for automatic reconnection, using the autoReconnect parameter, do note, though, that this approach is not recommended. The answer is provided here only for completeness.
Just take an help of DBCP CONNECTION and create your Connection pool like below code. You can see there is validation query which pings database at regular interval and refreshes the pool. There are other property like validateConnection you can set this property to true.
BasicDataSource dataSource = new BasicDataSource();
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUsername("username");
dataSource.setPassword("password");
dataSource.setUrl("jdbc:mysql://<host>:<port>/<database>");
dataSource.setMaxActive(50);
dataSource.setMaxIdle(5);
dataSource.setInitialSize(5);
dataSource.setValidationQuery("SELECT 1");
Try this.
String driver = "com.microsoft.jdbc.sqlserver.SQLServerDriver";
Class.forName(driver);
String url = "jdbc:microsoft:sqlserver://host:1433/database";
Connection conn = DriverManager.getConnection(url, "username", "password");