Develop Reference

How to fix Ldap connection error "javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]" in Java?

There are 7563 Employees in AD. I need to pick them all using (employeeId=*)
Following is my Java code for the same:
Hashtable<String, String> ldapEnv = new Hashtable<>();
ldapEnv.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.ldap.LdapCtxFactory");
ldapEnv.put(Context.PROVIDER_URL, PROVIDER_URL);
ldapEnv.put("com.sun.jndi.Ldap.connect.timeout", CONNECTION_TIMEOUT); // specify timeout out to be 5 seconds
ldapEnv.put(Context.SECURITY_AUTHENTICATION, AUTHENTICATION_TYPE);
ldapEnv.put(Context.SECURITY_PRINCIPAL, AD_USER);
ldapEnv.put(Context.SECURITY_CREDENTIALS, AD_PASS);
DirContext context = new InitialDirContext(ldapEnv); // initial context created
System.out.println("(employeeID=" + aDemployeeID + ")");
String searchFilter = "(employeeID=*)";
SearchControls controls = new SearchControls();
String[] requiredAttribute = { "mail", "Targetaddress", "UserPrincipalName" };
controls.setSearchScope(SearchControls.SUBTREE_SCOPE);
controls.setReturningAttributes(requiredAttribute);
controls.setCountLimit(1200);
NamingEnumeration<SearchResult> users1 = context.search(SEARCH_DIRECTORY_1, searchFilter, controls);
adAttributesToEVRYIntegrator = new HashMap<>();
int count = 0;
while (users1.hasMore()) {
SearchResult sr = users1.next();
NamingEnumeration<? extends Attribute> aDattributes = sr.getAttributes().getAll();
count = count + 1;
while (aDattributes.hasMore()) {
Attribute requestedAttributesFromAD = aDattributes.next();
System.out.println(requestedAttributesFromAD.get().toString());
System.out.println(count);
}
Output:
After 1000 Records:
javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; remaining name 'OU=Users,OU=Sweden,OU=CCD,OU=Customers,DC=corp,DC=corpcommon,DC=com'
I have checked in multiple folder and i found that:
By Default, LDAP is returning 1000 Records.
Could anyone suggest how to get all records from LDAP
In Single Go
In Batches (as i do not have employee Id's with me so how to check that records already came does not come again when using *)
PS: I have done all changes in related posts but not able to fix the problem.

You need to use PagedResultsControl. You will have to change the declaration of context so it's an LdapContext, and you can pass the PagedResultsControl in the constructor of InitialLdapContext:
int pageSize = 1000;
LdapContext context = new InitialLdapContext(ldapEnv, new Control[] {
new PagedResultsControl(pageSize, Control.CRITICAL) });
The pageSize can be anything <= 1000.

It appears that you are encountering the Microsoft Active Directory MaxPageSize setting which controls the maximum number of objects that are returned in a single search result.
You can not change the MaxPageSize from the client.
The recommendation from Microsoft is to use the Simple Paged Results Control which will work.
I did notice that you have:
controls.setCountLimit(1200);
Which set the maximum number of entries to be returned as a result on the Client.

Accessing data in an object array with Java

I'm currently working with Cucumber and Java. I would like to retrieve that path of a file from ITestResult.
I'm currently retrieving the parameters with:
Object[] test = testResult.getParameters();
However the only thing I can access would seem the be the first objects name and nothing else.
test = {Object[1]#1492}
0 = {CucumberFeatureWrapper#1493} "Links at EDM Documents View,"
cucumberFeature = {CucumberFeature#1516}
path = "test/01-automation.feature"
feature = {Feature#1518}
cucumberBackground = null
currentStepContainer = {CucumberScenario#1519}
cucumberTagStatements = {ArrayList#1520} size = 1
i18n = {I18n#1521}
currentScenarioOutline = null
I cannot see anyway of retrieving path = "test/01-automation.feature" under cucumber feature.

Have you tried something like ((CucumberFeatureWrapper)test[0]).getCucumberFeature().getPath()?

Java ini4j - reading multiple options from .ini file

I'm trying to read multiple values using ini4j, the docs say it should be possible with the Options class.
Here is my sample .ini file (./dwarfs.ini)
[dopey]
age = 23
fortuneNumber = 11
fortuneNumber = 33
fortuneNumber = 55
here is the code to read it:
Ini ini = new Ini();
Config conf = new Config();
conf.setMultiOption(true);
ini.setConfig(conf);
ini.load(new FileReader("./dwarfs.ini"));
but fortuneNumber property is just 55 after reading and I'd want it to be an array or a list, anything.

The web presence for the ini4j project contains (among others) very simple tutorials. One of these tutorials explains, how to retrieve multiple values. How could you expect to get a list or an array, when using a fetch method that returns a single reference? Look at the API!
In the tutorial, there is a part explaining multi values:
[ini4j] library introduces MultiMap interface, which is extends normal
Map, but allows multiply values per keys. You can simply index values
for a given key, similar to indexed properties in JavaBeans api.
There is also an example:
String n1 = sneezy.get("fortuneNumber", 0); // = 11
String n2 = sneezy.get("fortuneNumber", 1); // = 22
String n3 = sneezy.get("fortuneNumber", 2); // = 33
String n4 = sneezy.get("fortuneNumber", 3); // = 44
In this example, sneezy is a Section, but it should also work with an Ini.
And just to make it complete: An Ini also knows a method List<V> getAll(Object key).

to handle mutliple properties in section use following code :
Ini oINI = new Wini();
Config conf = new Config();
conf.setMultiOption(true);
oINI.setConfig(conf);
oINI.load(new File("....../myfile.ini"));
Do not directly open the INI file in the class creation, set options before, otherwise the options won't be used and by default MultiOption si set to "false".
Have searched a while for this solution.

you need setConfig first, and then load file.
you need use List to handle multiple same values.
here the example:
Ini ini = new Wini();
Config config = new Config();
config.setMultiOption(true);
config.setMultiSection(true);
ini.setConfig(config); //set config
ini.load(new File(filename)); // load AFTER setConfig
for (String sectionName : ini.keySet()) {
List<Section> sectionList = ini.getAll(sectionName); // use List
for (Section section : sectionList) {
for (String optionName : section.keySet()) {
List<String> optionList = section.getAll(optionName);
for (String optionVaule : optionList) {
System.out.printf("%s %s %s\n", sectionName, optionName, optionVaule);
}
}
}
}
here the out put:
dopey age 23
dopey fortuneNumber 11
dopey fortuneNumber 33
dopey fortuneNumber 55

Check if table exists

What is the fastest way to check if Hbase table exists? Looking at this api :
http://hbase.apache.org/devapidocs/org/apache/hadoop/hbase/client/HBaseAdmin.html
Which of these is the fastest :
tableExists
isTableEnabled
isTableAvailable
listTables
With #4 you get list of all tables and iterate trough it and compare if one of those tables matches your table name.
Or there is another, more smart way ?

Here is my sample code. (scala)
import org.apache.hadoop.hbase.HBaseConfiguration
var TableName = "sample"
val conf = HBaseConfiguration.create()
var hbaseAdmin = new HBaseAdmin(conf)
if (!hbaseAdmin.tableExists(TableName)) {
println(TableName + " Does Not Exist")
}
Here, you just need to use "tableExists" to check whether this TableName exists.

HBaseAdmin hba = new HBaseAdmin(hbaseTemplate.getConfiguration());
if (hba.tableExists(tableName) == false) {
HTableDescriptor tableDescriptor = new HTableDescriptor(tableName);
HColumnDescriptor columnDescriptor = new HColumnDescriptor(columnFamilyProfile);
tableDescriptor.addFamily(columnDescriptor);
hba.createTable(tableDescriptor);
}

Using HBaseAdmin.tableExists only takes about 500ms to check if the table exists. We only have two nodes in our cluster, so it might be dependent on the size of your cluster, but it doesn't seem unreasonably slow.

You could attempt to open an HTable to the table and (I think) it will throw an exception/error (not at work yet so can't do a quick test) if the table doesn't exist.
Not 100% this will work, just an off the top of the head idea. :)

I have to check if table exist every time i start my app. I have made this in a configuration class, with spring boot
Here is the code, hope it helps.
#Configuration
public class CustomHbaseConfiguration {
#Bean
public Connection hbaseConnection() throws IOException {
// Create connection
final org.apache.hadoop.conf.Configuration configuration = HBaseConfiguration.create();
// Validate that Hbase is available
HBaseAdmin.available(configuration);
// return the hbaseConnection Bean
return ConnectionFactory.createConnection(configuration);
}
#PostConstruct
public void hbaseTableLogic() throws IOException {
// With the hbaseConnection bean, get the HbaseAdmin instance
Admin admin = hbaseConnection().getAdmin();
// The name of my table
TableName YOUR_TABLE_NAME_HERE = TableName.valueOf("PUT_YOUR_TABLE_NAME_HERE");
// Check if the table already exists ? else : create table and colum family
if (!admin.tableExists(YOUR_TABLE_NAME_HERE)) {
HTableDescriptor hTableDescriptor = new HTableDescriptor(YOUR_TABLE_NAME_HERE);
hTableDescriptor.addFamily(new HColumnDescriptor("PUT_YOUR_COLUM_FAMILY_HERE"));
admin.createTable(hTableDescriptor);
}
}
}

problem in determining whether second level cache is working in hibernate

I am trying to use ehcache in my project.. i have specified the following properties in hibernate config file -
config.setProperty("hibernate.cache.provider_class","org.hibernate.cache.EhCacheProvider");
config.setProperty("hibernate.cache.provider_configuration_file_resource_path","ehcache.xml");
config.setProperty("hibernate.cache.use_second_level_cache","true");
config.setProperty("hibernate.cache.use_query_cache","true");
Now i am still not sure whether the results are coming from DB or the cache..
I looked around and found - Hibernate second level cache - print result where the person is suggesting HitCount/Misscount API's
However when i tried using it the hitcount and miss count is always returned 0... here's my code
String rName = "org.hibernate.cache.UpdateTimestampsCache";
Statistics stat =
HibernateHelper.getInstance().getFactory().getStatistics();
long oldMissCount =
stat.getSecondLevelCacheStatistics(rName).getMissCount();
long oldHitCount =
stat.getSecondLevelCacheStatistics(rName).getHitCount();
UserDAO user = new UserDAO();
user.read(new Long(1)); long
newMissCount =
stat.getSecondLevelCacheStatistics(rName).getMissCount();
long newHitCount =
stat.getSecondLevelCacheStatistics(rName).getHitCount();
if(oldHitCount+1 == newHitCount &&
oldMissCount+1 == newMissCount) {
System.out.println("came from DB"); }
else if(oldHitCount+1 == newHitCount
&& oldMissCount == newMissCount) {
System.out.println("came from cache");
}
Please let me know if i am using it wrong.. and what should be the rName(region Name) in this case..
Is there any other way of determining whether the second level cache is working ??
Thanks

You need to enable statistics collection:
config.setProperty("hibernate.generate_statistics", "true");