Develop Reference

Analyzing heap dump from OutOfMemoryError in Java

my Java program is constantly getting OutOfMemoryError, and I believe there is a memory leak somewhere. While researching this issue, multiple sites suggested the Eclipse Memory Analyzer tool, so I added the -XX:+HeapDumpOnOutOfMemoryError flag to the command, to get the heap dump the next time the error occurs. Upon checking the dump, the objects taking up the most space were "17,481 instances of "com.couchbase.client.core.deps.org.LatencyUtils.LatencyStats", loaded by "org.springframework.boot.loader.LaunchedURLClassLoader # 0x6c7c24510" occupy 1,978,652,856 (59.03%) bytes."
I thought this was the logger printing out too many logs, since the Java Couchbase code prints a LOT of logs on the INFO level, so I tried setting the log level to WARN but after trying it out, same result. Would appreciate any insight or suggestions, thank you.
EDIT: some parts of our code that calls Couchbase:
#Autowired
private CouchbaseConfig couchbaseConfig;
public List<ArLedger> getBranchArLedgers(String branchId, String fromDate, String toDate) {
String query = Queries.GET_AR_LEDGER_BY_BRANCH_AND_DATE_RANGE;
query = MessageFormat.format(query, branchId, fromDate, toDate);
Cluster cluster = null;
try {
cluster = couchbaseConfig.connectToCouchbase();
QueryResult queryResult = cluster.query(query);
return queryResult.rowsAs(ArLedger.class);
} catch (Exception e) {
e.printStackTrace();
return Collections.emptyList();
} finally {
if (cluster != null) {
cluster.disconnect();
}
}
}
And the connectToCouchbase() from the injected CouchbaseConfig:
#Value("${app.couchbase.connection-string}")
private String connectionString;
#Value("${app.couchbase.username}")
private String username;
#Value("${app.couchbase.password}")
private String password;
public Cluster connectToCouchbase() {
return Cluster.connect(connectionString, username, password);
}
EDIT 2: Updated the code to follow dnault's suggestion, and a screenshot of the error that occurs when running the code:
CouchbaseConfig:
#Configuration
public class CouchbaseConfig extends AbstractCouchbaseConfiguration {
#Autowired
private ApplicationContext context;
#Value("${app.couchbase.connection-string}")
private String connectionString;
#Value("${app.couchbase.username}")
private String username;
#Value("${app.couchbase.password}")
private String password;
#Bean
public Cluster couchbaseCluster() {
return Cluster.connect(connectionString, username, password);
}
}
The repository code:
#Repository
public class ArLedgerRepository {
#Autowired
private Cluster couchbaseCluster;
public List<ArLedger> getAllBranchArLedgers(String branchId, String fromDate, String toDate) {
String query = Queries.GET_ALL_AR_LEDGERS_BY_BRANCH_AND_DATE_RANGE;
query = MessageFormat.format(query, branchId, fromDate, toDate);
try {
QueryResult queryResult = couchbaseCluster.query(query);
return queryResult.rowsAs(ArLedger.class);
} catch (Exception e) {
e.printStackTrace();
return Collections.emptyList();
} finally {
couchbaseCluster.disconnect();
}
}
}
And the screenshot of the error that occurs when the repository method is called:

#kei101895
There is already a couchbaseCluster bean defined in AbstractCouchbaseConfiguration. If I'm not mistaken, that is the Cluster that #Autowired will use (I believe because it was needed previously by other #Beans and already created).
That couchbaseCluster uses the couchbaseClusterEnvironment bean which has a destroyMethod specified. This will ensure that shutdown() is called on the ClusterEnvironment
#Bean(destroyMethod = "shutdown")
public ClusterEnvironment couchbaseClusterEnvironment() {...
To customize the environment for the provided Cluster #Bean, one can #Override the configureEnvironment(builder) method in the couchbase config class.
If you really want/need to have your own Cluster bean, you can give it a name in #Bean("myBeanName") and then reference it with:
ApplicationContext ac = new AnnotationConfigApplicationContext(Config.class);
myCluster = (Cluster) ac.getBean("myBeanName");

Downcasting a CompletableFuture's object type - Java/Spring Boot

I have the following classes:
public class AccountDetail {
private String accountNumber;
private Date effectiveDate;
private String status;
// a bunch of other properties
}
public class AccountDetailWithAlerts extends AccountDetail {
private LowMediumAlerts alerts;
}
public class AccountsAndAlerts {
private List<AccountDetailWithAlerts> accounts;
private HighCustomerAccountAlerts accountAlerts;
// getters and setters
}
public class CustomerAndAccountAlerts {
private List<AlertMessage> customerAlerts;
private List<AccountAlertMessages> accountAlerts;
}
public Class CompanyResponse<T> {
#JsonInclude(JsonInclude.Include.NON_NULL)
private T response;
// other things that aren't relevant
}
I have a controller, AccountsController, that does a #GetMapping and has a ResponseEntity method:
public ResponseEntity<CompanyResponse<AccountsAndAlerts> getAccountDetails {
#RequestParam MultiValueMap<String, String> queryParms,
// some #ApiParams for client-header, end-user-id & accountNumber
String accountId = queryParms.getFirst("accountId");
// setting RestHeaders, contentType
CompanyResponse<AccountsAndAlerts> response = accountDetailService.getAccountsWithAlerts(restHeaders, accountNumber, queryParms, accountId);
return new ResponseEntity<CompanyResponse<AccountsAndAlerts>>(response, headers, HttpStatus.valueOf(response.getStatus()));
}
Here is the method in accountDetailService:
public CompanyResponse<AccountsAndAlerts> getAccountsWithAlerts(RestHeaders restHeaders, String accountNumber, MultiValueMap<String, String> queryParms, String accountId) throws... {
CompanyResponse<AccountsAndAlerts> newResponse = new CompanyResponse<AccountsAndAlerts>();
try {
CompletableFuture<List<AccountDetailWithAlerts>> accountsFuture = accountDetails.getAccounts(newResponse, restHeaders, accountNumber, queryParms);
CompletableFuture<CustomerAndAccountAlerts> alertsFuture = accountDetails.getAlerts(newResponse, restHeaders, accountId);
accountsFuture.thenAcceptBoth(alertsFuture, (s1, s2) -> newResponse.setResponse(getResponse(s1, s2))).get();
} catch {
// catch code
}
return newResponse;
}
Finally, the getAccounts method in AccountDetails:
public CompletableFuture<List<AccountDetailWithAlerts>> getAccounts(CompanyResponse<AccountsAndAlerts> newResponse, RestHeaders restHeaders, String accountNumber, MultiValueMap<String, String> queryParms) throws ... {
// this has the restTemplate and the .supplyAsync()
}
What I need to do is create a new ResponseEntityMethod in the Controller:
public ResponseEntity<CompanyResponse<AccountDetail> getCertainAccountDetails
I have put in a return of that type, and I am attempting to create a new method in the accountDetailService, getCertainAccounts().
The problem is trying to set this all up without creating a whole other CompletableFuture method with an invoke and supplyAsync() and restTemplate and such.
It appears that I still need to call getAccounts(), but then I have to somewhere along this line downcast the AccountDetailWithMessages to AccountDetail. I don't know if I can somehow downcast CompletableFuture<List<AccountDetailWithAlerts>> to CompletableFuture<List<AccountDetail>> or how to do it, or if I really need to downcast CompanyResponse<AccountsAndAlerts> or how to do that.
Can anyone help?
PS. I changed the names of everything to protect my Company's code. If you see errors in methods or names or anything, please be assured that is not an issue and is just the result of my typing things out instead of copying and pasting. The only issue is how to do the downcasting.
Thanks!
PPS. In case it wasn't clear, with my new method and code I do not want to get the alerts. I am trying to get account details only without alerts.

java - Autowiring Repository in class that is instantiated in Spring Boot

So I have an API client type class right now, which I am trying to connect to my repository so that I can store data in the MySQL database.
The problem I'm having is that the API client class instantiates a new object of itself, so the Autowiring doesn't work correctly. I've looked around for a workaround for this problem, and I've seen a couple options, but I'm confused on how to apply them to my problem.
For reference, here are parts of some of the relevant files:
GeniusApiClient.java:
#Component
public final class GeniusApiClient {
private final OkHttpClient client = new OkHttpClient();
#Autowired
private ArtistDao artistDao;
public static void main(String[] args) throws Exception {
GeniusApiClient geniusApiClient = new GeniusApiClient();
String artistId = (geniusApiClient.getArtistId("Ugly Duckling"));
ArrayList<String> artistSongIds = geniusApiClient.getArtistSongIds(artistId);
System.out.println(geniusApiClient.getAllSongAnnotations(artistSongIds, artistId));
}
public String getAllSongAnnotations(ArrayList<String> songIds, String artistId) {
Artist artist = new Artist("test name for now", "string123", "223");
artistDao.save(artist);
return "finished";
}
}
ArtistDao.java:
#Transactional
public interface ArtistDao extends CrudRepository<Artist, Long> {
public Artist findByGeniusId(String geniusId);
}
ArtistController.java:
#Controller
public class ArtistController {
#Autowired
private ArtistDao artistDao;
/**
* GET /create --> Create a new artist and save it in the database.
*/
#RequestMapping("/create")
#ResponseBody
public String create(String name, String annotations, String genius_id) {
String userId = "";
try {
genius_id = genius_id.replaceAll("/$", "");
Artist artist = new Artist(name, annotations, genius_id);
artistDao.save(artist);
userId = String.valueOf(artist.getId());
}
catch (Exception ex) {
return "Error creating the artist: " + ex.toString();
}
return "User succesfully created with id = " + userId;
}
/**
* GET /get-by-email --> Return the id for the user having the passed
* email.
*/
#RequestMapping("/get")
#ResponseBody
public String getByEmail(String genius_id) {
String artistId = "";
try {
Artist artist = artistDao.findByGeniusId(genius_id);
artistId = String.valueOf(artist.getId());
}
catch (Exception ex) {
return "User not found";
}
return "The user id is: " + artistId;
}
}
The problem is that in GeniusApiClient.java in the getAllSongAnnotations method, I have a null pointer exception when I try and access the artistDao. I understand that my instantiation of this class is what is messing up the Autowiring, but I'm curious on what the best way to go about fixing this might be.
I considered making all of my methods in the class static so that I wouldn't have to instantiate a new method, but I don't think this would work very well. Any suggestions?
Thanks
EDIT:
Removed some irrelevant code for clarity.
EDIT2:
Added ArtistController.java

To be able to autowire/inject an object, that object must be a Spring bean.
Here you can't autowire ArtistDao because it's not a bean. There are several annotation options to make it bean but the one suits in this case is #Repository annotation. It's just a specialized version of #Component which you used in GeniusApiClient class.
So,
#Repository
#Transactional
public interface ArtistDao extends CrudRepository<Artist, Long> {
public Artist findByGeniusId(String geniusId);
}
should work.
I'd suggest you to read: http://docs.spring.io/spring/docs/current/spring-framework-reference/html/beans.html
If reading reference documentation sounds scary to you, you can also take a look at Core Spring part of Spring in Action.

Don't make GeniusApiClient.class final. Spring will use CGLIB to dynamically extend your class in order to make a proxy. And the requirement for CGLIB to work is to have your classes non-final.
More on this here: Make Spring Service Classes Final?

What you are trying to do in your catch block is not clear to me,you have to correct that and replace it with desired action to be taken on any exception occurrence.

Spring CGLIB AopProxy interferes with Jersey resource method parameter validation

Stack is Spring Boot w/ Jetty/Jersey. Here's the resource method in question:
#GET
#Path("campaignTargets")
#Produces(MediaType.APPLICATION_JSON)
#Transactional(readOnly=true)
public List<CampaignTargetOutputDTO> getCampaignTargets(
#PathParam("businessUnitId") Integer id,
#QueryParam("name") String name,
#Pattern(regexp = DATE_VALIDATION_PATTERN) #QueryParam("startDate") String startDate,
#Pattern(regexp = DATE_VALIDATION_PATTERN) #QueryParam("endDate") String endDate,
#Pattern(regexp = INTEGER_CSV_VALIDATION_PATTERN) #QueryParam("targetTypeIds") String targetTypeIds,
#Pattern(regexp = ALPHANUM_CSV_VALIDATION_PATTERN) #QueryParam("statuses") String statuses) {
return ResourceUtil.entityOr404(campaignService.getAdvertiserCampaignTargets(id, name, startDate, endDate, targetTypeIds, statuses));
}
When Jersey intercepts the call to this method to perform the validation, it doesn't (always) get this method. The reason I know this is because I have taken the advice of the Jersey documentation and created the following ValidationConfig:
#Provider
public class ValidationConfigurationContextResolver implements
ContextResolver<ValidationConfig> {
#Context
private ResourceContext resourceContext;
#Override
public ValidationConfig getContext(Class<?> type) {
final ValidationConfig config = new ValidationConfig();
config.constraintValidatorFactory(
resourceContext.getResource(InjectingConstraintValidatorFactory.class));
config.parameterNameProvider(new CustomParameterNameProvider());
return config;
}
private static class CustomParameterNameProvider extends DefaultParameterNameProvider {
private static final Pattern PROXY_CLASS_PATTERN = Pattern.compile("(.*?)\\$\\$EnhancerBySpringCGLIB\\$\\$.*$");
public CustomParameterNameProvider() {
}
#Override
public List<String> getParameterNames(Method method) {
/*
* Since we don't have a full object here, there's no good way to tell if the method we are receiving
* is from a proxy or the resource object itself. Proxy objects have a class containing the string
* $$EnhancerBySpringCGLIB$$ followed by some random digits. These proxies don't have the same annotations
* on their method params as their targets, so they can actually interfere with this parameter naming.
*/
String className = method.getDeclaringClass().getName();
Matcher m = PROXY_CLASS_PATTERN.matcher(className);
if(m.matches()) {
try {
return getParameterNames(method.getDeclaringClass().getSuperclass().
getMethod(method.getName(), method.getParameterTypes()));
} catch (Exception e) {
return super.getParameterNames(method);
}
}
Annotation[][] annotationsByParam = method.getParameterAnnotations();
List<String> paramNames = new ArrayList<>(annotationsByParam.length);
for(Annotation[] annotations : annotationsByParam) {
String name = getParamName(annotations);
if(name == null) {
name = "arg" + (paramNames.size() + 1);
}
paramNames.add(name);
}
return paramNames;
}
private String getParamName(Annotation[] annotations) {
for(Annotation annotation : annotations) {
if(annotation.annotationType() == QueryParam.class) {
return ((QueryParam) annotation).value();
} else if(annotation.annotationType() == PathParam.class) {
return ((PathParam) annotation).value();
}
}
return null;
}
}
}
My main problem with this solution is that it requires a paragraph of comment to (hopefully) prevent future confusion. Otherwise it seems to work. Without this, I get uninformative parameter names like arg1 and so on, which I'd like to avoid. Another big problem with this solution is that it relies too heavily on the implementation of Aop proxying in Spring. The pattern may change and break this code at some point in the future and I may not be here to explain this code when the comment fails to illuminate its purpose. The weirdest thing about this is that it seems to be intermittent. Sometimes the parameter names are good and sometimes they're not. Any advice is appreciated.

It turns out this happens as a result of running the server from eclipse. I haven't quite figured out why, but running the server from the command line fixes the problem. If anyone can figure out why eclipse does this and how to turn off whatever "feature" of eclipse is causing this, I will upvote/accept your answer. For now the answer is, don't run the service in eclipse.

How to return a partial JSON response using Java?

I'm building a RESTful API and want to provide developers with the option to choose which fields to return in the JSON response. This blog post shows examples of how several API's (Google, Facebook, LinkedIn) allow developers to customize the response. This is referred to as partial response.
An example might look like this:
/users/123?fields=userId,fullname,title
In the example above the API should return the userId, fullName and title fields for User "123".
I'm looking for ideas of how to implement this in my RESTful web service. I'm currently using CXF (edit: and Jackson) but willing to try another JAX-RS implementation.
Here's what I currently have. It returns a full User object. How can I return only the fields the API caller wants at runtime based on the "fields" paramaeter? I don't want to make the other fields Null. I simply don't want to return them.
#GET
#Path("/{userId}")
#Produces("application/json")
public User getUser(#PathParam("userId") Long userId,
#DefaultValue("userId,fullname,title") #QueryParam("fields") String fields) {
User user = userService.findOne(userId);
StringTokenizer st = new StringTokenizer(fields, ",");
while (st.hasMoreTokens()) {
// here's where i would like to select only the fields i want to return
}
return user;
}
UPDATE:
I followed unludo's link which then linked to this: http://wiki.fasterxml.com/JacksonFeatureJsonFilter
With that info I added #JsonFilter("myFilter") to my domain class. Then I modified my RESTful service method to return String instead of User as follows:
#GET
#Path("/{userId}")
#Produces("application/json")
public String getUser(#PathParam("userId") Long userId,
#DefaultValue("userId,fullname,title") #QueryParam("fields") String fields) {
User user = userService.findOne(userId);
StringTokenizer st = new StringTokenizer(fields, ",");
Set<String> filterProperties = new HashSet<String>();
while (st.hasMoreTokens()) {
filterProperties.add(st.nextToken());
}
ObjectMapper mapper = new ObjectMapper();
FilterProvider filters = new SimpleFilterProvider().addFilter("myFilter",
SimpleBeanPropertyFilter.filterOutAllExcept(filterProperties));
try {
String json = mapper.filteredWriter(filters).writeValueAsString(user);
return json;
} catch (IOException e) {
e.printStackTrace();
return e.getMessage();
}
}
I need to do more testing but so far so good.

If you use Jackson (a great JSON lib - kind of the standard for Java I believe), you may use the #View annotation to filter what you want in the resulting object.
I understand that you want something dynamic so it's a bit more complicated. You will find what you are looking for here: http://www.cowtowncoder.com/blog/archives/2011/02/entry_443.html (look at 6. Fully dynamic filtering: #JsonFilter).
I would be interested in the solution you will find.

Creating an ObjectMapper instance inside the resource method for every request can have significant performance overhead. According to the Jackson performance best practices object mappers are expensive to create.
Instead you can customize the JAX-RS provider's Jackson object writer inside the resource method using the Jackson 2.3 ObjectWriterModifier/ObjectReaderModifier feature.
Here is an example shows how to register an ObjectWriterModifier thread local object that changes the set of the filters applied for the JAX-RS Jackson provider being used inside a resource method. Note that I have not tested the code against an JAX-RS implementation.
public class JacksonObjectWriterModifier2 {
private static class FilterModifier extends ObjectWriterModifier {
private final FilterProvider provider;
private FilterModifier(FilterProvider provider) {
this.provider = provider;
}
#Override
public ObjectWriter modify(EndpointConfigBase<?> endpoint, MultivaluedMap<String, Object> responseHeaders,
Object valueToWrite, ObjectWriter w, JsonGenerator g) throws IOException {
return w.with(provider);
}
}
#JsonFilter("filter1")
public static class Bean {
public final String field1;
public final String field2;
public Bean(String field1, String field2) {
this.field1 = field1;
this.field2 = field2;
}
}
public static void main(String[] args) throws IOException {
Bean b = new Bean("a", "b");
JacksonJsonProvider provider = new JacksonJsonProvider();
ObjectWriterInjector.set(new FilterModifier(new SimpleFilterProvider().addFilter("filter1",
SimpleBeanPropertyFilter.filterOutAllExcept("field1"))));
provider.writeTo(b, Bean.class, null, null, MediaType.APPLICATION_JSON_TYPE, null, System.out);
}
}
Output:
{"field1":"a"}

The Library jersey-entity-filtering Can do that :
https://github.com/jersey/jersey/tree/2.22.2/examples/entity-filtering-selectable
https://jersey.java.net/documentation/latest/entity-filtering.html
Exemple :
My Object
public class Address {
private String streetAddress;
private String region;
private PhoneNumber phoneNumber;
}
URL
people/1234?select=streetAddress,region
RETURN
{
"streetAddress": "2 square Tyson",
"region": "Texas"
}
Add to Maven
<dependency>
<groupId>org.glassfish.jersey.ext</groupId>
<artifactId>jersey-entity-filtering</artifactId>
<version>2.22.2</version>
</dependency>