Could someone let me know why the below code only fetching few entries from the parameter store ?
GetParametersByPathRequest getParametersByPathRequest = new GetParametersByPathRequest();
getParametersByPathRequest.withPath("/").setRecursive(true);
getParametersByPathRequest.setWithDecryption(true);
GetParametersByPathResult result = client.getParametersByPath(getParametersByPathRequest);
result.getParameters().forEach(parameter -> {
System.out.println(parameter.getName() + " - > " + parameter.getValue());
});
GetParametersByPath is a paged operation. After each call you must retrieve NextToken from the result object, and if it's not null and not empty you must make another call with it added to the request.
Here's an example using DescribeParameters, which has the same behavior:
DescribeParametersRequest request = new DescribeParametersRequest();
DescribeParametersResult response;
do
{
response = client.describeParameters(request);
for (ParameterMetadata param : response.getParameters())
{
// do something with metadata
}
request.setNextToken(response.getNextToken());
}
while ((response.getNextToken() != null) && ! respose.getNextToken.isEmpty());
Here is the code, based on the code above, for the new 2.0 version of AWS SSM manager. Notice I have set the maxResults to 1 to prove out the loop. You will want to remove that. AWS has mentioned that in the new code they wanted to emphasize immutability.
Using this dependency:
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>ssm</artifactId>
<version>2.10.32</version>
</dependency>
I came up with this code:
private void refreshCache() {
StopWatch sw = StopWatch.createStarted();
GetParametersByPathRequest request = GetParametersByPathRequest.builder()
.path(prefix)
.withDecryption(useDecryption)
.maxResults(1)
.build();
GetParametersByPathResponse response;
do {
response = ssm.getParametersByPath(request);
for (Parameter p : response.parameters()) {
//do something with the values.
}
request = GetParametersByPathRequest.builder()
.path(prefix)
.withDecryption(useDecryption)
.nextToken(response.nextToken())
.maxResults(1)
.build();
}
while (StringUtils.isNotBlank(response.nextToken()));
LOG.trace("Refreshed parameters in {}ms", sw.getTime());
}
private void getSsmParams() {
AWSSimpleSystemsManagement client = AWSSimpleSystemsManagementClientBuilder.defaultClient();
GetParametersByPathRequest request = new GetParametersByPathRequest();
request.withRecursive(true);
request.withPath('/your/path/parameterName').setWithDecryption(true);
GetParametersByPathResult response;
do {
response = client.getParametersByPath(request);
for (Parameter p : response.parameters()) {
//do something with the values. maybe add to a list
}
request.setNextToken(response.getNextToken())
}
while (StringUtils.isNotBlank(response.getNextToken()));
}
Above piece of code worked for me .ssm only sends 10 parameters at a time, so if you want to fetch more than 10 parameters from ssm parameter store programatically you will have to use multiple calls to fetch them. here the token is important , if there are more values in the path (request.withPath('/your/path/parameterName')) you have given, it will send a token indicating that there are more values in the given path ,and you will have to make the following request with the token received from the previous request in order to get the rest of the values.
Related
I'm trying to create from an async client a method to retrieve items from a CosmosDB but I'm afraid I'm full of questions and little to no documentation from Microsoft side
I've created a function that will read from a cosmosDB a list of items, page by page, which continuation will depend on a continuityToken. The methos looks like this. Please, be aware there could be some minor mistakes non related to the core functionality which is reading page by page:
#FunctionName("Feed")
public HttpResponseMessage getFeed(
#HttpTrigger(
name = "get",
methods = { HttpMethod.GET },
authLevel = AuthorizationLevel.ANONYMOUS,
route = "Feed"
) final HttpRequestMessage<Optional<String>> request,
#CosmosDBInput(
name = "Feed",
databaseName = Constants.DATABASE_NAME,
collectionName = Constants.LOG_COLLECTION_NAME,
sqlQuery = "SELECT * FROM c", // This won't be used actually as we use our own query
connectionStringSetting = Constants.CONNECTION_STRING_KEY
) final LogEntry[] logEntryArray,
final ExecutionContext context
) {
context
.getLogger()
.info("Query with paging and continuation token");
String query = "SELECT * FROM c"
int pageSize = 10; //No of docs per page
int currentPageNumber = 1;
int documentNumber = 0;
String continuationToken = null;
double requestCharge = 0.0;
// First iteration (continuationToken = null): Receive a batch of query response pages
// Subsequent iterations (continuationToken != null): Receive subsequent batch of query response pages, with continuationToken indicating where the previous iteration left off
do {
context
.getLogger()
.info("Receiving a set of query response pages.");
context
.getLogger()
.info("Continuation Token: " + continuationToken + "\n");
CosmosQueryRequestOptions queryOptions = new CosmosQueryRequestOptions();
Flux<FeedResponse<LogEntry>> feedResponseIterator =
container.queryItems(query, queryOptions, LogEntry.class).byPage(continuationToken,pageSize);
try {
feedResponseIterator.flatMap(fluxResponse -> {
context
.getLogger()
.info("Got a page of query result with " +
fluxResponse.getResults().size() + " items(s)"
+ " and request charge of " + fluxResponse.getRequestCharge());
context
.getLogger()
.info("Item Ids " + fluxResponse
.getResults()
.stream()
.map(LogEntry::getDate)
.collect(Collectors.toList()));
return Flux.empty();
}).blockLast();
} catch (Exception e) {
}
} while (continuationToken != null);
context
.getLogger()
.info(String.format("Total request charge: %f\n", requestCharge));
return request
.createResponseBuilder(HttpStatus.OK)
.header("Content-Type", "application/json")
.body("ALL READ")
.build();
}
For simplicity the read items are merely logged.
First question: We are using an async document client that returns a Flux. Will the client keep track of the token? It is a stateless client in principle. I understand that the sync client could take easily care of this case, but wouldn't the async client reset its memory of tokens after the first page and token has been generated?
Second: Is the while loop even appropriated? My assumption is a big no, as we need to send back the token in a header and the frontend UI will need to send the token to the Azure Function in a header or other similar fashion. The token should be extracted from the context then
Third: Is the flatMap and blockList way to read the flux appropriate? I was trying to play with the subscribe method but again I don't see how it could work for an async client.
Thanks a lot,
Alex.
UPDATE:
I have observed that Flux only uses the items per page value to set the number of items to be retrieved per batch, but after retrieval of one page it doesn't stop and keeps retrieving pages! I don't know how to stop it. I have tried substituting the Flux.empty() per Mono.empty() and setting a LIMIT clause in the sql query. The first option does the same and the second freezes the query and never returns apparently. How can I return one page an only one page along with the continuation token to do the following query once the user clicks on the next page button?
I am using AWS Java SDK v2 to list users using the code defined here on the AWS GitHub repo.
public static void listAllUsers(IamClient iam) {
try {
boolean done = false;
String newMarker = null;
while (!done) {
ListUsersResponse response;
ListUsersRequest request;
if (newMarker == null) {
request = ListUsersRequest.builder().build();
} else {
request = ListUsersRequest.builder()
.marker(newMarker).build();
}
response = iam.listUsers(request);
for (User user : response.users()) {
System.out.format("\n Retrieved user %s", user.userName());
System.out.println("\nPermission Boundary: " + user.permissionsBoundary());
}
if (!response.isTruncated()) {
done = true;
} else {
newMarker = response.marker();
}
}
} catch (IamException e) {
System.err.println(e);
System.exit(1);
}
}
It returns NULL for user.permissionsBoundary(). Here is the output for print statements in the above code.
Retrieved user jamshaid
Permission Boundary: null
Retrieved user luminadmin
Permission Boundary: null
Retrieved user test
Permission Boundary: null
When I run the following command in AWS CloudShell on AWS console, it returns the PermissionBoundary for the users it is defined.
aws iam get-user --user-name test
Here is the sample output from AWS CloudShell.
I am using the same account to make both requests.
I have confirmed this behavior by setting a permission boundary on an IAM user in the AWS Management Console. I changed the ListUsers example to include this code:
for(User user : response.users()) {
System.out.format("\n Retrieved user %s", user.userName());
AttachedPermissionsBoundary permissionsBoundary = user.permissionsBoundary();
if (permissionsBoundary != null)
System.out.format("\n Permissions boundary details %s", permissionsBoundary.permissionsBoundaryTypeAsString());
}
...
The permissionsBoundary() method does return null - even though the permission is set. This is a bug.
My advice here is to log a Github issue here:
https://github.com/aws/aws-sdk-java-v2
I also tested this with Kotlin SDK. Same result.
suspend fun listAllUsers() {
IamClient { region = "AWS_GLOBAL" }.use { iamClient ->
val response = iamClient.listUsers(ListUsersRequest { })
response.users?.forEach { user ->
println("Retrieved user ${user.userName}")
val permissionsBoundary = user.permissionsBoundary
if (permissionsBoundary != null)
println("Permissions boundary details ${permissionsBoundary.permissionsBoundaryType.toString()}")
}
}
}
I do not think it is an issue, but the programmed behavior. From the API docs:
IAM resource-listing operations return a subset of the available attributes
for the resource. For example, this operation does not return tags, even
though they are an attribute of the returned object. To view all of the
information for a user, see GetUser.
This is stated as well in the API javadocs.
In the console you are using get-user, not list-users, and this is why the command is returning all the information about the user, PermissionsBoundary within it.
Please, try instead using:
aws iam list-users
and check the output, it should match the result you obtained with the Java SDK, it will not contain PermissionsBoundary either.
If you want to obtain the same results that you are currently getting with the command aws iam get-user --user-name test from Java code, you can use the getUser method in IamClient. Try Something like:
GetUserRequest request = GetUserRequest.builder()
.userName("test")
.build()
;
GetUserResponse response = iam.getUser(request);
User user = response.user();
System.out.println("\nPermission Boundary: " + user.permissionsBoundary());
The User class is reused in both operations, get and list, but only in the former all the fields are populated.
I am trying to iterate over a paginated list of accounts but when I send a request using the value from "next_uri" I receive an error from the server:
{"errors":[{"id":"not_found","message":"Not found"}]}
I am correctly adding headers etc as all other API calls work fine, its just the request using the "next_uri" that is not working. I think I am following the api spec correctly so I am unsure what is the issue and how to fix it. Does anyone know what is wrong with the code / logic please?
Simplified code:
ArrayList<X> results = new ArrayList<>();
String uri = "/v2/accounts";
javax.ws.rs.client.Client client = getClient();
while(uri != null){
T response = client.target("https://api.coinbase.com")
.path(uri).request(MediaType.APPLICATION_JSON).get(responseType);
results.addAll(response.getData());
uri = response.getPagination()==null ? null :response.getPagination().getNextUri();
}
return results;
The results are this:
Request 1:
https://api.coinbase.com/v2/accounts
Response 1: pagination":
{"ending_before":null,"starting_after":null,"previous_ending_before":null,"next_starting_after":"ef35df6c-a45b-5858-b755-f12a709cf26e","limit":25,"order":"desc","previous_uri":null,"next_uri":"/v2/accounts?starting_after=ef35df6c-a45b-5858-b755-f12a709cf26e"},"data":[{....}]
Request 2:
https://api.coinbase.com/v2/accounts%3Fstarting_after=ef35df6c-a45b-5858-b755-f12a709cf26e
Response 2:
{"errors":[{"id":"not_found","message":"Not found"}]}
This was related to how the jax-rs library needs the query params adding. Just relying on the uri is not enough, the parameters also need adding specifically:
target = target.queryParam(e.getKey(), e.getValue());
so the final code is something like
WebTarget target = client.target(e"https://api.coinbase.com");
if(params !=null){
for(Map.Entry<String, String> e : params.entrySet()){
target = target.queryParam(e.getKey(), e.getValue());
}
}
target = target.path(path);
return target.request(MediaType.APPLICATION_JSON).get(responseType);
iam trying to orderlookup droplet API by passing some parameters.I assume that the parameters which are mandatory is userId and organisationIds which i have passed and additionally i have also passed "state" parameter.All these params are passed thru request and then the service method of droplet is invoked.But the service method returns nothing.My goal is to check whether this droplet this retrieving the expected set of orders or not.We can use droplet invoker but i tried that way but it didnt work may be i missed something.Please help me out!!
this is my code when i tried to use OrderLookUp API
DynamoHttpServletRequest request = ServletUtil.getCurrentRequest();
mTestService.setCurrentRequest(request);
if (request == null) {
mTestService.vlogError("Request is null.");
Assert.fail("Request is null ");
}
else
{
Object droplet = mTestService
.getRequestScopedComponent("OrderLookupDroplet");
OrderLookupDroplet=(OrderLookup) droplet;
request.setParameter("state", "submitted");
request.setParameter("organisationIds", organizationIds);
request.setParameter("userId", userId);
ByteBuffer buffer = ByteBuffer.allocate(1024);
DynamoHttpServletRequest dynRequest = (DynamoHttpServletRequest) request;
TestingDynamoHttpServletRequest wrappedRequest = new TestingDynamoHttpServletRequest(
dynRequest, buffer);
TestingDynamoHttpServletResponse wrappedResponce = new TestingDynamoHttpServletResponse(
dynRequest.getResponse());
OrderLookupDroplet.service(wrappedRequest, wrappedResponce);
}
the above sample is only part of the code..
this is the code when i tried using droplet invoker
DropletInvoker invoker = new DropletInvoker(mNucleus);
invoker.getRequest().setParameter("state", "submitted");
// String [] siteIds = {"siteA", "siteB"};
// invoker.getRequest().setParameter("siteIds", Arrays.asList(siteIds));
String [] organizationIds = {"OrgA", "OrgB"};
invoker.getRequest().setParameter("organizationIds", organizationIds);
String [] orderIds = {"orderautouser001OrgA" , "orderautouser001OrgB"};
invokeDroplet(invoker, "autouser001", orderIds);
......
protected void invokeDroplet(DropletInvoker pInvoker, String pUserId, String[] pOrderIds) throws Exception
{
Map<String, Object> localParams = new HashMap();
localParams.put("userId", pUserId);
DropletResult result = pInvoker.invokeDroplet("/atg/commerce/order/OrderLookup", localParams);
RenderedOutputParameter oparam = result.getRenderedOutputParameter("output", 0);
assertNotNull("'output' oparam was not rendered", oparam);
assertEquals("Check totalCount.", pOrderIds.length, oparam.getFrameParameter("totalCount"));
List<Order> orders = (List<Order>)oparam.getFrameParameter("result");
assertEquals("Check order array length.", pOrderIds.length, orders.size());
for (int index = 0; index < pOrderIds.length; index++) {
boolean found = false;
for (Order order: orders) {
if (pOrderIds[index].equals(order.getId())) {
found = true;
break;
}
}
assertTrue("Expected orderId " + pOrderIds[index] + " not found in result array", found);
}
in first case i donno how to retrieve the orders by directly using orderlookup api....and in second case though i know how to use it ,iam still failing!! please help me out..thanks in advance
You should't use droplets in java classes they should be used only inside jsp pages. Documentation of OrderLookup with example hot to use it on jsp page is here.
If you want to get orders or any other data stored in a repository you should use repository API with RQL (Repository Query Language). Example how to get data from repository you can find here and RQL grammar here.
Thanks for giving your opinions.Good news is we can invoke droplets from any other API
OrderLookup droplet = (OrderLookup) sNucleus.resolveName("/atg/commerce/order/OrderLookup");
ServletTestUtils utils = new ServletTestUtils();
mRequest = utils.createDynamoHttpServletRequestForSession(sNucleus, null, null);
ServletUtil.setCurrentRequest(mRequest);
mResponse = new DynamoHttpServletResponse();
mRequest.setResponse(mResponse);
mResponse.setRequest(mRequest);
mResponse.setResponse(new GenericHttpServletResponse());
mRequest.setParameter("userId", "publishing");
droplet.setSearchByUserId(true);
droplet.service(mRequest, mResponse);
ArrayList<Order> orders = (ArrayList<Order>) mRequest.getObjectParameter("result");
here the "result" param is output param which this droplet sets.and the userId i have hardcoded as "publishing" which i have created.Ignore servletTestUtils class that is created by me which has not much to do with droplet theory here :)
I assume from your code example, and the fact that you mention DropletInvoker that you are writing a unit test, and that this is not functional code.
If it is functional code, you really, really, should not invoke a droplet from another Nucleus component. A droplet exists solely to be used in a JSP page. If you need the functionality of the droplet in Java code, you should refactor the droplet into a service that holds the main logic, and a droplet that simply acts as a façade to the service to allow it to be invoked from a page.
In the case of the OrderLookup look droplet, you don't need to refactor anything. The service to use should be OrderManager or OrderTools depending on what you need. Note, there is a difference between Order objects and Order repository items, and you should prefer to use order objects - so only use the Order Repository directly if you really need to.
How to get the details of Multiple products in a Single Call in Android using XMLRPC from Magento.I am able to get the list of products using the function catalog_product.list using XMLRPC.
Now, i have the SKU id's of all the products.I am able to get the media details of each product using the function product_media.list.
If suppose I have 10 products,i have to call 10 times product_media.list method for each product which takes long time.
So,how can I call the multiCall function of Magento in Android. Many tutorials in php for calling the multiCall function are posted but I am not able to imitate the same in Android.
So please help me if you have similar code snippet that can make me understand multiCall function(for Android) so that I can Advance further using that.
Thanks.
PHP code Example from Josua Marcel C 's Answer:
$session = $client->call('login', array('apiUser', 'apiKey'));
$client->call('call', array($session,'somestuff.method', array('arg1', 'arg2', 'arg3')));
$client->call('call', array($session, 'somestuff.method', 'arg1'));
$client->call('call', array($session, 'somestuff.method'));
$client->call('multiCall',
array($session,
array(
array('somestuff.method', 'arg1'),
array('somestuff.method', array('arg1', 'arg2')),
array('somestuff.method')
)
)
);
I would like to imitate the above php code in Android that calls the multiCall() function of the Magento.
After making long long Research, I got half-way Solution that calls the multiCall() method without any Error,but Still I don't know how to get the response of the Server in a variable and use it.
AnyOne who has knowledge of it can Edit my Answer, I will be thankful to him.
The Code that I have Used is :
Object[] skuid=new Object[product_list.size()];
Object calling[]=new Object[product_list.size()];
for(int m=0;m<product_list.size();m++)
{
skuid[m]=new Object[]{product_list.get(m).getp_Sku()};
calling[m]=new Object[]{"catalog_product_attribute_media.list",skuid[m]};
}
try
{
client.callEx("multiCall",new Object[]{Utils.sessionId,calling});
}
catch (XMLRPCException e)
{
e.printStackTrace();
}
AcknowledgeMents :
I have worked on the Answer posted by Iain.
The Answer
since android is based java application, You can use this.
package org.apache.xmlrpc;
import java.util.Hashtable;
import java.util.Vector;
public class MultiCall
implements ContextXmlRpcHandler
{
public Object execute(String method, Vector params, XmlRpcContext context)
throws Exception
{
if ("multicall".equals(method))
{
return multicall(params, context);
}
throw new NoSuchMethodException("No method '" + method + "' in " + this.getClass().getName());
}
public Vector multicall(Vector requests, XmlRpcContext context)
{
// The array of calls is passed as a single parameter of type array.
requests=(Vector)requests.elementAt(0);
Vector response = new Vector();
XmlRpcServerRequest request;
for (int i = 0; i < requests.size(); i++)
{
try
{
Hashtable call = (Hashtable) requests.elementAt(i);
request = new XmlRpcRequest((String) call.get("methodName"),
(Vector) call.get("params"));
Object handler = context.getHandlerMapping().getHandler(request.getMethodName());
Vector v = new Vector();
v.addElement(XmlRpcWorker.invokeHandler(handler, request, context));
response.addElement(v);
}
catch (Exception x)
{
String message = x.toString();
int code = (x instanceof XmlRpcException ?
((XmlRpcException) x).code : 0);
Hashtable h = new Hashtable();
h.put("faultString", message);
h.put("faultCode", new Integer(code));
response.addElement(h);
}
}
return response;
}
}
Source
Since Magento support SOAP API why didn't you use SOAP API v1? because SOAP is powerful. try to go here What's the difference between XML-RPC and SOAP?
Parsing of Soap messages is not included in Android runtime, so it isn't really straightforward. You should use an external library. I'm using ksoap2.
If you search here on StackOverflow you'll see many examples on how to use it. For instance here
more references: link 1 link 2
MultiCall with PHP
$client = new Zend_XmlRpc_Client('http://magentohost/api/xmlrpc/');
// If somestuff requires api authentification,
// we should get session token
$session = $client->call('login', array('apiUser', 'apiKey'));
$client->call('call', array($session, 'somestuff.method', array('arg1', 'arg2', 'arg3')));
$client->call('call', array($session, 'somestuff.method', 'arg1'));
$client->call('call', array($session, 'somestuff.method'));
$client->call('multiCall', array($session,
array(
array('somestuff.method', 'arg1'),
array('somestuff.method', array('arg1', 'arg2')),
array('somestuff.method')
)
));
// If you don't need the session anymore
$client->call('endSession', array($session));
First login in whatever way works for calling catalog_product.list. Make sure session, client and product_ids have the right values. If you don't need to log in for these operations, set session = null (and if that doesn't work, try not passing session at all :) ). Then:
Object[][] calls = new Object[product_ids.length];
for (int i = 0; i < product_ids.length; i++) {
calls[i] = new Object[] { "product_media.list", product_ids[i] };
}
product_media_ids = client.call("multiCall", new Object[] { session, calls });
product_media_ids should then be an array of arrays of product images - that is, each element of product_media_ids will be a return value from product_media.list.
The code is untested, I'm afraid.