Develop Reference

Is it possible to get lineage metadata from the pipeline in my Data Fusion Action plugin?

I'm trying to get data lineage metadata like data source/schema and data target/schema in a custom Action plugin which gets executed after the successful run of the other steps in the pipeline.
I have a basic Action plugin that executes but I'm having trouble finding a way to get the metadata I'm after.
The use case I'm working on is pushing data lineage into a third party data governance tool.
I would very much appreciate if someone could point me in the right direction!

As was suggested in my comment, you might consider to use CDAP system metadata inventory to extract the particular property for the desired entity via CDAP existed RESTfull API methods by sending appropriate HTTP request as explained in CDAP Metadata Microservices documentation. Said this entity properties can also depict lineage of dataset fields returning the result in JSON format.
However, adjusting appropriate HTTP method mostly depends on the particular use case, therefore feel free to further contribute and share your further discovering.

How do I update a Pentaho transformations connection data using Java?

I’m using Java to build a restful API around the functionality of Pentaho data integration.
I’ve implemented several end points such as the ability to create a repository
containing jobs and transformations, running the jobs and transformations, displaying the image of the job and transformations, displaying the database connections within a repo, plus quite a few more.
I’m trying to build an endpoint that allows for the data sources to be changed, such as the hostname, database name, etc. But I’ve run into an issue when it comes to saving the new connection details.
Here’s a snippet of code I’ve got. I’ve hard coded the values simply for testing purposes.
I loop through an array list containing the DatabaseMeta and then change the values of the fields.
for(DatabaseMeta meta: databaseMeta) {
meta.setHostName(“test_host”) ;
meta.setDBPort(“test_port”);
meta.setDBName(“test_database”);
repositoryService.updateDataSource(databaseMeta);
}
The updateDatasource() method simply invokes repository.save() (which is part of the org.pentaho.di.repository package) and passes in the DatabaseMeta.
When this method executes, it creates a .kdb file in my repository, with the values I set above, and making a GET request to the endpoint returns the connection details from the new file.
However, I simply want to overwrite the values in the existing transformation connection and return them in the GET request.
Is there any way that this can be achieved?
Any help will be greatly appreciated.
Thanks.

I don't know about the JAVA integration part, but as far as pure Pentaho goes, the Database Connection stated in the KTR/KJB needs to have the same parameters declared in the KTR/KJB, as such:
This way, whatever parameters you pass through to the KTR/KJB will be swapped out in the connection.

AEM - Post CQ dynamic forms to external system using web service

We are building a site using AEM, where we have lot of forms (login, change address, change email) that will POST their data to an external system using web service. We had achieved this by using "slingServlet", where we used to get values using “sling.getParameter(“ ”). With this approach we are fixed with the form components used inside the particular forms.
For example login form, we are using username and password parameter, but in future we decided to add “dob” field too. To achieve this we need to do the code changes again. By doing so I believed we are not leveraging the CMS concept. So the question is there any other way to achieve this without doing code changes.

Instead of getting parameters by using sling.getParameter("") try using one of these instead
sling.getRequestParameterMap();
or
sling.getParamterMap();
This will give you a map of your parameters regardless of their name. Returns the parameters and name | value pairs.
Here are some helpful links:
https://sling.apache.org/apidocs/sling5/org/apache/sling/api/SlingHttpServletRequest.html#getRequestParameterMap()
http://docs.oracle.com/javaee/6/api/javax/servlet/ServletRequest.html#getParameterMap()

How to add a flag to existing POST api

I have a REST api built for creating orders. The behavior is such that the person who creates an order received an email back saying "You created an order XXX". This email is triggered all the time.
The api appears like this
http://api.mytestevnt.com/ordering/orders - POST with request body as the order entity json.
Now i want to give a feature to the api caller to indicate if the email notification is necessary or not. What's the best way to do this?

I think it depends on whether email notification is data or metadata. If it's part of the order, then definitely add it to the request body. If it's metadata, you have two choices. If you think there will be lots of metadata, you can either edit the order to have a metadata section or you can POST the metadata separately. If there will only be a limited amount of metadata, I would suggest using a query parameter.
You should avoid using a header unless you control the entire path from the client to the server, because proxies or load balancers are allowed to strip non-standard headers.

Include in the POST body a send_email=1 or send_email=0 param. You'll extract that, and see what the user wants to do.
Search "how to get POST variables in JAVA".
Accessing post variables using Java Servlets

You can do like this:
Add a new Java attribute(like boolean emailEnabled) in your Java Request Object for your REST service.
Client side which invokes your REST service need to provide that parameter you added in your server side, you can set a default value for that too.

Modifying AMF response at runtime - How to?

I have built a simple flex application (using BlazeDS) which displays "Person" details by invoking a java service using flex remoting.
So, my Person class looks something like this:
class Person {
public int age;
public String name;
}
As a java developer, here is my understanding:
When I run the front flex app in my webapp, an RPC call is executed and BlazeDS does the hard work of invoking the right method, obtaining the result as an object, converting the object to AMF format (serialization). And, then the web/app server sends back this response to the requesting app over http. The flex app now does the task of deserializing the object received in AMF format and somehow make use of it.
Here is my question:
I would like to modify the response before it reaches the Flex application by modifying the attributes of the person object. I have a Filter in place to do this. However, I am not sure as to how I would deserialize the AMF stream, modfiy the object, serialize the object back into the stream / reconstruct the stream somehow.
Is there a way to doing this? Where should I start?
PS
If the question was too confusing, here is a shorter version:
How do I modify the response stream of content type AMF-X, before it reaches a flex app using a java filter?

Serializing and deserializing AMF on your own is not going to be easy so definitely avoid that. A better approach is to convert the data to the right format in your back-end code. From Flex, call a service method on the back-end that then fetches the data and transforms it into the correct object structure before sending it back to Flex.

To further James' answer, use something like spring to interecpt the return of the call and interfere with it there (again, before it gets serialized).
You would basically have to reimplement the (de) serialization process, and that's not simple.

Well, It's not exactly what you want, but if you only need to change something occasionally while testing there's a solution. Charles Proxy (http://www.charlesproxy.com/) does let you modify the return data 'by hand'. It's a tool I use daily for monitoring AMF traffic, as it does a great job deserializing it, and showing it human readable format. There's a free version you can try.
Short instruction on how to edit the data:
0) Install Charles (including the firefox plugin if you use it
1) Choose Proxy->Breakpoints from the main menu.
2) Fill in the host (you can do it using wildcards - eg. mydomain.com* )
3) Check the "Response" box
4) Call the server
5) The breakpoint should fire when data comes back. Now just find the "Edit Response" tab, choose "AMF" on the bottom, and you can edit anything you want.
Hope this helps