CDT API: Issues looking up values using the IIndex - java

Part of a program I am working on requires looking up preprocessor macros by name, and then getting their values. I opted to use the CDT Indexer API. In order to make sure I am on the right track, I wrote a test method that does nothing but create a simple C file and confirm that it can find certain symbols in the index. However, I failed to get that test to run properly. Attempting to use IIndex.findBindings(char[], IndexFilter, IProgressMonitor) returns empty arrays for symbols that I know exist in the AST because they are part of the example file in the test method.
I can't post the exact test method because I use some custom classes and it would be overkill to post all of them, so I will just post the important code. First, my example file:
final String exampleCode =
"#define HEAVY 20\n" +
"#define TEST 5\n" +
"void function() { }\n" +
"int main() { return 0; }\n";
IFile exampleFile = testProject.getFile("findCodeFromIndex.c");
exampleFile.create(new ByteArrayInputStream(exampleCode.getBytes("UTF-8") ), true, null);
I have a custom class that automatically gets the IASTTranslationUnit from that file. The translation unit is fine (I can see the nodes making up everything except the macros). I get the index from that AST, and the code I use to look up in the index is
try {
index.acquireReadLock();
returnBinding = index.findBindings(name.toCharArray(), IndexFilter.ALL, null);
... catch stuff...
} finally {
index.releaseReadLock();
}
Where 'name' is going to be either "HEAVY", "TEST", or "function". None of them are found, despite existing in the example test c file.
I am guessing that the issue is the index is not rebuilt, which causes findBindings to return an empty array even if I know the given variable name exists in the AST.
My current attempt to start up the indexer looks like this:
final ICProject cProject = CoreModel.getDefault().getCModel().getCProject(testProject.getName());
CCorePlugin.getIndexManager().reindex(cProject);
CCorePlugin.getIndexManager().joinIndexer(IIndexManager.FOREVER, new NullProgressMonitor() );
Question Breakdown:
1) Is my method for searching the index sound?
2) If the issue is the index needing to be rebuilt, how should I properly force the index to be up to date for my test methods? Otherwise, what exactly is the reason I am not resolving the bindings for macros/functions I know exist?

I solved my own issue so I will post it here. I was correct in my comment that the lack of the project being a proper C project hindered the Indexer from working properly, however I also discovered I had to use a different method in the indexer to get the macros I needed.
Setting up the test enviornment:
Here is the code I have that creates a basic C project. The only purpose it serves is to allow the indexer to work for test methods. Still, it is large:
public static IProject createBareCProject(String name) throws Exception {
IProject bareProjectHandle = ResourcesPlugin.getWorkspace().getRoot().getProject(name);
IProjectDescription description =
bareProjectHandle.getWorkspace().newProjectDescription("TestProject");
description.setLocationURI(bareProjectHandle.getLocationURI() );
IProject bareProject =
CCorePlugin.getDefault().createCDTProject(description, bareProjectHandle, new NullProgressMonitor() );
IManagedBuildInfo buildInfo = ManagedBuildManager.createBuildInfo(bareProject);
IManagedProject projectManaged =
ManagedBuildManager
.createManagedProject(bareProject,
ManagedBuildManager.getExtensionProjectType("cdt.managedbuild.target.gnu.mingw.exe") );
List<IConfiguration> configs = getValidConfigsForPlatform();
IConfiguration config =
projectManaged.createConfiguration(
configs.get(0),
ManagedBuildManager.calculateChildId(configs.get(0).getId(), null));
ICProjectDescription cDescription =
CoreModel.getDefault().getProjectDescriptionManager().createProjectDescription(bareProject, false);
ICConfigurationDescription cConfigDescription =
cDescription.createConfiguration(ManagedBuildManager.CFG_DATA_PROVIDER_ID, config.getConfigurationData() );
cDescription.setActiveConfiguration(cConfigDescription);
cConfigDescription.setSourceEntries(null);
IFolder srcFolder = bareProject.getFolder("src");
srcFolder.create(true, true, null);
ICSourceEntry srcFolderEntry = new CSourceEntry(srcFolder, null, ICSettingEntry.RESOLVED);
cConfigDescription.setSourceEntries(new ICSourceEntry[] { srcFolderEntry });
buildInfo.setManagedProject(projectManaged);
cDescription.setCdtProjectCreated();
IIndexManager indexMgr = CCorePlugin.getIndexManager();
ICProject cProject = CoreModel.getDefault().getCModel().getCProject(bareProject.getName() );
indexMgr.setIndexerId(cProject, IPDOMManager.ID_FAST_INDEXER);
CoreModel.getDefault().setProjectDescription(bareProject, cDescription);
ManagedBuildManager.setDefaultConfiguration(bareProject, config );
ManagedBuildManager.setSelectedConfiguration(bareProject, config );
ManagedBuildManager.setNewProjectVersion(bareProject);
ManagedBuildManager.saveBuildInfo(bareProject, true);
return bareProject;
}
As I discovered when debugging, it is indeed important to set proper configurations and descriptions as the indexer was postponed so long as the project didn't have those features set. To get the configurations for the platform as a starting point for an initial configuration:
public static List<IConfiguration> getValidConfigsForPlatform() {
List<IConfiguration> configurations =
new ArrayList<IConfiguration>();
for (IConfiguration cfg : ManagedBuildManager.getExtensionConfigurations() ) {
IToolChain currentToolChain =
cfg.getToolChain();
if ( (currentToolChain != null ) &&
(ManagedBuildManager.isPlatformOk(currentToolChain) ) &&
(currentToolChain.isSupported() ) ) {
configurations.add(cfg);
}
}
return configurations;
}
This basically answers the second part of the question, and thus I can create a c project for the purposes of testing code using the index. The testing code still needs to do some work.
Testing Code
I create files in the the "src" folder in the project (created in the above code), and I either have to name them .c, or if I want to name them .h have them included by some .c file (otherwise the indexer won't see them). Finally, I can populate the files with some test code. To answer number 1,
I need to block on both auto refresh jobs in Eclipse and then the index:
public static void forceIndexUpdate(IProject project) throws Exception {
ICProject cProject = CoreModel.getDefault().create(project);
Job.getJobManager().join(ResourcesPlugin.FAMILY_AUTO_REFRESH, null);
CCorePlugin.getIndexManager().reindex(cProject);
CCorePlugin.getIndexManager().joinIndexer(IIndexManager.FOREVER, new NullProgressMonitor() );
assertTrue(CCorePlugin.getIndexManager().isIndexerIdle() );
assertFalse(CCorePlugin.getIndexManager().isIndexerSetupPostponed(cProject));
}
After I change the files in the project. This makes sure Eclipse is refreshed, and then makes sure the indexer completes without being postponed. Finally, I can run tests depending on the indexer.
And the last point, I was wrong about using IBinding. The correct way in which I was able to get the macros was using the method IIndex.findMacros(char[] name, IndexFilter filter, IProgressMonitor monitor)
I hope this helps at least someone out there. I would also appreciate it if there was some feedback regarding the validity of this solution, as this is simply the first solution I managed to create that worked. Just to confirm, I am not testing the indexer itself, but rather code I wrote that uses the indexer and I want to test it under as realistic conditions as I can given how critical it is.

Related

Trouble splitting an array in #PostConstruct Method of my Kafka SpringBoot Application

I'm creating a Kafka Springboot listener that tracks the state of an object. I have the kafka portion working and am able to listen to the topics. I am using a Tree map to map a string key that is the topic to an object. The topic name actually contains some information that I want to use to initialize the object. My question is this ( I apologize, I'm not very familiar with SpringBoot).
In my Post Construct method, I have a string array that is coming in called locoTopics. I've verified that strings are coming in and they are of the form "ignore.ignore.mark.id". The issue I'm running into is that when I try to split the string in the Post Construct method it appears to return null even though the string definitely contains the "." expression. I say it appears, because the program appears to jump forward and skip several lines of code. To be precise, when I debug. It appears to go from the "String [] locoArray = locoTopics[i].split("\.");" line to my Object constructor immediately and skip all the steps in between. Then it appears to jump in and out of the loop. I'm very confused why this is. Is this something related to SpringBoot that is happening or am I missing something in my code? Any help would be greatly appreciated.
public class InitTrackerReceiver {
// instance variables
Map mapr = new TreeMap<String, Locomotive>();
String[] locoTopics;
public List<String> m_failureCodes;
// Moved Functionality to get loco ids into separate class, Locomotive IDs now
// gets locoids
#Autowired
public LocomotiveIDs locomotiveIDs;
//This is where loco objects will be instantiated
#PostConstruct
public void initializeLocomotives() {
// grab loco topic strings and separate into a string array
locoTopics = locomotiveIDs.getLocoString();
// Go through the loco topics from application.properties and create a Loco
// object with that string as identifier
for (int i = 0; i < locoTopics.length; i++) {
String [] locoArray = locoTopics[i].split("\\.");
String markString = locoArray[2];
String scacString = "name";
String idString = locoArray[3];
mapr.put(locoTopics[i], new Locomotive(locoTopics[i], markString, scacString, idString));
}
}
#Value("#{'${failure.init.messagefields}'.split(',')}")
List<String> typesToFail;
I figured I'd go ahead and follow up with the answer. I have no clue how this happened, but the code that was in Eclipse that was in the debugger was not the same code that was running. I don't know if this was a bug in EGIT or Eclipse. It became apparent when I ran a Maven clean and my project would no longer launch. Whenever I tried to debug, it would launch and say that it couldn't find the main class, which was my Spring application launch class. I did a Maven Build and that seemed to get everything in sync. Then the code started working as expected.

Not able to reference Java class from my report in BIRT

I am very new to BIRT. I am wokring on a BIRT project where I am trying to reference Java class inside script 'open' section but am unable to do so.
I do not get any errors but I am not able to see any data in my dataset preview.
Script - open
count = 0;
// create instance of
// the GetStockHistory class
gsh = new Packages.de.vogella.birt.stocks.daomock.StockDaoMock(); //cause of error somehow
//Load the List
stock = gsh.getStockValues("Java");
Script-Fetch
if(count < stock.size()){
row["columnDate"] = stock.get(count).getDate();
row["columnOpen"] = stock.get(count).getOpen();
row["columnHigh"] = stock.get(count).getHigh();
row["columnLow"] = stock.get(count).getLow();
row["columnClose"] = stock.get(count).getClose();
row["columnVolume"] = stock.get(count).getVolume();
count++;
return true;
}
return false;
StockDaoMock is a class which returns a dummy list of values.
Referring this blog BIRT sample app
Can anyone please help me here and let me know what am I doing wrong ?
Why can't I see any data in preview dataset. Is there a specific way in which I need to make reference to java classes because I am sure the error is somewhere in that part only. If I remove the reference part and just hardcode a string, then it is working fine and I can see it in the preview. Things mess up as soon as I refer a java class by importing it.
BIRT-4.8
EDIT---
even this inside my Script 'open' doesn't work
importPackage(Packages.de.vogella.birt.stocks.daomock);
gsh = new StockDaoMock();
BIRT does not use the java sources directly. You have to generate a JAR from your classes and add that JAR to your BIRT class path (Window / Preferences / Report Design / Classpath).

recognize parameter change from git repository

I want to extract signature changes (method parameter changes to be exact) from commits to git repository by a java program. I have used the following code:
for (Ref branch : branches) {
String branchName = branch.getName();
for (RevCommit commit : commits) {
boolean foundInThisBranch = false;
RevCommit targetCommit = walk.parseCommit(repo.resolve(
commit.getName()));
for (Map.Entry<String, Ref> e : repo.getAllRefs().entrySet()) {
if (e.getKey().startsWith(Constants.R_HEADS)) {
if (walk.isMergedInto(targetCommit, walk.parseCommit(
e.getValue().getObjectId()))) {
String foundInBranch = e.getValue().getName();
if (branchName.equals(foundInBranch)) {
foundInThisBranch = true;
break;
}
}
}
}
I can extract commit message, commit data and Author name from that, however, I am not able to extract parameter changes from them. I mean it is unable for me to identify parameter changes. I want to know if there is any way to recognize that. I mean it is impossible to recognize them from commit notes that are generated by programmers; I am looking for something like any specific annotation or something else.
This is my code to extract differences:
CanonicalTreeParser oldTreeIter = new CanonicalTreeParser();
oldTreeIter.reset(reader, oldId);
CanonicalTreeParser newTreeIter = new CanonicalTreeParser();
newTreeIter.reset(reader, headId);
List<DiffEntry> diffs= git.diff()
.setNewTree(newTreeIter)
.setOldTree(oldTreeIter)
.call();
ByteArrayOutputStream out = new ByteArrayOutputStream();
DiffFormatter df = new DiffFormatter(out);
df.setRepository(git.getRepository());
The export is really huge and impossible to extract method changes.
You show a way you've found to examine the diffs, but say that the output is too large and you can't extract the method signature changes. If by that you mean that you're asking about specific git support for telling you that a method signature changes, then no - no such support exists. This is because git does not "know" anything about the languages you may or may not have used in the files under source control. Everything is just content that is, or is not, different from other content.
Since a method signature could be split across lines in any number of ways, it's not even guaranteed that just because a method's signature changed its name would appear anywhere in the diff. What you would really have to do is perform a sort of "structural diff". That is, you would have to
check out the "old" version, and pass it to a java parser
check out the "new" version, and pass it to a java parser
compare the resulting parse trees, looking for methods that belong to the same object, but have changed
Even that won't be terribly easy, because methods could be renamed, and because method overloading could make it unclear which signature change goes with which version of a method.
From there what you have is a non-trivial coding problem, which is beyond the scope of SO to answer. If you decide to tackle this problem and run into specific programming questions along the way, of course you could post those questions and perhaps someone will be able to help.

How to resolve External Control of File Name or Path (CWE ID 73)

I am working on fixing Veracode issues in my application. Veracode has highlighted the flaw "External Control of File Name or Path (CWE ID 73) " in below code.
Thread.currentThread().getContextClassLoader().getResourceAsStream(lookupName)
How do I validate the parameter? If I need to use below ESAPI validation, then what is the exact parameter I should be passing in getValidFileName() method. Currently I am passing the parameters as below.
ESAPI.validator().getValidFileName(lookupName, lookupName,
ESAPI.securityConfiguration().getAllowedFileExtensions(), false);
Correct me whether I am following the right approach for fixing this issue.
There are several suggestions at: https://community.veracode.com/s/article/how-do-i-fix-cwe-73-external-control-of-file-name-or-path-in-java
You can use hardcoded values, if these files are stored in the server side.
(i.e.: in a HashMap).
Another solution is to use a custom validator (from veracode page) :
// GOOD Code
String extension = request.getParameter("extension");
File f = new File(buildValidAvatarPath(extension))
#FilePathCleanser
public String buildValidAvatarPath(extension) {
String[] allowedExtensions = new String[]{"jpg","gif","png"};
String extension = "png"; // Default extension
for (String allowedExtension: allowedExtensions) {
if (allowedExtension.equals(request.getParameter("extension"))) {
extension = request.getParameter("extension");
}
}
// See "Note on authorization"
User user = getCurrentUser();
if (!userMayAccessFile(user, path)) {
throw new AuthorizationException("User may not access this file", user);
}
File(configPath + "avatar." + extension)
return path;
}
Okay, so the problem is that you are allowing user-control of that file path. Imagine its on a UNIX box and they enter:
../../../../../../../etc/shadow
Whatever user privileges are granted to the user running that java Thread is possible to expose to the user in question. I don't know what processing is going on in your application, but the danger is that you need to prevent user control of that lookup variable.
The call you're making is consistent with the single test in ValidatorTest.java, which is definitely a deficiency in code coverage on our behalf.
Now, there's an excellent chance that even if you use this call that Veracode might still flag it: the default file list in ESAPI.properties will need to be either truncated for your use case, or you'll have to create your own Validator rule for legal file extensions for your specific use case.
Which brings up the next bit: There's a lot of mischief that can happen in regards to file uploads.
In short, to be actually secure about file uploads will require more than what ESAPI currently offers, which is unfortunately, only an extension check. In your particular case, make sure you try some directory traversal attacks. And use that OWASP link to help analyze your application.
Given that the OP wants to clear the issue in Veracode, you would want to chain a couple calls:
ESAPI.validator().getValidDirectoryPath() and ESAPI.Validator.getValidFileName()
But be sure you've properly truncated the extension list in HttpUtilities.ApprovedUploadExtensions in validator.properties as the default list is too permissive, at least until we release 2.1.0.2.
I have to stress however that even with this particular combination there is absolutely nothing ESAPI does to prevent a user from renaming "netcat.exe" to "puppies.xlsx" and bypassing your validation check, that's why the rant on the first part of this answer.
ESAPI's file validation is NOT secure, it's quite simply better than nothing at all.
Doing this correctly requires more work than just using 1-2 calls to ESAPI.
DISCLAIMER: as of this writing I am the project co-lead for ESAPI.
You can change file name by sanitizing it as below code snippet:
private static String sanitizeFileName(String name) {
return name
.chars()
.mapToObj(i -> (char) i)
.map(c -> Character.isWhitespace(c) ? '_' : c)
.filter(c -> Character.isLetterOrDigit(c) || c == '-' || c == '_' || c == ':')
.map(String::valueOf)
.collect(Collectors.joining());
}

How to run vbscript function from java?

From java code i am able to run the vbscript by using this code
Runtime.getRuntime().exec("wscript C:\\ppt\\test1.vbs ");
But want to know how to call the method of vbscript from java..for example in test1.vbs
Set objPPT = CreateObject("PowerPoint.Application")
objPPT.Visible = True
Set objPresentation = objPPT.Presentations.Open("C:\ppt\Labo.ppt")
Set objSlideShow = objPresentation.SlideShowSettings.Run.View
sub ssn1()
objPPT.Run "C:\ppt\Labo.ppt!.SSN"
End sub
how to call only ssn1() method from java.Otherwise can we run the macro of a power point from java code..kindly help!!
This should make you happy :) Go to the WScript section : http://technet.microsoft.com/library/ee156618.aspx
Here's my idea... in your vbscript file, make your script listen to a command line parameter that would specify which method to call. Then, in Java, you could only have to use this parameter whenever you want to call a specific method in the file.
Otherwise, if you want to access powerpoint in java, you will need to access its API like you did in vbscript, which is possible if vbscript can do it but the approach / syntax may change.
I'm not so much into the visual basic script side, but if you can expose your visual basic script as a COM object, the you can access the methods of it from java by usage of frameworks such as for example com4j:
http://com4j.java.net/
The PowerPoint application object's .Run method lets you call any public subroutine or function in any open presentation or loaded add-in
This post answers the OP's question:
Otherwise can we run the macro of a power point from java code..kindly help!!
(but does not address the original vbscript question)
There's the JACOB library, which stands for Java COM Bridge, you can find here: http://sourceforge.net/projects/jacob-project/?source=directory
With it you can invoke Excel, Word, Outlook, PowerPoint application object model methods.
I've tried this with Excel but not PowerPoint. (This is just some sample code, one might want to make it more object oriented.)
public class Excel {
private static ActiveXComponent xl = null;
public static Init() {
try {
ComThread.InitSTA();
xl = ActiveXComponent.connectToActiveInstance("Excel.Application.14");
// 14 is Office 2010, if you don't know what version you can do "Excel.Application"
if (xl==null) {
// code to launch Excel if not running:
xl = new ActiveXComponent("Excel.Application");
Dispatch.put(xl, "Visible", Constants.kTrue);
}
}
catch (Exception e) {
ComThread.Release();
}
}
public static String Run(String vbName) {
// Variant v = Dispatch.call(xl, "Run", vbName); // using string name lookup
Variant v = Dispatch.call(xl, 0x103, vbName); // using COM offset
// return Dispatch.get(this, "Name").getString();
return v.getString();
}
public static Variant Run1p(String vbName, Object param) {
// Variant v = Dispatch.call(xl, "Run", vbName, param);
return Dispatch.call(xl, 0x103, vbName, param);
// return Dispatch.get(this, "Name").getString();
}
public static Worksheet GetActiveWorksheet () {
// Dispatch d = xl.getProperty("ActiveSheet").toDispatch();
Dispatch d = Dispatch.get(xl, 0x133).toDispatch ();
return d; // you may want to put a wrapper around this...
}
}
Notes:
For Excel, at least, to get Run to invoke a VBA macro/subroutine several things have to be true:
The Excel workbook containing the macro must be "Active" (i.e. must
be the ActiveWorkbook) otherwise Run will not find the VBA subroutine. (However the workbook does not have to be
screen visible!! This means you can call a VBA Macro that is in an add-in!).
You can then pass the name of the macro using the following syntax as a string literal:
VBAProjectName.VBAModuleName.SubroutineName
For COM object invocations, you can use the name lookup version or the id number version. The id numbers come from the published COM interfaces (which you can find in C++ header files, or possibly have JACOB look them up for you).
If you successfully did the connection to Excel, be sure to call ComThread.Release() when you're done. Put it in some appropriately surrounding finally. If the process of your Java code terminates without calling it, the COM reference count on Excel will be wrong, and the Excel process will never terminate, even after you exit the Excel application. Once that happens, needless to say, Excel starts to behave screwy then (when you try to use it next, it runs but will fail to load any plug-ins/add-ons). If that happens (as it can during debugging esp. if you are bypassing finally's for better debugging) you have to use the task manager to kill the Excel process.

Categories

Resources