I am trying to fetch parents from below xml with stax, note that there can be multiple parents and childs at the same structure.
<sm:Structure>
<sm:Parent>
<sm:parentCode>PARENT-CODE-1</sm:parentCode>
<sm:parentName>PARENT-NAME-1</sm:parentName>
</sm:Parent>
<sm:Child>
<sm:childCode>CHILD-CODE-1</sm:childCode>
<sm:childName>CHILD-NAME-1</sm:childName>
<sm:parentCode>PARENT-CODE-1</sm:parentCode>
</sm:Child>
</sm:Structure>
With the following code:
XMLStreamReader xmlr = null;
try {
XMLInputFactory xmlif = XMLInputFactory.newInstance();
File file = new File(fileName);
xmlr = xmlif.createXMLStreamReader(new FileReader(file));
JAXBContext context = JAXBContext.newInstance(type);
Unmarshaller unmarshaller = context.createUnmarshaller();
while (xmlr.hasNext() && (!xmlr.isStartElement() || !xmlr.getLocalName().equalsIgnoreCase(localName))) {
xmlr.next();
}
List<T> objectList = new ArrayList<>();
int numberOfRead = 0, chunkIndex = 0;
StopWatch watch = new StopWatch();
watch.start();
while (xmlr.getEventType() == XMLStreamConstants.START_ELEMENT) {
if (numberOfRead == chunkSize) {
chunkConsumer.consumeChunk(objectList, chunkIndex, chunkSize, breakChildProcessOnParentError));
numberOfRead = 0;
chunkIndex++;
objectList = new ArrayList<>();
}
JAXBElement<T> objNode = unmarshaller.unmarshal(xmlr, type);
T obj = objNode.getValue();
objectList.add(obj);
numberOfRead++;
if (xmlr.getEventType() == XMLStreamConstants.CHARACTERS) {
xmlr.next();
}
}
if (numberOfRead != 0) {
chunkConsumer.consumeChunk(objectList, chunkIndex, chunkSize, breakChildProcessOnParentError));
}
watch.stop();
log.info("Time Elapsed to trigger all " + type.getName() + "-Chunk-Consumers: " +
watch.toSplitString());
} catch (Exception e) {
throw new CustomException("Error during " + type.getName()+ "-Chunk-Consumer process.", e);
} finally {
try {
if (xmlr != null) {
xmlr.close();
}
} catch (Exception exception) {
log.error(exception.getMessage(), exception);
}
}
It reads the parent twice interestingly, first one is okay, second one without parentName. Why it is parsing twice?
Missed a condition in while, adding resolved my problem.
while (xmlr.getEventType() == XMLStreamConstants.START_ELEMENT && xmlr.getLocalName().equalsIgnoreCase(localName)) {
Related
in my custom processor i need to update config file(it is placed in one of nifi folder, it is not original config ) based on data which i get from upstream connection but i can't get desired result what should i do?
is there any way i can controll upstream connection flowfile
destination i mean when i make debugging i use this function fileQueue.drainTo(file, batchSize) in this part "file " was null
here is one more thing i am interested in, on this line
session.commit() i always get errors that :"trnasfer operation isn't specified" and tanks to it whole operation is failed ?
it seems that when i try to make session.exportTo() operations it doesnt write anything in dotCopyFile, how should i fix it?
should i clean fileQueue and then make session.commit()?
FlowFile flowfile;
#Override
public void onTrigger(ProcessContext context, ProcessSession session) throws ProcessException {
final String conflictResponse = context.getProperty(CONFLICT_RESOLUTION).getValue();
final ArrayList value = new ArrayList<>();
flowfile = session.get();
if (flowfile == null) {
return;
}
value.add(flowfile.getAttribute("filename"));
session.remove(flowfile);
final File directory = new File(context.getProperty(DIRECTORY).evaluateAttributeExpressions().getValue());
final boolean keepingSourceFile = context.getProperty(KEEP_SOURCE_FILE).asBoolean();
final ComponentLog logger = getLogger();
if (fileQueue.size() < 100) {
final long pollingMillis = context.getProperty(POLLING_INTERVAL).asTimePeriod(TimeUnit.MILLISECONDS);
if ((queueLastUpdated.get() < System.currentTimeMillis() - pollingMillis) && listingLock.tryLock()) {
try {
final Set<File> listing = performListing(directory, fileFilterRef.get(), context.getProperty(RECURSE).asBoolean().booleanValue());
queueLock.lock();
try {
listing.removeAll(inProcess);
if (!keepingSourceFile) {
listing.removeAll(recentlyProcessed);
}
fileQueue.clear();
fileQueue.addAll(listing);
queueLastUpdated.set(System.currentTimeMillis());
recentlyProcessed.clear();
if (listing.isEmpty()) {
context.yield();
}
} finally {
queueLock.unlock();
}
} finally {
listingLock.unlock();
}
}
}
final int batchSize = context.getProperty(BATCH_SIZE).asInteger();
final List<File> file = new ArrayList<>(batchSize);
queueLock.lock();
try {
fileQueue.drainTo(file, batchSize);
if (file.isEmpty()) {
return;
} else {
inProcess.addAll(file);
}
} finally {
queueLock.unlock();
}
//make xml parsing
DocumentBuilderFactory dbFactory = DocumentBuilderFactory.newInstance();
try {
dBuilder = dbFactory.newDocumentBuilder();
} catch (ParserConfigurationException e) {
e.printStackTrace();
}
try {
File f = file.get(0);
doc = dBuilder.parse(f);
} catch (IOException e) {
e.printStackTrace();
} catch (org.xml.sax.SAXException e) {
e.printStackTrace();
}
NodeList nList = doc.getElementsByTagName("localAttributes");
for (int temp = 0; temp < nList.getLength(); temp++) {
Node nNode = nList.item(temp);
if (nNode.getNodeType() == Node.ELEMENT_NODE) {
Element eElement = (Element) nNode;
start = eElement.getElementsByTagName("start").item(0).getTextContent();
startDate = eElement.getElementsByTagName("startDate").item(0).getTextContent();
endDate = eElement.getElementsByTagName("endDate").item(0).getTextContent();
patch = eElement.getElementsByTagName("patch").item(0).getTextContent();
runAs = eElement.getElementsByTagName("runAs").item(0).getTextContent();
makeVersion = eElement.getElementsByTagName("makeVersion").item(0).getTextContent();
///parameter = eElement.getElementsByTagName("parameter").item(0).getTextContent();
}
}
final ListIterator<File> itr = file.listIterator();
FlowFile flowFile1 = null;
try {
final Path directoryPath = directory.toPath();
while (itr.hasNext()) {
final File files = itr.next();
final Path filePath = files.toPath();
final Path relativePath = directoryPath.relativize(filePath.getParent());
String relativePathString = relativePath.toString() + "/";
if (relativePathString.isEmpty()) {
relativePathString = "./";
}
final Path absPath = filePath.toAbsolutePath();
final String absPathString = absPath.getParent().toString() + "/";
flowFile1 = session.create();
final long importStart = System.nanoTime();
flowFile1 = session.importFrom(filePath, keepingSourceFile, flowFile1);
final long importNanos = System.nanoTime() - importStart;
final long importMillis = TimeUnit.MILLISECONDS.convert(importNanos, TimeUnit.NANOSECONDS);
flowFile1 = session.putAttribute(flowFile1, CoreAttributes.FILENAME.key(), files.getName());
flowFile1 = session.putAttribute(flowFile1, CoreAttributes.PATH.key(), relativePathString);
flowFile1 = session.putAttribute(flowFile1, CoreAttributes.ABSOLUTE_PATH.key(), absPathString);
Map<String, String> attributes = getAttributesFromFile(filePath);
if (attributes.size() > 0) {
flowFile1 = session.putAllAttributes(flowFile1, attributes);
}
InputStream ffStream = session.read(flowFile1);
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = builderFactory.newDocumentBuilder();
Document xmlDocument = builder.parse(ffStream);
XPath xPath = XPathFactory.newInstance().newXPath();
XPathExpression myNodeList = (XPathExpression) xPath.compile("/localAttributes");
Node nodeGettingChanged = (Node) myNodeList.evaluate(xmlDocument, XPathConstants.NODE);
NodeList childNodes = nodeGettingChanged.getChildNodes();
boolean make=false;
for (int i = 0; i != childNodes.getLength(); ++i) {
Node child = childNodes.item(i);
if (!(child instanceof Element))
continue;
if(child.getNodeName().equals("start")){
String date;
for(int j=0;j<value.size();j++) {
if(value.get(j).length()>10){
date=value.get(j).substring(0,10);
}
else{
date=value.get(j);
}
if (date == child.getFirstChild().getTextContent()){
child.getFirstChild().setNodeValue(addOneDay(child.getFirstChild().getTextContent()));
make=true;
}
}
}
if(make){
if(child.getNodeName().equals("runAs")){
child.getFirstChild().setNodeValue("true");
}
}
}
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = null;
transformer = transformerFactory.newTransformer();
DOMSource source = new DOMSource(xmlDocument);
String path = "C:/Users/user/Desktop/nifi-1.3.0/nifi-assembly/target/nifi-1.3.0-bin/nifi-1.3.0/1/conf.xml";
File f = new File(path);
StreamResult file1 = new StreamResult(f);
try {
transformer.transform(source, file1);
} catch (TransformerException e) {
e.printStackTrace();
}
session.write(flowFile1, new StreamCallback() {
#Override
public void process(InputStream inputStream, OutputStream outputStream) throws IOException {
TransformerFactory transformerFactory = TransformerFactory.newInstance();
Transformer transformer = null;
try {
transformer = transformerFactory.newTransformer();
} catch (TransformerConfigurationException e) {
e.printStackTrace();
}
DOMSource source = new DOMSource(xmlDocument);
ffStream.close();
ByteArrayOutputStream bos = new ByteArrayOutputStream();
StreamResult result = new StreamResult(bos);
try {
transformer.transform(source, result);
} catch (TransformerException e) {
e.printStackTrace();
}
byte[] array = bos.toByteArray();
outputStream.write(array);
}
});
Path tempDotCopyFile = null;
try {
final Path rootDirPath = Paths.get("C://Users//user//Desktop//try2//nifi-1.3.0//1");
final Path tempCopyFile = rootDirPath.resolve("." + flowFile1.getAttribute(CoreAttributes.FILENAME.key()));
final Path copyFile = rootDirPath.resolve(flowFile1.getAttribute(CoreAttributes.FILENAME.key()));
if (!Files.exists(rootDirPath)) {
if (context.getProperty(CREATE_DIRS).asBoolean()) {
Files.createDirectories(rootDirPath);
} else {
flowFile1 = session.penalize(flowFile1);
session.transfer(flowFile1,REL_FAILURE);
logger.error("Penalizing {} and routing to 'failure' because the output directory {} does not exist and Processor is "
+ "configured not to create missing directories", new Object[]{flowFile1, rootDirPath});
return;
}
}
final Path dotCopyFile = tempCopyFile;
tempDotCopyFile = dotCopyFile;
Path finalCopyFile = copyFile;
final Path finalCopyFileDir = finalCopyFile.getParent();
if (Files.exists(finalCopyFileDir)) { // check if too many files already
final int numFiles = finalCopyFileDir.toFile().list().length;
if (numFiles >= 34) {
flowFile1 = session.penalize(flowFile1);
logger.warn("Penalizing {} and routing to 'failure' because the output directory {} has {} files, which exceeds the "
+ "configured maximum number of files", new Object[]{flowFile1, finalCopyFileDir, numFiles});
session.transfer(flowFile1,REL_FAILURE);
return;
}
}
if (Files.exists(finalCopyFile)) {
switch (conflictResponse) {
case REPLACE_RESOLUTION:
Files.delete(finalCopyFile);
logger.info("Deleted {} as configured in order to replace with the contents of {}", new Object[]{finalCopyFile, flowFile1});
break;
case IGNORE_RESOLUTION:
session.transfer(flowFile1, REL_SUCCESS);
logger.info("Transferring {} to success because file with same name already exists", new Object[]{flowFile1});
return;
case FAIL_RESOLUTION:
flowFile1 = session.penalize(flowFile1);
logger.warn("Penalizing {} and routing to failure as configured because file with the same name already exists", new Object[]{flowFile1});
session.transfer(flowFile1,REL_FAILURE);
return;
default:
break;
}
}
session.exportTo(flowFile1, dotCopyFile, false);
final String permissions = "-rwxrwx---";
if (permissions != null && !permissions.trim().isEmpty()) {
try {
String perms = stringPermissions(permissions);
if (!perms.isEmpty()) {
Files.setPosixFilePermissions(dotCopyFile, PosixFilePermissions.fromString(perms));
}
} catch (Exception e) {
logger.warn("Could not set file permissions to {} because {}", new Object[]{permissions, e});
}
}
boolean renamed = false;
for (int i = 0; i < 10; i++) { // try rename up to 10 times.
if (dotCopyFile.toFile().renameTo(finalCopyFile.toFile())) {
renamed = true;
break;// rename was successful
}
Thread.sleep(100L);// try waiting a few ms to let whatever might cause rename failure to resolve
}
if (!renamed) {
if (Files.exists(dotCopyFile) && dotCopyFile.toFile().delete()) {
logger.debug("Deleted dot copy file {}", new Object[]{dotCopyFile});
}
throw new ProcessException("Could not rename: " + dotCopyFile);
} else {
logger.info("Produced copy of {} at location {}", new Object[]{flowFile1, finalCopyFile});
}
/*session.getProvenanceReporter().send(flowFile, finalCopyFile.toFile().toURI().toString(), stopWatch.getElapsed(TimeUnit.MILLISECONDS));
session.transfer(flowFile, REL_SUCCESS);*/
session.getProvenanceReporter().receive(flowFile1, files.toURI().toString(), importMillis);
//session.transfer(flowFile1, REL_SUCCESS);
session.remove(flowFile1);
} catch (final Throwable t) {
if (tempDotCopyFile != null) {
try {
Files.deleteIfExists(tempDotCopyFile);
} catch (final Exception e) {
logger.error("Unable to remove temporary file {} due to {}", new Object[]{tempDotCopyFile, e});
}
}
flowFile1 = session.penalize(flowFile1);
logger.error("Penalizing {} and transferring to failure due to {}", new Object[]{flowFile1, t});
session.transfer(flowFile1,REL_FAILURE);
}
}
if (!isScheduled()) { // if processor stopped, put the rest of the files back on the queue.
queueLock.lock();
try {
while (itr.hasNext()) {
final File nextFile = itr.next();
fileQueue.add(nextFile);
inProcess.remove(nextFile);
}
} finally {
queueLock.unlock();
}
}
} catch (IOException e1) {
e1.printStackTrace();
} catch (TransformerConfigurationException e1) {
e1.printStackTrace();
} catch (ParserConfigurationException e1) {
e1.printStackTrace();
} catch (XPathExpressionException e1) {
e1.printStackTrace();
} catch (org.xml.sax.SAXException e) {
e.printStackTrace();
}
session.commit();
}
I'm trying to add 360° metadata to a mp4 file with this library : https://github.com/copiousfreetime/mp4parser
after have check the code, i had created this :
public void injectSphericalMetaV2(TrackBox trackBox) throws IOException {
String sphericalVideoGlobalMetadata = "<rdf:SphericalVideo xmlns:rdf=\"http://www.w3.org/1999/02/22-rdf-syntax-ns#\"xmlns:GSpherical=\"http://ns.google.com/videos/1.0/spherical/\"><GSpherical:Spherical>true</GSpherical:Spherical><GSpherical:Stitched>true</GSpherical:Stitched><GSpherical:StitchingSoftware>GIROPTIC 360 CAM</GSpherical:StitchingSoftware><GSpherical:ProjectionType>equirectangular</GSpherical:ProjectionType></rdf:SphericalVideo>";
String UUID_ID = "FFCC8263-F855-4A93-8814-587A02521FDD";
UserBox sphericalAtom = new UserBox(UUID_ID.getBytes());
sphericalAtom.setData(sphericalVideoGlobalMetadata.getBytes());
trackBox.addBox(sphericalAtom);
}
and i execute it here :
protected TrackBox createTrackBox(Track track, Movie movie, Map<Track, int[]> chunks) {
TrackBox trackBox = new TrackBox();
TrackHeaderBox tkhd = new TrackHeaderBox();
tkhd.setEnabled(true);
tkhd.setInMovie(true);
tkhd.setMatrix(track.getTrackMetaData().getMatrix());
tkhd.setAlternateGroup(track.getTrackMetaData().getGroup());
tkhd.setCreationTime(track.getTrackMetaData().getCreationTime());
if (track.getEdits() == null || track.getEdits().isEmpty()) {
tkhd.setDuration(track.getDuration() * getTimescale(movie) / track.getTrackMetaData().getTimescale());
} else {
long d = 0;
for (Edit edit : track.getEdits()) {
d += (long) edit.getSegmentDuration();
}
tkhd.setDuration(d * track.getTrackMetaData().getTimescale());
}
tkhd.setHeight(track.getTrackMetaData().getHeight());
tkhd.setWidth(track.getTrackMetaData().getWidth());
tkhd.setLayer(track.getTrackMetaData().getLayer());
tkhd.setModificationTime(new Date());
tkhd.setTrackId(track.getTrackMetaData().getTrackId());
tkhd.setVolume(track.getTrackMetaData().getVolume());
trackBox.addBox(tkhd);
trackBox.addBox(createEdts(track, movie));
MediaBox mdia = new MediaBox();
trackBox.addBox(mdia);
MediaHeaderBox mdhd = new MediaHeaderBox();
mdhd.setCreationTime(track.getTrackMetaData().getCreationTime());
mdhd.setDuration(track.getDuration());
mdhd.setTimescale(track.getTrackMetaData().getTimescale());
mdhd.setLanguage(track.getTrackMetaData().getLanguage());
mdia.addBox(mdhd);
HandlerBox hdlr = new HandlerBox();
mdia.addBox(hdlr);
hdlr.setHandlerType(track.getHandler());
MediaInformationBox minf = new MediaInformationBox();
if (track.getHandler().equals("vide")) {
try {
injectSphericalMetaV2(trackBox);
} catch (IOException e) {
e.printStackTrace();
}
minf.addBox(new VideoMediaHeaderBox());
} else if (track.getHandler().equals("soun")) {
minf.addBox(new SoundMediaHeaderBox());
} else if (track.getHandler().equals("text")) {
minf.addBox(new NullMediaHeaderBox());
} else if (track.getHandler().equals("subt")) {
minf.addBox(new SubtitleMediaHeaderBox());
} else if (track.getHandler().equals("hint")) {
minf.addBox(new HintMediaHeaderBox());
} else if (track.getHandler().equals("sbtl")) {
minf.addBox(new NullMediaHeaderBox());
}
// dinf: all these three boxes tell us is that the actual
// data is in the current file and not somewhere external
DataInformationBox dinf = new DataInformationBox();
DataReferenceBox dref = new DataReferenceBox();
dinf.addBox(dref);
DataEntryUrlBox url = new DataEntryUrlBox();
url.setFlags(1);
dref.addBox(url);
minf.addBox(dinf);
Box stbl = createStbl(track, movie, chunks);
minf.addBox(stbl);
mdia.addBox(minf);
LOG.logDebug("done with trak for track_" + track.getTrackMetaData().getTrackId());
return trackBox;
}
these 2 methods are in my MP4BuilderV2 and i call it here a TrimActivity like this :
Container out = new DefaultMp4BuilderV2().build(movie);
MovieHeaderBox mvhd = Path.getPath(out, "moov/mvhd");
mvhd.setMatrix(Matrix.ROTATE_180);
if (!dst.exists()) {
dst.createNewFile();
} else {
dst.delete();
dst.createNewFile();
}
FileOutputStream fos = new FileOutputStream(dst);
WritableByteChannel fc = fos.getChannel();
try {
out.writeContainer(fc);
}catch (Exception e) {
Log.e("erreur", e.toString());
}finally {
fc.close();
fos.close();
file.close();
}
file.close();
But in the last try/catch i get this error : java.nio.BufferOverflowException
If someone has the solution thanks in advance
I was with the same problem as you, searching for a solution I found a meta tag insertion implementation that worked for me, maybe it helps you.
Sorry I'm new here but I have and issue I'm hoping someone can help me solve.
This code runs perfect while in eclipse, but after compiled it say's:
java.lang.IllegalArgumentException: URI is not hierarchical
Any help would be appropriated, thanks!
public void loadMods(String pkg) throws IllegalAccessException, IllegalArgumentException, InvocationTargetException, InstantiationException {
List<Class<?>> classes = getClasses(pkg);
for(Class<?> c : classes) {
for (Method m : c.getMethods()) {
Object o = null;
o = c.newInstance();
if (m.getName().contains("load")) {
m.setAccessible(true);
m.invoke(o);
}
}
}
}
public static List<Class<?>> getClasses(String pkg) {
String pkgname = pkg;
List<Class<?>> classes = new ArrayList<Class<?>>();
File directory = null;
String fullPath;
String relPath = pkgname.replace('.', '/');
URL resource = ClassLoader.getSystemClassLoader().getResource(relPath);
if (resource == null) {
throw new RuntimeException("No resource for " + relPath);
}
fullPath = resource.getFile();
try {
directory = new File(resource.toURI());
} catch (URISyntaxException e) {
throw new RuntimeException(pkgname + " (" + resource + ") invalid URL / URI.", e);
} catch (IllegalArgumentException e) {
directory = null;
}
if (directory != null && directory.exists()) {
String[] files = directory.list();
for (int i = 0; i < files.length; i++) {
if (files[i].endsWith(".class")) {
String className = pkgname + '.' + files[i].substring(0, files[i].length() - 6);
try {
classes.add(Class.forName(className));
} catch (ClassNotFoundException e) {
throw new RuntimeException("ClassNotFoundException loading " + className);
}
} else {
String pkgnamex = pkgname + '.' + files[i];
List<Class<?>> classesx = new ArrayList<Class<?>>();
File directoryx = null;
String fullPathx;
String relPathx = pkgnamex.replace('.', '/');
URL resourcex = ClassLoader.getSystemClassLoader().getResource(relPathx);
if (resourcex == null) {
throw new RuntimeException("No resource for " + relPathx);
}
fullPathx = resourcex.getFile();
try {
directoryx = new File(resourcex.toURI());
} catch (URISyntaxException e) {
throw new RuntimeException(pkgnamex + " (" + resourcex + ") invalid URL / URI.", e);
} catch (IllegalArgumentException e) {
directoryx = null;
}
if (directoryx != null && directoryx.exists()) {
String[] filesx = directoryx.list();
for (int ix = 0; ix < filesx.length; ix++) {
if (filesx[ix].endsWith(".class")) {
String classNamex = pkgnamex + '.' + filesx[ix].substring(0, filesx[ix].length() - 6);
try {
classes.add(Class.forName(classNamex));
} catch (ClassNotFoundException e) {
throw new RuntimeException("ClassNotFoundException loading " + classNamex);
}
}
}
}
}
}
}
return classes;
}
When you run the code from within Eclipse it uses the compiled classes (by default in folder 'target'). However if you run the code from external normally you use a JAR file created by Eclipse.
And this problem arises when referencing something inside the JAR which is explained by the linked questions.
In short: URIs in the file system are syntactically correct. An URI referencing something into a JAR is no more a valid URI.
Well what i am trying to achieve is to save pairs of words in a sentence and if the word is already there , i am trying to save a list of words against one.
To save the pairing as there could many millions as my data set file is very large , i opted for orientdb. I dont know if i am approaching it correctly but orientdb is very slow. After 8 hours of running it has only made pairs for 12000 sentences.
As far as i have checked the major slowdown was in browsing cluster.
Attached is my code, please if ant one can give any pointers over my approach.
public static void main(String[] args) {
// TODO Auto-generated method stub
Main m = new Main();
m.openDatabase();
m.readFile("train_v2.txt");
m.closeDatabase();
}
}
class Main {
ODatabaseDocumentTx db;
Map<String, Object> index;
List<Object> list = null;
String pairing[];
ODocument doc;
Main() {
}
public void closeDatabase() {
if (!db.isClosed()) {
db.close();
}
}
void openDatabase() {
db = new ODatabaseDocumentTx("local:/databases/model").open("admin",
"admin");
doc = new ODocument("final");
}
public void readFile(String filename) {
InputStream ins = null; // raw byte-stream
Reader r = null; // cooked reader
int i = 1;
BufferedReader br = null; // buffered for readLine()
try {
String s;
ins = new FileInputStream(filename);
r = new InputStreamReader(ins, "UTF-8"); // leave charset out
// for
// default
br = new BufferedReader(r);
while ((s = br.readLine()) != null) {
System.out.println("" + i);
createTermPair(s.replaceAll("[^\\w ]", "").trim());
i++;
}
} catch (Exception e) {
System.err.println(e.getMessage()); // handle exception
} finally {
closeDatabase();
if (br != null) {
try {
br.close();
} catch (Throwable t) { /* ensure close happens */
}
}
if (r != null) {
try {
r.close();
} catch (Throwable t) { /* ensure close happens */
}
}
if (ins != null) {
try {
ins.close();
} catch (Throwable t) { /* ensure close happens */
}
}
}
}
private void createTermPair(String phrase) {
phrase = phrase + " .";
String[] word = phrase.split(" ");
for (int i = 0; i < word.length - 1; i++) {
if (!word[i].trim().equalsIgnoreCase("")
&& !word[i + 1].trim().equalsIgnoreCase("")) {
String wordFirst = word[i].toLowerCase().trim();
String wordSecond = word[i + 1].toLowerCase().trim();
String pair = wordFirst + " " + wordSecond;
checkForPairAndWrite(pair);
}
}
}
private void checkForPairAndWrite(String pair) {
try {
pairing = pair.trim().split(" ");
if (!pairing[1].equalsIgnoreCase(" ")) {
index = new HashMap<String, Object>();
for (ODocument docr : db.browseCluster("final")) {
list = docr.field(pairing[0]);
}
if (list == null) {
list = new ArrayList<>();
}
list.add("" + pairing[1]);
if (list.size() >= 1)
index.put(pairing[0], list);
doc.fields(index);
doc.save();
}// for (int i = 0; i < list.size(); i++) {
// System.out.println("" + list.get(i));
// }
} catch (Exception e) {
}
return;
}
}
I am getting the following issues after scanning the code using fortify....
1>path manipulation issue:
private MimeMessage prepareMessage(EmailMessage req) throws EmailProviderException {
long start=System.currentTimeMillis(),finish=0;
try {
MimeMessage message = emailSender.createMimeMessage();
// Create a multipart message
MimeMessageHelper helper = new MimeMessageHelper(message, true);
// Set email addresses
helper.setFrom(convertAddress(req.getFromAddress()));
helper.setTo(convertAddress(req.getToAddress()));
helper.setCc(convertAddress(req.getCcAddress()));
helper.setBcc(convertAddress(req.getBccAddress()));
// Set subject and body
helper.setSubject(req.getEmailSubject());
String emailBody = req.getEmailBody();
String emailMime = req.getEmailMimeType();
MimeBodyPart messagePart = new MimeBodyPart();
DataSource bodyDataSource = new ByteArrayDataSource(emailBody, emailMime);
messagePart.setDataHandler(new DataHandler(bodyDataSource));
helper.getMimeMultipart().addBodyPart(messagePart);
// Add attachments
List<EmailAttachment> lAttach = req.getEmailAttachment();
if (lAttach != null) {
for (EmailAttachment attachMnt: lAttach) {
DataSource dSource = new ByteArrayDataSource(attachMnt
.getContent(), attachMnt
.getMimeType());
helper.addAttachment(attachMnt.getFileName(), dSource);
}
}
finish=System.currentTimeMillis();
statsLogger.info(new FedExLogEntry("prepareMessage took {0}ms",new Object[]{finish-start}));
return message;
} catch (Exception e) {
// Covers MessagingException, IllegalStateException, IOException, MailException
String emsg = new StringBuilder("Unable to prepare smtp message.")
.append("\n").append(req.toString()).toString();
logger.warn(emsg, e);
throw new EmailProviderException(emsg, e);
}
}
Null dereference issues
issue 1.
public byte[] toByteArray(Notification nd) throws EmailProviderException {
String message = null;
try {
JAXBContext jc = JAXBContext.newInstance(nd.getClass());
if (jc != null) {
Marshaller m = jc.createMarshaller();
StringWriter sw = new StringWriter();
m.marshal(nd, sw);
message = sw.toString();
}
} catch (JAXBException e) {
throw new EmailProviderException("Unable to convert NDS notification to byte array.", e);
}
return message.getBytes();
}
null dereference issue 2..
private void addLastUpdatedHours(
List<LocationHoursForADate> masterHours, List<LocationHoursWithSearchedDate> hoursToAdd, Map<String,String> scheduleTypeIncludesMap){
String prevScheduleTypeCode = null;
String prevHourTypeCode = null;
Date searchedDate = null;
// Build map of locationHours to searchDates
List<LocationHours> locationHours = null;
Map<Date, List<LocationHours>> locationHoursSearchDatesMap = new HashMap<Date, List<LocationHours>>();
for(LocationHoursWithSearchedDate locationHoursWithSearchedDate : hoursToAdd) {
if(scheduleTypeIncludesMap.containsKey(locationHoursWithSearchedDate.getLocationHoursPK().getScheduleTypeCd())) {
searchedDate = locationHoursWithSearchedDate.getLocationHoursPK().getSearchedDate();
locationHours = locationHoursSearchDatesMap.get(searchedDate);
if(locationHours==null) {
locationHours = new ArrayList<LocationHours>();
locationHoursSearchDatesMap.put(searchedDate,locationHours);
}
locationHours.add(locationHoursWithSearchedDate.createLocationHours());
}
}
for(Map.Entry<Date,List<LocationHours>> entry : locationHoursSearchDatesMap.entrySet()) {
prevHourTypeCode = null;
prevScheduleTypeCode = null;
searchedDate = entry.getKey();
for(LocationHours hour: entry.getValue()){
// new ST & new 01, add it
if((prevScheduleTypeCode == null) && (prevHourTypeCode == null)){
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
else{
//same ST
if(prevScheduleTypeCode.equals(hour.getLocationHoursPK().getScheduleTypeCd())){
// same 01, skip this schedule
if(prevHourTypeCode.equals(hour.getHoursType().getHoursTypeCd())){
continue;
}
else { //new 01, add it
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
}
else{ //new ST, add it
masterHours.add(new LocationHoursForADate(searchedDate, hour));
prevScheduleTypeCode = hour.getLocationHoursPK().getScheduleTypeCd();
prevHourTypeCode = hour.getLocationHoursPK().getHoursTypeCd();
}
}
}
}
}
}
Well, the second case (null deference 1) is easy to spot.
The problem is that if there is an exception in the try-catch block then the object message will be null, so the final instruction return message.getBytes() will raise a NullPointerException.
One way to avoid it is changing the return statement like this:
return message != null ? message.getBytes() : null;
But maybe, you will prefer to raise the exception...