folder structure is here
console output is here
I'd like to write a test class for the 2 methods below
package jfe;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import org.apache.commons.compress.archivers.sevenz.SevenZArchiveEntry;
import org.apache.commons.compress.archivers.sevenz.SevenZFile;
import org.apache.commons.compress.archivers.tar.TarArchiveEntry;
import org.apache.commons.compress.archivers.tar.TarArchiveInputStream;
import org.apache.commons.compress.utils.IOUtils;
public class JThreadFile {
/**
* uncompresses .tar file
* #param in
* #param out
* #throws IOException
*/
public static void decompressTar(String in, File out) throws IOException {
try (TarArchiveInputStream tin = new TarArchiveInputStream(new FileInputStream(in))){
TarArchiveEntry entry;
while ((entry = tin.getNextTarEntry()) != null) {
if (entry.isDirectory()) {
continue;
}
File curfile = new File(out, entry.getName());
File parent = curfile.getParentFile();
if (!parent.exists()) {
parent.mkdirs();
}
IOUtils.copy(tin, new FileOutputStream(curfile));
}
}
}
/**
* uncompresses .7z file
* #param in
* #param destination
* #throws IOException
*/
public static void decompressSevenz(String in, File destination) throws IOException {
//#SuppressWarnings("resource")
SevenZFile sevenZFile = new SevenZFile(new File(in));
SevenZArchiveEntry entry;
while ((entry = sevenZFile.getNextEntry()) != null){
if (entry.isDirectory()){
continue;
}
File curfile = new File(destination, entry.getName());
File parent = curfile.getParentFile();
if (!parent.exists()) {
parent.mkdirs();
}
FileOutputStream out = new FileOutputStream(curfile);
byte[] content = new byte[(int) entry.getSize()];
sevenZFile.read(content, 0, content.length);
out.write(content);
out.close();
}
sevenZFile.close();
}
}
I use testNG and try to read from a folder, filter the folder for certain extensions (.tar and .7z) feed those files to the uncompress methods and compare the result to the actualOutput folder with AssertEquals. I manage to read the file names from the folder (see console output) but can't feed them to decompressTar(String in, File out). Is this because "result" is a String array and I need a String? I have no clue how DataProvider of TestNG handles data. Any help would be appreciated :) Thank you :)
import java.io.File;
import java.io.FilenameFilter;
import java.io.IOException;
import org.testng.Assert;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
public class JThreadFileTest {
protected static final File ACT_INPUT = new File("c:\\Test\\TestInput\\"); //input directory
protected static final File EXP_OUTPUT = new File("c:\\Test\\ExpectedOutput\\"); //expected output directory
protected static final File TEST_OUTPUT = new File("c:\\Test\\TestOutput\\");
#DataProvider(name = "tarJobs")
public Object[] getTarJobs() {
//1
String[] tarFiles = ACT_INPUT.list(new FilenameFilter()
{
public boolean accept(File dir, String name)
{
return name.endsWith(".tar");
}
});
//2
Object[] result = new Object[tarFiles.length];
int i = 0;
for (String filename : tarFiles) {
result[i] = filename;
i++;
}
return result;
}
#Test(dataProvider = "tarJobs")
public void testTar(String result) throws IOException {
System.out.println("Running test" + result);
--> JThreadFile.decompressTar(result, TEST_OUTPUT);
Assert.assertEquals(TEST_OUTPUT, EXP_OUTPUT);
}
}
Related
I’m currently using PDFBox to read the text of a set of pdfs that I’ve inherited.
I’m only interested in reading the text, not making any changes to the file.
The code that works for most of the files is:
File pdfFile = myPath.toFile();
PDDocument document = PDDocument.load(pdfFile );
Writer sw = new StringWriter();
PDFTextStripper stripper = new PDFTextStripper();
stripper.setStartPage( 1 );
stripper.writeText( document, sw );
String documentText = sw.toString()
For most files, I wind up with the text in the documentText field.
But, for 3 of 24 files, the documentText content for the first file is “\r\n”, for the second “\r\n\r\n”, and for the third “\r\n\r\n\r\n:, But the three files are not consecutive. Multiple good files are between each of these files.
The File is derived from a java.nio.Path. The WindowsFileAttribute that is part of the Path has a size of 279K, so the file is not empty on disk.
I can open the file and view the data, and it looks like the other files that my code reads.
I’m using Java 8.0.121, and PDFBox 2.0.4. (this is the latest version, I believe.)
Any suggestions? Is there a better way to read the text? (I’m not interested in the formatting, or fonts used, just the text.)
Thanks.
Reading multiple PDF docs using pdfbox in java
package readwordfile;
import java.io.BufferedReader;
import org.apache.pdfbox.pdmodel.PDDocument;
import org.apache.pdfbox.text.PDFTextStripper;
import org.apache.pdfbox.text.TextPosition;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStreamWriter;
import java.io.Writer;
import java.util.ArrayList;
import java.util.List;
/**
* This is an example on how to extract words from PDF document
*
* #author saravanan
*/
public class GetWordsFromPDF extends PDFTextStripper {
static List<String> words = new ArrayList<String>();
public GetWordsFromPDF() throws IOException {
}
/**
* #param args
* #throws IOException If there is an error parsing the document.
*/
public static void main(String[] args) throws IOException {
String files;
// FileWriter fs = new FileWriter("C:\\Users\\saravanan\\Desktop\\New Text Document (2).txt");
// FileInputStream fstream1 = new FileInputStream("C:\\Users\\saravanan\\Desktop\\New Text Document (2).txt");
// DataInputStream in1 = new DataInputStream(fstream1);
// BufferedReader br1 = new BufferedReader(new InputStreamReader(in1));
String path = "C:\\Users\\saravanan\\Desktop\\New folder\\"; //local folder path name
File folder = new File(path);
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
if (listOfFiles[i].isFile()) {
files = listOfFiles[i].getName();
if (files.endsWith(".pdf") || files.endsWith(".PDF")) {
String nfiles = "C:\\Users\\saravanan\\Desktop\\New folder\\";
String fileName1 = nfiles + files;
System.out.print("\n\n" + files+"\n");
PDDocument document = null;
try {
document = PDDocument.load(new File(fileName1));
PDFTextStripper stripper = new GetWordsFromPDF();
stripper.setSortByPosition(true);
stripper.setStartPage(0);
stripper.setEndPage(document.getNumberOfPages());
Writer dummy = new OutputStreamWriter(new ByteArrayOutputStream());
stripper.writeText(document, dummy);
int x = 0;
System.out.println("");
for (String word : words) {
if (word.startsWith("xxxxxx")) { //here you can give your pdf doc starting word
x = 1;
}
if (x == 1) {
if (!(word.endsWith("YYYYYY"))) { //here you can give your pdf doc ending word
System.out.print(word + " ");
// fs.write(word);
} else {
x = 0;
break;
}
}
}
} finally {
if (document != null) {
document.close();
words.clear();
}
}
}
}
}
}
/**
* Override the default functionality of PDFTextStripper.writeString()
*
* #param str
* #param textPositions
* #throws java.io.IOException
*/
#Override
protected void writeString(String str, List<TextPosition> textPositions) throws IOException {
String[] wordsInStream = str.split(getWordSeparator());
if (wordsInStream != null) {
for (String word : wordsInStream) {
words.add(word); //store the pdf content into the List
}
}
}
}
I wrote java code to extract zip file in alfresco.
But through that code some zip files are not getting extracted properly.
And i have one more observation, that if i manually unzip that file and again manually create zip file then its working fine and its successfully getting extracted in alfresco site.
So i am not getting whats the problem exactly.
Is it problem with file or problem with my code or problem with zip file
Can anyone help me with this scenario...
Please refer below Logs for refrence...
2017-01-25 12:10:12,069 ERROR [tandf.ingestion.TransformExceptionService] [org.springframework.jms.listener.DefaultMessageListenerContainer#22-1] Unhandled Exception occured in ingestion pipeline: org.alfresco.error.AlfrescoRuntimeException: 00250134 Exception in Transaction.
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:542)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:326)
at com.ixxus.tandf.service.XmlZipExtractService.lambda$extract$4(XmlZipExtractService.java:235)
at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:548)
at
I am Using below code for extracting zip file..
private void unzipToNode(final String zipFilePath, final NodeRef destinationFolder, final String isbn, final String zipFileNoderef, final String assetType, final NodeRef ingestedNodeRef) throws IOException {
LOG.debug("Inside : Start :-> unzipToNode, zipFileNoderef : {}", zipFileNoderef);
String rootDisplayPath = nodeUtils.getDisplayPath(destinationFolder);
List<NodeRef> folderNodes = new ArrayList<>();
try (ZipFile zipFile = new ZipFile(zipFilePath); FileInputStream fis = new FileInputStream(zipFilePath); ZipInputStream zipInput = new ZipInputStream(fis);) {
ZipEntry entry = zipInput.getNextEntry();
int zipFileSize = zipFile.size();
LOG.info("{} : zipFileSize from zip api for zipFileNoderef : {}", zipFileSize, zipFileNoderef);
int zipManualFileCount = 0;
while (entry != null) {
zipManualFileCount++;
LOG.debug("Processing the zip entry : {}, zipFileNoderef : {}", entry.getName(), zipFileNoderef);
InputStream inputStream = zipFile.getInputStream(entry);
/** create or get final folder path for current entry */
NodeRef nodeRef = createOrGetFolderStructure(destinationFolder, entry);
folderNodes.add(nodeRef);
String name;
if (!entry.isDirectory()) {
name = getFileNameFromEntry(entry);
/** if zip entry is file, then create and write the new node in validation site */
createNodeOnValidationSite(nodeRef, name, inputStream, assetType, ingestedNodeRef, rootDisplayPath);
}
/** close current entry and fetch next one */
zipInput.closeEntry();
entry = zipInput.getNextEntry();
}
LOG.info("{} : zipManualFileCount from zip api for zipFileNoderef : {}", zipManualFileCount, zipFileNoderef);
/** Close the last entry */
zipInput.closeEntry();
}
for(NodeRef folderNode : folderNodes){
if(nodeUtils.isNodeEmpty(folderNode)){
LOG.debug("Found empty folder [{}] within .zip [{}]", folderNode, ingestedNodeRef);
/** if the folder is empty we need to copy all properties to it as well to archive it */
copyAllProperties(ingestedNodeRef, folderNode, rootDisplayPath, assetType);
}
}
LOG.debug("Inside : End :-> unzipToNode : zipFileNoderef : {}", zipFileNoderef);
}
For extract a zip file can you please try this code.
package org.alfresco.repo.action.executer;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.Serializable;
import java.util.Enumeration;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import java.util.zip.ZipException;
import org.alfresco.model.ContentModel;
import org.alfresco.repo.action.ParameterDefinitionImpl;
import org.alfresco.repo.action.executer.ActionExecuterAbstractBase;
import org.alfresco.repo.content.MimetypeMap;
import org.alfresco.service.cmr.action.Action;
import org.alfresco.service.cmr.action.ParameterDefinition;
import org.alfresco.service.cmr.dictionary.DataTypeDefinition;
import org.alfresco.service.cmr.dictionary.InvalidAspectException;
import org.alfresco.service.cmr.dictionary.InvalidTypeException;
import org.alfresco.service.cmr.model.FileExistsException;
import org.alfresco.service.cmr.model.FileFolderService;
import org.alfresco.service.cmr.model.FileInfo;
import org.alfresco.service.cmr.repository.ChildAssociationRef;
import org.alfresco.service.cmr.repository.ContentIOException;
import org.alfresco.service.cmr.repository.ContentReader;
import org.alfresco.service.cmr.repository.ContentService;
import org.alfresco.service.cmr.repository.ContentWriter;
import org.alfresco.service.cmr.repository.InvalidNodeRefException;
import org.alfresco.service.cmr.repository.MimetypeService;
import org.alfresco.service.cmr.repository.NodeRef;
import org.alfresco.service.cmr.repository.NodeService;
import org.alfresco.service.namespace.InvalidQNameException;
import org.alfresco.service.namespace.NamespaceService;
import org.alfresco.service.namespace.QName;
import org.alfresco.util.TempFileProvider;
import org.alfresco.web.bean.repository.Repository;
import org.apache.tools.zip.ZipEntry;
import org.apache.tools.zip.ZipFile;
/**
* Zip action executor
*
* #author Davide Taibi
*/
public class ZipActionExecuter extends ActionExecuterAbstractBase
{
public static final String NAME = "importzip";
public static final String PARAM_ENCODING = "encoding";
public static final String PARAM_DESTINATION_FOLDER = "destination";
private static final String TEMP_FILE_PREFIX = "alf";
private static final String TEMP_FILE_SUFFIX = ".zip";
/**
* The node service
*/
private NodeService nodeService;
/**
* The content service
*/
private ContentService contentService;
private MimetypeService mimetypeService;
private FileFolderService fileFolderService;
/**
* Sets the NodeService to use
*
* #param nodeService The NodeService
*/
public void setNodeService(NodeService nodeService)
{
this.nodeService = nodeService;
}
/**
* Sets the ContentService to use
*
* #param contentService The ContentService
*/
public void setContentService(ContentService contentService)
{
this.contentService = contentService;
}
public void setFileFolderService(FileFolderService fileFolderService) {
this.fileFolderService = fileFolderService;
}
public void setMimetypeService(MimetypeService mimetypeService) {
this.mimetypeService = mimetypeService;
}
private void extractFile(ZipFile archive,String mExtractToDir){
String fileName;
String destFileName;
byte[] buffer = new byte[16384];
mExtractToDir=mExtractToDir+File.separator;
try {
for ( Enumeration e = archive.getEntries(); e.hasMoreElements(); )
{
ZipEntry entry = (ZipEntry)e.nextElement();
if ( ! entry.isDirectory() )
{
fileName = entry.getName();
fileName = fileName.replace('/', File.separatorChar);
destFileName = mExtractToDir + fileName;
File destFile = new File(destFileName);
String parent = destFile.getParent();
if ( parent != null )
{
File parentFile = new File(parent);
if ( ! parentFile.exists() )
parentFile.mkdirs();
}
InputStream in = archive.getInputStream(entry);
OutputStream out = new FileOutputStream(destFileName);
int count;
while ( (count = in.read(buffer)) != -1 )
out.write(buffer, 0, count );
in.close();
out.close();
}else{
File newdir = new File( mExtractToDir + entry.getName() );
newdir.mkdir();
}
}
} catch (ZipException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
private void deleteDir(File dir){
File elenco=new File(dir.getPath());
for (File file:elenco.listFiles()){
if (file.isFile())
file.delete();
else
deleteDir(file);
}
dir.delete();
}
public void executeImpl(Action ruleAction, NodeRef actionedUponNodeRef) {
String spaceType;
NodeRef parentNodeRef;
FileInfo fileInfo;
NodeRef nodeRef;
if (this.nodeService.exists(actionedUponNodeRef) == true) {
ContentReader reader = this.contentService.getReader(
actionedUponNodeRef, ContentModel.PROP_CONTENT);
if (reader != null) {
if (MimetypeMap.MIMETYPE_ZIP.equals(reader.getMimetype())) {
File zFile = null;
ZipFile zipFile = null;
try {
spaceType = ContentModel.TYPE_FOLDER.toString();
String dirName = ""+this.nodeService.getProperty(actionedUponNodeRef, ContentModel.PROP_NAME);
dirName=dirName.substring(0,dirName.indexOf(".zip"));
parentNodeRef = (NodeRef)ruleAction.getParameterValue(PARAM_DESTINATION_FOLDER);// this.nodeService.getRootNode(Repository.getStoreRef());
fileInfo = fileFolderService.create(
parentNodeRef,
dirName,
Repository.resolveToQName(spaceType));
nodeRef = fileInfo.getNodeRef();
zFile = TempFileProvider.createTempFile(
TEMP_FILE_PREFIX, TEMP_FILE_SUFFIX);
reader.getContent(zFile);
zipFile = new ZipFile(zFile, "Cp437");
File tempDir=new File(TempFileProvider.getTempDir().getPath()+File.separator+dirName);
if (tempDir.exists()) deleteDir(tempDir);
extractFile(zipFile,tempDir.getPath());
importFile(tempDir.getPath(),nodeRef);
deleteDir(tempDir);
} catch (ContentIOException e) {
e.printStackTrace();
} catch (InvalidNodeRefException e) {
e.printStackTrace();
} catch (FileExistsException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
private void importFile(String dir,NodeRef root){
File elenco=new File(dir);
for (File file:elenco.listFiles()){
try {
if (file.isFile()){
InputStream contentStream = new FileInputStream(file);
// assign name
String fileName=file.getName();
Map<QName, Serializable> contentProps = new HashMap<QName, Serializable>();
contentProps.put(ContentModel.PROP_NAME, fileName);
// create content node
ChildAssociationRef association = nodeService
.createNode(
root,
ContentModel.ASSOC_CONTAINS,
QName.createQName(
NamespaceService.CONTENT_MODEL_PREFIX,
fileName),
ContentModel.TYPE_CONTENT,
contentProps);
NodeRef content = association.getChildRef();
// add titled aspect (for Web Client display)
Map<QName, Serializable> titledProps = new HashMap<QName, Serializable>();
titledProps.put(ContentModel.PROP_TITLE, fileName);
titledProps.put(ContentModel.PROP_DESCRIPTION,fileName);
nodeService.addAspect(content,
ContentModel.ASPECT_TITLED,
titledProps);
ContentWriter writer = contentService.getWriter(content,
ContentModel.PROP_CONTENT, true);
writer.setMimetype(mimetypeService.guessMimetype(file.getAbsolutePath()));
writer.setEncoding("Cp437");
writer.putContent(contentStream);
}else{
String spaceType = ContentModel.TYPE_FOLDER.toString();
FileInfo fileInfo = fileFolderService.create(
root,
file.getName(),
Repository.resolveToQName(spaceType));
importFile(file.getPath(),fileInfo.getNodeRef());
}
} catch (InvalidTypeException e) {
e.printStackTrace();
} catch (InvalidAspectException e) {
e.printStackTrace();
} catch (InvalidQNameException e) {
e.printStackTrace();
} catch (ContentIOException e) {
e.printStackTrace();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (InvalidNodeRefException e) {
e.printStackTrace();
} catch (FileExistsException e) {
e.printStackTrace();
}
}
}
protected void addParameterDefinitions(List<ParameterDefinition> paramList)
{
paramList.add(new ParameterDefinitionImpl(PARAM_DESTINATION_FOLDER, DataTypeDefinition.NODE_REF,
true, getParamDisplayLabel(PARAM_DESTINATION_FOLDER)));
paramList.add(new ParameterDefinitionImpl(PARAM_ENCODING, DataTypeDefinition.TEXT,
true, getParamDisplayLabel(PARAM_ENCODING)));
}
}
Moreover the file importzip-action-messages.properties :
importzip.title=Import Zip File
importzip.description=This Action import zip file into alfresco
importzip.aspect-name.display-label=The name of the aspect to apply to the node.
and add the following line to …-context.xml, (you can use action-services-context.xml)
<bean id="importzip" class="org.alfresco.repo.action.executer.ZipActionExecuter" parent="action-executer">
<property name="nodeService">
<ref bean="NodeService"></ref>
</property>
<property name="contentService">
<ref bean="ContentService" />
</property>
<property name="mimetypeService">
<ref bean="MimetypeService"></ref>
</property>
<property name="fileFolderService">
<ref bean="FileFolderService"></ref>
</property>
</bean>
<bean id="importzip-messages" class="org.alfresco.i18n.ResourceBundleBootstrapComponent">
<property name="resourceBundles">
<list>
<value>org.alfresco.repo.action.importzip-action-messages</value>
</list>
</property>
</bean>
and finally add this line to my web-client-config-custom.xml
<config evaluator="string-compare" condition="Action Wizards">
<action-handlers>
<handler name="importzip" class="org.alfresco.web.bean.actions.handlers.ImportHandler" />
</action-handlers>
</config>
How would I programmatically unzip backed up data directly into a /data/data/com.appname folder from the SD Card?
I am able to directly unzip folders in the /data/data/com.appname folder using ES File Explorer, however I need to automate this via the app I'm developing.
I've tried the following code, Unfortunately I am only able to unzip to the SD Card and data folder of my app.
I'm guessing this is due to some form of app/folder security?
MainActivity.java
package fb.ziptester;
import android.provider.Settings;
import android.support.v7.app.AppCompatActivity;
import android.os.Bundle;
import android.view.View;
import android.widget.TextView;
public class MainActivity extends AppCompatActivity {
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
}
public void buttonOnClick(View v) {
TextView zipSource = (TextView)findViewById(R.id.zip_source);
TextView zipDest = (TextView)findViewById(R.id.zip_dest);
String sZipSource = "/storage/sdcard1/ACC/BU.zip";
String sZipDest = "/data/data/com.appname/";
zipSource.setText(sZipSource);
zipDest.setText(sZipDest);
UnzipUtility zipUtil = new UnzipUtility();
try {
zipUtil.unzip(sZipSource, sZipDest);
} catch(Exception ex) {
//TODO
}
}
}
package fb.ziptester;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.zip.ZipEntry;
import java.util.zip.ZipInputStream;
UnzipUtility.java
public class UnzipUtility {
/**
* Size of the buffer to read/write data
*/
private static final int BUFFER_SIZE = 4096;
/**
* Extracts a zip file specified by the zipFilePath to a directory specified by
* destDirectory (will be created if does not exists)
* #param zipFilePath
* #param destDirectory
* #throws IOException
*/
public void unzip(String zipFilePath, String destDirectory) throws IOException {
File destDir = new File(destDirectory);
if (!destDir.exists()) {
destDir.mkdir();
}
ZipInputStream zipIn = new ZipInputStream(new FileInputStream(zipFilePath));
ZipEntry entry = zipIn.getNextEntry();
// iterates over entries in the zip file
while (entry != null) {
String filePath = destDirectory + File.separator + entry.getName();
if (!entry.isDirectory()) {
// if the entry is a file, extracts it
extractFile(zipIn, filePath);
} else {
// if the entry is a directory, make the directory
File dir = new File(filePath);
dir.mkdir();
}
zipIn.closeEntry();
entry = zipIn.getNextEntry();
}
zipIn.close();
}
/**
* Extracts a zip entry (file entry)
* #param zipIn
* #param filePath
* #throws IOException
*/
private void extractFile(ZipInputStream zipIn, String filePath) throws IOException {
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(filePath));
byte[] bytesIn = new byte[BUFFER_SIZE];
int read = 0;
while ((read = zipIn.read(bytesIn)) != -1) {
bos.write(bytesIn, 0, read);
}
bos.close();
}
}
Any help would be much appreciated.
Thank you.
Hello everyone,
I am trying to to use Apache Lucene in my log parser project. Now i have used the bellow code but i am not able to import org.apache.lucene.analysis.SimpleAnalyzer
I have included the lucene4.1 jar
/*
* To change this template, choose Tools | Templates
* and open the template in the editor.
*/
package logsearchengine;
import java.io.File;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.store.FSDirectory;
/**
*
* #author maclean
*/
public class LogSearchEngine {
private int index(File indexDir, File dataDir, String suffix) throws Exception {
IndexWriter indexWriter = new IndexWriter(FSDirectory.open(indexDir), new SimpleAnalyzer(),true,IndexWriter.MaxFieldLength.LIMITED);
indexWriter.setUseCompoundFile(false);
indexDirectory(indexWriter, dataDir, suffix);
int numIndexed = indexWriter.maxDoc();
indexWriter.optimize();
indexWriter.close();
return numIndexed;
}
private void indexDirectory(IndexWriter indexWriter, File dataDir,
String suffix) throws IOException {
File[] files = dataDir.listFiles();
for (int i = 0; i < files.length; i++) {
File f = files[i];
if (f.isDirectory()) {
indexDirectory(indexWriter, f, suffix);
}
else {
indexFileWithIndexWriter(indexWriter, f, suffix);
}
}
}
private void indexFileWithIndexWriter(IndexWriter indexWriter, File f,
String suffix) throws IOException {
if (f.isHidden() || f.isDirectory() || !f.canRead() || !f.exists()) {
return;
}
if (suffix!=null && !f.getName().endsWith(suffix)) {
return;
}
System.out.println("Indexing file " + f.getCanonicalPath());
Document doc = new Document();
doc.add(new Field("contents", new FileReader(f)));
doc.add(new Field("filename", f.getCanonicalPath(),
Field.Store.YES, Field.Index.ANALYZED));
indexWriter.addDocument(doc);
}
/**
* #param args the command line arguments
*/
public static void main(String[] args) {
// TODO code application logic here
File indexDir = new File("C:/index/");
File dataDir = new File("C:/programs/eclipse/workspace/");
String suffix = "java";
SimpleFileIndexer indexer = new SimpleFileIndexer();
int numIndex = indexer.index(indexDir, dataDir, suffix);
System.out.println("Total files indexed " + numIndex);
}
}
I added only one jar file that is lucene-core-4.1.0.jar. I am not able to import sample analyzer As a result of which I cannot create index writer. What am i missing. I am new to lucene
You also need lucene-analyzers-common-4.1.0.jar.
I'm writing code to upload a file in FileNet.
A standalone java program to take the some inputs, and upload it in FileNet. I'm new to FileNet. Can you help me out, How to do it?
You can use Document.java provided by IBM for your activities and many other Java classes
package fbis.apitocasemanager;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import com.user.DocumentUtil;
public class Addfilescasemanager {
/**
* #param args
*/
public static void addfiles_toicm(String directory, String lFolderPath)
{
try {
DocumentUtil.initialize();
String path = directory;
System.out.println("This is the path:..............................."
+ path);
String file_name;
File folder = new File(directory);
File[] listOfFiles = folder.listFiles();
for (int i = 0; i < listOfFiles.length; i++)
{
if (listOfFiles[i].isFile())
{
file_name = listOfFiles[i].getName();
System.out.println(file_name);
String filePaths = directory + file_name;
// File file = new File("C:\\FNB\\att.jpg");
File file = new File(filePaths);
InputStream attStream = null;
attStream = new FileInputStream(file);
DocumentUtil.addDocumentWithStream(lFolderPath, attStream,
"image/jpeg", file_name, "Document");
}
}
} catch (FileNotFoundException e)
{
e.printStackTrace();
}
}//end of method
public static void addfile_toicm(File file_name, String lFolderPath)
{
try {
DocumentUtil.initialize();
InputStream attStream = null;
attStream = new FileInputStream(file_name);
DocumentUtil.addDocumentWithStream(lFolderPath, attStream,
"image/jpeg", file_name.getName(), "Document");
System.out.println("File added successfully");
} catch (Exception e)
{
System.out.println(e.getMessage());
}
}//end of method
public static void main(String nag[])
{
addfiles_toicm("E:\\FSPATH1\\BLR_14122012_001F1A\\","/IBM Case Manager/Solution Deployments/Surakshate Solution for form 2/Case Types/FISB_FactoriesRegistration/Cases/2012/12/06/16/000000100103");
}
}
and my DocumentUtil class is
package com.user;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.InputStream;
import javax.security.auth.Subject;
import com.filenet.api.collection.ContentElementList;
import com.filenet.api.constants.AutoClassify;
import com.filenet.api.constants.AutoUniqueName;
import com.filenet.api.constants.CheckinType;
import com.filenet.api.constants.DefineSecurityParentage;
import com.filenet.api.constants.RefreshMode;
import com.filenet.api.core.Connection;
import com.filenet.api.core.ContentTransfer;
import com.filenet.api.core.Document;
import com.filenet.api.core.Domain;
import com.filenet.api.core.Factory;
import com.filenet.api.core.Folder;
import com.filenet.api.core.ObjectStore;
import com.filenet.api.core.ReferentialContainmentRelationship;
import com.filenet.api.util.UserContext;
public class DocumentUtil {
public static ObjectStore objectStore = null;
public static Domain domain = null;
public static Connection connection = null;
public static void main(String[] args)
{
initialize();
/*
addDocumentWithPath("/FNB", "C:\\Users\\Administrator\\Desktop\\Sample.txt.txt",
"text/plain", "NNN", "Document");
*/
File file = new File("E:\\Users\\Administrator\\Desktop\\TT.txt");
FileInputStream fis = null;
try {
fis = new FileInputStream(file);
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
addDocumentWithStream("/FNB", fis, "text/plain", "My New Doc", "Document");
}
public static void initialize()
{
System.setProperty("WASP.LOCATION", "C:\\Progra~1\\IBM\\WebSphere\\AppServer\\profiles\\AppSrv01\\installedApps\\P8Node01Cell\\FileNetEngine.ear \\cews.war\\WEB-INF\\classes\\com\\filenet\\engine\\wsi");
System.setProperty("SECURITY.AUTH.LOGIN.CONFIG",
"C:\\Progra~1\\IBM\\WebSphere\\AppServer\\profiles\\AppSrv01\\installedApps\\P8Node01Cell\\FileNetEngine.ear\\client-download.war\\FileNet\\Download\\dap501.153\\jaas.conf.wsi");
System.setProperty(":SECURITY.AUTH.LOGIN.CONFIG",
"C:\\Progra~1\\IBM\\WebSphere\\AppServer\\profiles\\AppSrv01\\installedApps\\P8Node01Cell\\FileNetEngine.ear\\client-download.war\\FileNet\\Download\\dap501.153\\jaas.conf.wsi");
System.setProperty("java.security.auth.login.config","C:\\Progra~1\\IBM\\WebSphere\\AppServer\\java\\jre");
connection = Factory.Connection.getConnection(CEConnection.uri);
Subject sub = UserContext.createSubject(connection,
com.user.CEConnection.username, CEConnection.password,
CEConnection.stanza);
UserContext.get().pushSubject(sub);
domain = Factory.Domain.getInstance(connection, null);
objectStore = Factory.ObjectStore.fetchInstance(domain, "TARGET", null);
System.out.println("\n\n objectStore--> " + objectStore.get_DisplayName());
}
public static void addDocumentWithPath(String folderPath, String filePath,
String mimeType, String docName, String docClass) {
Folder folder = Factory.Folder.fetchInstance(objectStore,
folderPath, null);
System.out.println("\n\n Folder ID: " + folder.get_Id());
// Document doc = Factory.Document.createInstance(os, classId);
Document doc = CEUtil.createDocWithContent(new File(filePath), mimeType,
objectStore, docName, docClass);
doc.save(RefreshMode.REFRESH);
doc = CEUtil.createDocNoContent(mimeType, objectStore, docName, docClass);
doc.save(RefreshMode.REFRESH);
CEUtil.checkinDoc(doc);
ReferentialContainmentRelationship rcr = CEUtil.fileObject(objectStore, doc, folderPath);
rcr.save(RefreshMode.REFRESH);
}
public static void addDocumentWithStream(String folderPath,
InputStream inputStream, String mimeType,
String docName, String docClass) {
Folder folder = Factory.Folder.fetchInstance(objectStore,
folderPath, null);
System.out.println("\n\n Folder ID: " + folder.get_Id());
// Document doc = Factory.Document.createInstance(os, classId);
Document doc = Factory.Document.createInstance(objectStore, null);
ContentElementList contEleList = Factory.ContentElement.createList();
ContentTransfer ct = Factory.ContentTransfer.createInstance();
ct.setCaptureSource(inputStream);
ct.set_ContentType(mimeType);
ct.set_RetrievalName(docName);
contEleList.add(ct);
doc.set_ContentElements(contEleList);
doc.getProperties().putValue("DocumentTitle", docName);
doc.set_MimeType(mimeType);
doc.checkin(AutoClassify.AUTO_CLASSIFY, CheckinType.MAJOR_VERSION);
doc.save(RefreshMode.REFRESH);
ReferentialContainmentRelationship rcr = folder.file(doc,
AutoUniqueName.AUTO_UNIQUE, docName,
DefineSecurityParentage.DO_NOT_DEFINE_SECURITY_PARENTAGE);
rcr.save(RefreshMode.REFRESH);
/*
doc.save(RefreshMode.REFRESH);
doc = CEUtil.createDocNoContent(mimeType, objectStore, docName, docClass);
CEUtil.checkinDoc(doc);
ReferentialContainmentRelationship rcr = CEUtil.fileObject(objectStore, doc, folderPath);
rcr.save(RefreshMode.REFRESH);
*/
}
public static ObjectStore getObjecctStore()
{
if (objectStore != null) {
return objectStore;
}
// Make connection.
com.filenet.api.core.Connection conn = Factory.Connection
.getConnection(CEConnection.uri);
Subject subject = UserContext.createSubject(conn,
CEConnection.username, CEConnection.password, null);
UserContext.get().pushSubject(subject);
try {
// Get default domain.
Domain domain = Factory.Domain.getInstance(conn, null);
// Get object stores for domain.
objectStore = Factory.ObjectStore.fetchInstance(domain, "TARGET",
null);
System.out.println("\n\n Connection to Content Engine successful !!");
} finally {
UserContext.get().popSubject();
}
return objectStore;
}
}
The above answer is extremely good. Just wanted to save people some time but I don't have the points to comment so am adding this as an answer.
Eclipse wasted a lot of my time getting the above to work because it suggested the wrong classes to import. Here's the list of correct ones:
import com.filenet.api.collection.ContentElementList;
import com.filenet.api.constants.AutoClassify;
import com.filenet.api.constants.AutoUniqueName;
import com.filenet.api.constants.CheckinType;
import com.filenet.api.constants.DefineSecurityParentage;
import com.filenet.api.constants.RefreshMode;
import com.filenet.api.core.Document;
import com.filenet.api.core.ObjectStore;
import com.filenet.api.core.ContentTransfer;
import com.filenet.api.core.Folder;
import com.filenet.api.core.Factory;
import com.filenet.api.core.ReferentialContainmentRelationship;