Develop Reference

How to resolve a custom option in a protocol buffer FileDescriptor

I'm using protocol buffers 2.5 with Java. I have a proto file that defines a custom option. Another proto file uses that custom option. If I persist the corresponding FileDescriptorProto's and then read them and convert them to FileDescriptors, the reference to the custom option is manifested as unknown field. How do I cause that custom option to be resolved correctly?
Here's the code. I have two .proto files. protobuf-options.proto looks like this:
package options;
import "google/protobuf/descriptor.proto";
option java_package = "com.example.proto";
option java_outer_classname = "Options";
extend google.protobuf.FieldOptions {
optional bool scrub = 50000;
}
The imported google/protobuf/descriptor.proto is exactly the descriptor.proto that ships with Protocol Buffers 2.5.
example.proto looks like this:
package example;
option java_package = "com.example.protos";
option java_outer_classname = "ExampleProtos";
option optimize_for = SPEED;
option java_generic_services = false;
import "protobuf-options.proto";
message M {
optional int32 field1 = 1;
optional string field2 = 2 [(options.scrub) = true];
}
As you can see, field2 references the custom option defined by protobuf-options.proto.
The following code writes a binary-encoded version of all three protos to /tmp:
package com.example;
import com.google.protobuf.ByteString;
import com.google.protobuf.DescriptorProtos.FileDescriptorProto;
import com.google.protobuf.Descriptors.FileDescriptor;
import com.example.protos.ExampleProtos;
import java.io.FileOutputStream;
import java.io.OutputStream;
/**
*
*/
public class PersistFDs {
public void persist(final FileDescriptor fileDescriptor) throws Exception {
System.out.println("persisting "+fileDescriptor.getName());
try (final OutputStream outputStream = new FileOutputStream("/tmp/"+fileDescriptor.getName())) {
final FileDescriptorProto fileDescriptorProto = fileDescriptor.toProto();
final ByteString byteString = fileDescriptorProto.toByteString();
byteString.writeTo(outputStream);
}
for (final FileDescriptor dependency : fileDescriptor.getDependencies()) {
persist(dependency);
}
}
public static void main(String[] args) throws Exception {
final PersistFDs self = new PersistFDs();
self.persist(ExampleProtos.getDescriptor());
}
}
Finally, the following code loads those those protos from /tmp, converts them back into FileDescriptors, and then checks for the custom option on field2:
package com.example;
import com.google.protobuf.ByteString;
import com.google.protobuf.DescriptorProtos.FileDescriptorProto;
import com.google.protobuf.Descriptors.FieldDescriptor;
import com.google.protobuf.Descriptors.FileDescriptor;
import com.google.protobuf.UnknownFieldSet.Field;
import java.io.FileInputStream;
import java.io.InputStream;
/**
*
*/
public class LoadFDs {
public FileDescriptorProto loadProto(final String filePath) throws Exception {
try (final InputStream inputStream = new FileInputStream(filePath)) {
final ByteString byteString = ByteString.readFrom(inputStream);
final FileDescriptorProto result = FileDescriptorProto.parseFrom(byteString);
return result;
}
}
public static void main(final String[] args) throws Exception {
final LoadFDs self = new LoadFDs();
final FileDescriptorProto descriptorFDProto = self.loadProto("/tmp/google/protobuf/descriptor.proto");
final FileDescriptorProto optionsFDProto = self.loadProto("/tmp/protobuf-options.proto");
final FileDescriptorProto fakeBoxcarFDProto = self.loadProto("/tmp/example.proto");
final FileDescriptor fD = FileDescriptor.buildFrom(descriptorFDProto, new FileDescriptor[0]);
final FileDescriptor optionsFD = FileDescriptor.buildFrom(optionsFDProto, new FileDescriptor[] { fD });
final FileDescriptor fakeBoxcarFD = FileDescriptor.buildFrom(fakeBoxcarFDProto, new FileDescriptor[] { optionsFD });
final FieldDescriptor optionsFieldDescriptor = optionsFD.findExtensionByName("scrub");
if (optionsFieldDescriptor == null) {
System.out.println("Did not find scrub's FieldDescriptor");
System.exit(1);
}
final FieldDescriptor sFieldDescriptor = fakeBoxcarFD.findMessageTypeByName("M").findFieldByName("field2");
System.out.println("unknown option fields "+sFieldDescriptor.getOptions().getUnknownFields());
final boolean hasScrubOption = sFieldDescriptor.getOptions().hasField(optionsFieldDescriptor);
System.out.println("hasScrubOption: "+hasScrubOption);
}
}
When I run LoadFDs, it fails with this exception:
unknown option fields 50000: 1
Exception in thread "main" java.lang.IllegalArgumentException: FieldDescriptor does not match message type.
at com.google.protobuf.GeneratedMessage$ExtendableMessage.verifyContainingType(GeneratedMessage.java:812)
at com.google.protobuf.GeneratedMessage$ExtendableMessage.hasField(GeneratedMessage.java:761)
at com.example.LoadFDs.main(LoadFDs.java:42)
The options for FieldDescriptor for the s field ought to have a field for that custom option, but instead it has an unknown field. The field number and value on the unknown field are correct. It's just that the custom option is not getting resolved. How do I fix that?

You need to use FileDescriptorProto.parseFrom(byte[] data, ExtensionRegistryLite extensionRegistry, and explicitly create an ExtentionRegistry. Here's one way to create an extension registry:
ExtensionRegistry extensionRegistry = ExtensionRegistry.newInstance();
com.example.proto.Options.registerAllExtensions(extensionRegistry);
(where com.example.proto.Options is the compiled custom options class)
This obviously only works if you have access to the custom options compiled file on client side. I don't know if there's a way to serialize the extension and deserialize it on the client side.

Unexpected results moving files between folders

I am trying to add a very simple feature to a Java program. The feature I want to add simply moves all the files from two folders to a third "archive" folder. The code is simple and I understand it 100% the problem is only one of the folder's contents is being moved. I have went over the code with a fine-tooth comb and tried repasting the directory several times, nothing seems to work. If anyone could help me figure out why my 2nd folder's contents aren't being moved I would REALLY appreciate it.
FYI in order to test this code you need to add a couple folders to "My Documents".
"Pain008Files", "Camt54 Files" and "archive". Also you just need to add some type of text file to the Pain008 and Camt5 folder, it can only have a random letter just something that can be moved.
At runtime the Pain008Files folder correctly has all it's files moved to the archive folder. The Camt54 Files does not. The only problem I can think of is that perhaps the space in the Camt54 Files name is causing a problem but that doesn't make sense so I thought I would hold off on changing it till I get some help. Thanks in advance!
Main Class
package fileHandling;
public class moveTestMain
{
public static void main(String args[]){
GetUser gUser = new GetUser();
gUser.getUser();
MoveFiles mFiles = new MoveFiles();
mFiles.moveCamtFiles();
mFiles.movePainFiles();
}
}
Gets the user-name class
package fileHandling;
public class GetUser
{
public static String currentUser = null;
public void getUser(){
currentUser = System.getProperty("user.name");
}
}
Move the files class
package fileHandling;
import java.io.File;
import java.nio.file.Files;
import java.util.ArrayList;
import java.util.Arrays;
public class MoveFiles
{
public static ArrayList<File> pain008Files;
public static ArrayList<File> camt54Files;;
public void movePainFiles(){
File pain008File = new File("C:\\Users\\"+GetUser.currentUser+"\\Documents\\Pain008Files");
pain008Files = new ArrayList<File>(Arrays.asList(pain008File.listFiles()));
System.out.println(pain008Files);
for(int i = 0; i < pain008Files.size(); i++){
System.out.println("Test");
int cutAmount = GetUser.currentUser.length();
String fileName = pain008Files.get(i).toString().substring(33+cutAmount,pain008Files.get(i).toString().length());
System.out.println(fileName);
System.out.println(pain008Files.get(i).toString());
pain008Files.get(i).renameTo(new File("C:\\Users\\"+GetUser.currentUser+"\\Documents\\archive\\"+
"archivedPain_"+fileName));
}
}
public void moveCamtFiles(){
File camt54File = new File("C:\\Users\\"+GetUser.currentUser+"\\Documents\\Camt54 Files");
camt54Files = new ArrayList<File>(Arrays.asList(camt54File.listFiles()));
for(int i = 0; i < camt54Files.size(); i++){
int cutAmount = GetUser.currentUser.length();
String fileName = camt54Files.get(i).toString().substring(32+cutAmount,camt54Files.get(i).toString().length());
camt54Files.get(i).renameTo(new File("C:\\Users\\"+GetUser.currentUser+"\\Documents\\archive\\"+
"archivedCamt_"+fileName));
}
}

SHORT ANSWER:
Your code has some typo errors in routes or somewhere...
LONG ANSWER:
I adapted it to local testing in my computer and works fine.
public void movePainFiles() {
File pain008File = new File("C:\\tmp\\pain");
pain008Files = new ArrayList<File>(Arrays.asList(pain008File.listFiles()));
System.out.println(pain008Files);
for (int i = 0; i < pain008Files.size(); i++) {
System.out.println(pain008Files.get(i).toString());
pain008Files.get(i).renameTo(new File("C:\\tmp\\archive\\" + "archivedPain_" + pain008Files.get(i).getName()));
}
}
public void moveCamtFiles() {
File camt54File = new File("C:\\tmp\\camt");
camt54Files = new ArrayList<File>(Arrays.asList(camt54File.listFiles()));
for (int i = 0; i < camt54Files.size(); i++) {
System.out.println(camt54Files.get(i).toString());
camt54Files.get(i).renameTo(new File("C:\\tmp\\archive\\" + "archivedCamt_" + camt54Files.get(i).getName()));
}
}
OUTPUT:
C:\tmp\camt\xxx.pdf
C:\tmp\camt\yyy.pdf
C:\tmp\camt\zzz.pdf
[C:\tmp\pain\Q37024973.txt, C:\tmp\pain\Q37545784.txt]
C:\tmp\pain\Q37024973.txt
C:\tmp\pain\Q37545784.txt

Jacob Fatal Error

I am trying to build a Java Drag and Drop that works with Outlook emails. I've been using Jacob because of an inability to transfer data from Outlook to Java using standard AWT Event stuff. That said, all of the solutions I've pulled from here or other sites have been causing a fatal crash in Java. Here's the code:
import java.awt.dnd.*;
import javax.swing.*;
import java.awt.event.*;
import java.awt.*;
import java.awt.datatransfer.*;
import java.io.*;
import java.util.List;
import sun.awt.datatransfer.*;
import com.jacob.com.*;
import com.jacob.activeX.*;
public class D2 extends JFrame
{
private static final String DIR = "FILES";
private static void saveSelectedOutlookMails(String directory) {
Dispatch xl = new Dispatch("Outlook.Application");
//Dispatch selection = Dispatch.get(xl, "Selection").toDispatch();
System.out.println(xl);
System.out.println(xl==null);
//PROGRAM CRASHES AFTER THIS LINE
Dispatch explorer = Dispatch.get(xl,"ActiveExplorer").toDispatch();
System.out.println("explorer");
Object selection = Dispatch.get(explorer, "Selection").toDispatch();
Variant count = Dispatch.get(selection, "Count");
for (int mailIndex = 1; mailIndex <= count.toInt(); mailIndex++ ) {
Object mailItem = Dispatch.call(selection, "Item", new Variant(mailIndex)).toDispatch();
Variant senderName = Dispatch.get(mailItem, "SenderName");
Variant subject = Dispatch.get(mailItem, "Subject");
Variant body = Dispatch.get(mailItem, "Body");
String emailFileName = subject.toString() +".txt";
String fullPath = directory + "/" + emailFileName;
try {
File email = new File(fullPath);
PrintWriter writer = new PrintWriter( new FileWriter(email) );
writer.println("From: "+ senderName );
writer.println("Subject: "+ subject);
writer.println("");
writer.print( body );
writer.close();
}
catch (IOException e) {
System.out.println(e.getMessage());
//logger.error("IOException writing e-mail with subject: '"+ subject +"'", e);
continue;
}
Object attachments = Dispatch.get(mailItem, "Attachments").toDispatch();
Variant attachmentCount = Dispatch.get(attachments, "Count");
if ( attachmentCount.toInt() > 0 ) {
for( int attachmentIndex = 1; attachmentIndex<=attachmentCount.toInt(); attachmentIndex++ ) {
Object attachment = Dispatch.call(attachments, "Item", new Variant(attachmentIndex)).toDispatch();
Variant fileNameVariant = Dispatch.get(attachment, "FileName");
String fileName = fileNameVariant.toString();
Variant saveResult = Dispatch.call(attachment, "SaveAsFile", directory, "/", fileName);
}
}
}
}
public D2() throws Exception
{
this.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
this.setBounds(0,0,300,300);
this.setVisible(true);
DropTarget dropTarget=new DropTarget();
dropTarget.setComponent(this);
dropTarget.addDropTargetListener(new DropTargetAdapter()
{
public void drop(DropTargetDropEvent dtde){
saveSelectedOutlookMails(DIR);
}
});
}
public static void main(String[] args)
{
try{
new D2();
}catch(Exception e){
e.printStackTrace();
}
}
}

Ok, firstly, you are creating Outlook.Application in a way I've never seen before - I've only ever seen the Active X Component way:
e.g.
ActiveXComponent xl = new ActiveXComponent("Outlook.Application");
Dispatch explorer = Dispatch.get(xl,"ActiveExplorer").toDispatch();
Dispatch selection = Dispatch.get(explorer, "Selection").toDispatch();
Variant count = Dispatch.get(selection, "Count");
// loop over selected mail items.
for (int mailIndex = 1; mailIndex <= count.getInt(); mailIndex++ ) {
Dispatch mailItem = Dispatch.call(selection, "Item", new Variant(mailIndex)).toDispatch();
Variant subject = Dispatch.get(mailItem, "Subject");
// .... and so on
}
Secondly, your code is not saving the mail, its pulling out all the fields and attachments and trying to recreate the mail, which seems inaccurate at best and will only be an approximation of what the message was.
Why don't you just use the COM objects to SaveAs the whole .msg file to disk? Then if you need to access it again you can use something like JDIC to launch Outlook and pop up the message in its original glory, including all attachments?

My guess is that you are trying to get PROPERTY with name "ActiveExplorer", but it is a method! Here is documentation of that particuliar method https://msdn.microsoft.com/en-us/library/office/ff870017.aspx . Try to use .call() Jacob method to invoke METHODS on MS objects.

Indirectly referenced from required .class file, even the build path is set correctly apache POI..?

import java.io.*;
import org.apache.poi.poifs.filesystem.*;
import org.apache.poi.hpsf.DocumentSummaryInformation;
import org.apache.poi.hwpf.*;
import org.apache.poi.hwpf.extractor.*;
import org.apache.poi.hwpf.usermodel.HeaderStories;
public class ReadDocFileInJava {
public static void main(String[] args)
{
/**This is the document that you want to read using Java.**/
String fileName = "C:\\Documents and Settings\\kushalp\\Desktop\\Test.doc";
/**Method call to read the document (demonstrate some useage of POI)**/
readMyDocument(fileName);
}
public static void readMyDocument(String fileName)
{
POIFSFileSystem fs = null;
try
{
fs = new POIFSFileSystem(new FileInputStream(fileName));
HWPFDocument doc = new HWPFDocument(fs);
/** Read the content **/
readParagraphs(doc);
int pageNumber=1;
/** We will try reading the header for page 1**/
readHeader(doc, pageNumber);
/** Let's try reading the footer for page 1**/
readFooter(doc, pageNumber);
/** Read the document summary**/
readDocumentSummary(doc);
}
catch (Exception e)
{
e.printStackTrace();
}
}
public static void readParagraphs(HWPFDocument doc) throws Exception
{
WordExtractor we = new WordExtractor(doc);
/**Get the total number of paragraphs**/
String[] paragraphs = we.getParagraphText();
System.out.println("Total Paragraphs: "+paragraphs.length);
for (int i = 0; i < paragraphs.length; i++)
{
System.out.println("Length of paragraph "+(i +1)+": "+ paragraphs[i].length());
System.out.println(paragraphs[i].toString());
}
}
public static void readHeader(HWPFDocument doc, int pageNumber)
{
HeaderStories headerStore = new HeaderStories( doc);
String header = headerStore.getHeader(pageNumber);
System.out.println("Header Is: "+header);
}
public static void readFooter(HWPFDocument doc, int pageNumber)
{
HeaderStories headerStore = new HeaderStories( doc);
String footer = headerStore.getFooter(pageNumber);
System.out.println("Footer Is: "+footer);
}
public static void readDocumentSummary(HWPFDocument doc)
{
DocumentSummaryInformation summaryInfo=doc.getDocumentSummaryInformation();
String category = summaryInfo.getCategory();
String company = summaryInfo.getCompany();
int lineCount=summaryInfo.getLineCount();
int sectionCount=summaryInfo.getSectionCount();
int slideCount=summaryInfo.getSlideCount();
System.out.println("---------------------------");
System.out.println("Category: "+category);
System.out.println("Company: "+company);
System.out.println("Line Count: "+lineCount);
System.out.println("Section Count: "+sectionCount);
System.out.println("Slide Count: "+slideCount);
}
I am getting error in these two packages
import org.apache.poi.poifs.filesystem.*;
import org.apache.poi.hpsf.DocumentSummaryInformation;
The type org.apache.poi.poifs.filesystem.POIFSFileSystem cannot be resolved. It is indirectly referenced from required .class files
I have attached a snapshot my java build path...since the program require
poi-scratchpad-3.2-FINAL-20081019.jar
It is correctly set in java build path..then why I am getting such error ..help..!!

found the solution it requires poi-3.7.jar
http://mvnrepository.com/artifact/org.apache.poi/poi/3.7

You have two problems. One is that you're using Apache POI 3.2, which dates from 6 years ago and there have been a huge number of bug fixes since then
Second problem, you've missed some of the POI jars and their dependencies. See the components page for details. Basically though, to use HWPF, you need both the poi and poi-scratchpad jars on your classpath

delete entire released directories located at C:\Users\user\.m2\repository\org\apache\poi.
Right Click on project > Maven then Update project.

Structures of files in java

Hello I am new in Java so I would like some guidelines on how to organize files of a project in java. Currently I am building an app with GUI so I want two files one that relates to the GUI and another file that relates to any functions that are called from the first. Right now I've named the second file Utilities.java and looks like this:
package Directory;
import java.io.BufferedInputStream;
import java.io.BufferedOutputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import javax.swing.JProgressBar;
import javax.swing.JTextArea;
import javax.swing.SwingWorker;
public class Utilities {
public class FileCopy{
private File Source;
private File Destination;
private long totalBytes=0L;
FileCopy(File source,File destination){
Source=source;
Destination=destination;
retrieveTotalBytes(source);
}
File getSource(){return Source;}
File getDestination(){return Destination;}
Long gettotalBytes(){return totalBytes;}
private void retrieveTotalBytes(File sourceFile)
{
if(sourceFile.isDirectory()==false){
totalBytes = sourceFile.length();
}
else{
File[] files = sourceFile.listFiles();
for(File file : files)
{
if(file.isDirectory()) retrieveTotalBytes(file);
else totalBytes += file.length();
}
}
System.out.print("Done retrieving");
}
}
public class Copy extends SwingWorker<Void,Integer>
{
File src,dest;
InputStream in;
OutputStream out;
JProgressBar progressBar;
JProgressBar all;
JTextArea txt;
public int progress;
//private int all_progress;
private long totalBytes = 0L;
private long copiedBytes = 0L;
boolean keepStructure=false;
boolean delete=false;
public Copy(File source,File dst,JProgressBar br,JTextArea text,boolean keep,boolean delete)
{
src=source;
dest=dst;
progressBar=br;
txt=text;
progressBar.setValue(0);
progressBar.setVisible(true);
txt.setText("Copying " + src.getName());
keepStructure=keep;
this.delete=delete;
}
#Override
public Void doInBackground() throws Exception
{
txt.setText(src.getName());
//retrieveTotalBytes(src);
copyFiles(src, dest);
return null;
}
#Override
public void process(java.util.List<Integer> chunks)
{
for(int i : chunks)
{
progressBar.setValue(i);
}
}
#Override
public void done()
{
setProgress(100);
}
public String GetParent(String input){
short pos=(short) input.lastIndexOf(File.separatorChar);
return input.substring(0, pos);
}
private void copyFiles(File sourceFile, File targetFile) throws IOException
{
if(sourceFile.isDirectory())
{
if(!targetFile.exists()) targetFile.mkdirs();
String[] filePaths = sourceFile.list();
for(String filePath : filePaths)
{
File destFile;
File srcFile = new File(sourceFile, filePath);
if(keepStructure==true)
destFile= new File(targetFile, filePath);
else{
String filepath2=GetParent(dest.toString())+File.separatorChar+srcFile.getName();
destFile=new File(filepath2);
}
System.out.print("\n\n name="+destFile.toString()+"\n");
System.out.print("dest to string =" +GetParent(dest.toString()) + " srcFile.getName()="+srcFile.getName()+"\n" );
copyFiles(srcFile, destFile);
}
}
else
{
BufferedInputStream bis = new BufferedInputStream(new FileInputStream(sourceFile));
BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream(targetFile));
long fileBytes = sourceFile.length();
long soFar = 0L;
int theByte;
while((theByte = bis.read()) != -1)
{
bos.write(theByte);
setProgress((int) (copiedBytes++ * 100 / totalBytes));
publish((int) (soFar++ * 100 / fileBytes));
}
bis.close();
bos.close();
if(delete==true)
sourceFile.delete();
publish(100);
txt.setText("Copying " + src.getName() + "complete");
}
}
}
}
Question 1:
Notice that in that file the I have two subclasses {FileCopy,Copy} that are completely different. Is that a good way to organize the code or should a move each class on each own file?
Question 2:
Also, in my main i try to create and object from each class but I do something wrong. I've added the import of the file but when I try to create an object e.g.
Copy worker = new Copy(source,dest,progressBar,textArea, keep_Structure,false);
I receive this error:
No enclosing instance of type Utilities is accessible. Must qualify
the allocation with an enclosing instance of type Utilities (e.g.
x.new A() where x is an instance of Utilities).

In Java you should moslty (at least while you are still learning the basics) keep each class in its own file.
You have 3 (and not 2) classes in your file : Utilities, FileCopy and Copy, with the later two being inner classes of Utilities (the Utilities class does nothing on its own). This is why you can't instanciate Copy without first instantiating Utilities.
I think you should have a package named utilities, containing two files : FileCopy.java and Copy.java each containing their own class. If you want a way to differentiate between parts of your application, this is a good way to start : have a package containing all gui-related classes and another one for the rest of the application.
This should solve your error.
The official tutorials on nested classes : http://docs.oracle.com/javase/tutorial/java/javaOO/nested.html