I am using this bencode https://github.com/dampcake/bencode to decode a torrent file. I am having an issue :
the encoded torrent file looks something like this :
d8:announce21:http://127.0.0.1: ....etc..... piece lengthi65536e6:pieces28300:a�ډ|E���� ���#-14 .....etc........
The thing is that when I enter this string in the 'decoder', I get an error because of the � symbols.
Here is my question: should I stop decoding just before those symbols ? Or is the whole string necessary to properly decode the .torrent file ?
From what I've read, I need to stop the decoding at the end of the dictionary, ie. when I encounter the final 'e', but I don't know how to properly identify it..
Thanks
UPDATE:
Here is my code :
byte[] to_decode = null;
try {
Path path = Paths.get("/user/.../file.torrent");
to_decode = Files.readAllBytes(path);
} catch (IOException e) {
System.out.println(e.toString());
}
//System.out.println(to_decode.toString());
Bencode bencode = new Bencode();
Map<String, Object> dict = bencode.decode(to_decode, Type.DICTIONARY);
System.out.println(dict);
When I run it, I have no errors but this kind of output:
f<�>�0�1FT���n" ......etc...... 4'}$�Q�3�� Җk�, private=0}}
So, considering the brackets, I think the output is a dictionary but not in a usable format, and I can't seem to make it work
Any advice ?
Following specification https://en.wikipedia.org/wiki/Bencode 6:pieces28300:a means there is a 28300 bytes long string. So it should be parsed too. You should stop at the end of dictionary but it is not in 6:pieces28300:a (it is at the end).
Both length and � indicate that you are dealing with binary data. You do not specify error, neither source code you are using, but you are using wrong character encoding. So check character encoding of encoded torrent file data and make sure to use same encoding in your Bencode constructor.
I am trying to output curly quotes in an HTML file that I am generating in Freemarker. The template file contains:
Kevin’s
When the HTML file is generated, it comes out as:
Kevin?s
At first I thought that the issue was happening during the generation of the HTML file. But I was able to track down the conversion to when the template was read in. Does anyone know how to prevent Freemarker from doing this conversion when reading the template? My code for the conversion:
// Freemarker configuration object
Configuration cfg = new Configuration(new Version(2, 3, 21));
try
{
cfg.setDirectoryForTemplateLoading(new File("templates"));
cfg.setDefaultEncoding("UTF-8");
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.HTML_DEBUG_HANDLER);
// Load template from source folder
Template template = cfg.getTemplate("curly.html");
template.setEncoding("UTF-8");
// Build the data-model
Map<String, Object> data = new HashMap<String, Object>();
// Console output
Writer out = new OutputStreamWriter(System.out);
template.process(data, out);
out.flush();
}
catch (IOException e)
{
e.printStackTrace();
}
catch (TemplateException e)
{
e.printStackTrace();
}
If the template file indeed contains Kevin’s, then the out would be Kevin’s too (as FreeMarker doesn't resolve HTML entities), so I suppose you mean that the character with that code is there as one character. In that case, the most probable culprit has nothing to do with FreeMarker: new OutputStreamWriter(System.out). You have omitted the encoding parameter of the constructor there, so it will use the system default encoding. Even if you do specify that, your console have a fixed encoding (which is not necessarily the system default BTW). So try to write the output into a file by explicitly specifying UTF-8 for the OutputStreamWriter. If the output will be still wrong, then check if you have indeed used UTF-8 to create the template file, and for reading the output file.
BTW, that template.setEncoding is not necessary. Remove it.
I am using JasperReports and DynamicReports with this piece of java code to create a report in pdf format which contains utf-8 characters, the problem is generated pdf file does not contain utf-8 characters at all, like if they have been replaced with "". is there any thing that i should be aware of when using OutputStream to create a utf-8 file?
public void toPdf(String path){
OutputStream outHtml;
try {
outHtml = new FileOutputStream(path);
jasperBuilder.toPdf(outHtml);
} catch (Exception e1) {
logger.error("failed to create PDF", e1);
}
}
this may be notable that creating XLS and HTML file faces no such problem.
note that there are lots of lines of code under jasperBuilder.toPdf(outHtml); that i have traced and no where in those lines my utf-8 characters are being eliminated. so i guess the devil is in outHtml = new FileOutputStream(path);
I managed to solve it. It was a font and encoding problem. Just followed tutorial here, but change <pdfEncoding>UTF-8</pdfEncoding> to <pdfEncoding>Identity-H</pdfEncoding> in fonts.xml
<fontFamilies>
<fontFamily name="FreeUniversal">
<normal>/home/moien/tahoma.ttf</normal>
<bold>/home/moien/tahoma.ttf</bold>
<italic>/home/moien/tahoma.ttf</italic>
<boldItalic>/home/moien/tahoma.ttf</boldItalic>
<pdfEncoding>Identity-H</pdfEncoding>
<pdfEmbedded>true</pdfEmbedded>
</fontFamily>
</fontFamilies>
Now I have another challenge to solve, making font URL relative!
A FileOutputStream is completely agnostic of the "stuff" that gets written to it. It just writes bytes. If characters are being eliminated or mangled, then this is being caused by whatever is generating the bytes to be written to the stream.
In this case, my money would be on the way that you have configured / used the jasperBuilder object prior to running this code.
I am trying to fetch the below xml from db using a java method but I am getting an error
Code used to parse the xml
DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
DocumentBuilder db = dbf.newDocumentBuilder();
InputSource is = new InputSource(new ByteArrayInputStream(cond.getBytes()));
Document doc = db.parse(is);
Element elem = doc.getDocumentElement();
// here we expect a series of <data><name>N</name><value>V</value></data>
NodeList nodes = elem.getElementsByTagName("data");
TableID jobId = new TableID(_processInstanceId);
Job myJob = Job.queryByID(_clientContext, jobId, true);
if (nodes.getLength() == 0) {
log(Level.DEBUG, "No data found on condition XML");
}
for (int i = 0; i < nodes.getLength(); i++) {
// loop through the <data> in the XML
Element dataTags = (Element) nodes.item(i);
String name = getChildTagValue(dataTags, "name");
String value = getChildTagValue(dataTags, "value");
log(Level.INFO, "UserData/Value=" + name + "/" + value);
myJob.setBulkUserData(name, value);
}
myJob.save();
The Data
<ContactDetails>307896043</ContactDetails>
<ContactName>307896043</ContactName>
<Preferred_Completion_Date>
</Preferred_Completion_Date>
<service_address>A-End Address: 1ST HELIERST HELIERJT2 3XP832THE CABLES 1 POONHA LANEST HELIER JE JT2 3XP</service_address>
<ServiceOrderId>315473043</ServiceOrderId>
<ServiceOrderTypeId>50</ServiceOrderTypeId>
<CustDesiredDate>2013-03-20T18:12:04</CustDesiredDate>
<OrderId>307896043</OrderId>
<CreateWho>csmuser</CreateWho>
<AccountInternalId>20100333</AccountInternalId>
<ServiceInternalId>20766093</ServiceInternalId>
<ServiceInternalIdResets>0</ServiceInternalIdResets>
<Primary_Offer_Name action='del'>MyMobile Blue £44.99 [12 month term]</Primary_Offer_Name>
<Disc_Reason action='del'>8</Disc_Reason>
<Sup_Offer action='del'>80000257</Sup_Offer>
<Service_Type action='del'>A-01-00</Service_Type>
<Priority action='del'>4</Priority>
<Account_Number action='del'>0</Account_Number>
<Offer action='del'>80000257</Offer>
<msisdn action='del'>447797142520</msisdn>
<imsi action='del'>234503184</imsi>
<sim action='del'>5535</sim>
<ocb9_ARM action='del'>false</ocb9_ARM>
<port_in_required action='del'>
</port_in_required>
<ocb9_mob action='del'>none</ocb9_mob>
<ocb9_mob_BB action='del'>
</ocb9_mob_BB>
<ocb9_LandLine action='del'>
</ocb9_LandLine>
<ocb9_LandLine_BB action='del'>
</ocb9_LandLine_BB>
<Contact_2>
</Contact_2>
<Acc_middle_name>
</Acc_middle_name>
<MarketCode>7</MarketCode>
<Acc_last_name>Port_OUT</Acc_last_name>
<Contact_1>
</Contact_1>
<Acc_first_name>.</Acc_first_name>
<EmaiId>
</EmaiId>
The ERROR
org.apache.xerces.impl.io.MalformedByteSequenceException: Invalid byte 1 of 1-byte UTF-8 sequence.
I read in some threads it's because of some special characters in the xml.
How to fix this issue ?
How to fix this issue ?
Read the data using the correct character encoding. The error message means that you are trying to read the data as UTF-8 (either deliberately or because that is the default encoding for an XML file that does not specify <?xml version="1.0" encoding="somethingelse"?>) but it is actually in a different encoding such as ISO-8859-1 or Windows-1252.
To be able to advise on how you should do this I'd have to see the code you're currently using to read the XML.
Open the xml in notepad
Make sure you dont have extra space at the beginning and end of the document.
Select File -> Save As
select save as type -> All files
Enter file name as abcd.xml
select Encoding - UTF-8 -> Click Save
Try:
InputStream inputStream= // Your InputStream from your database.
Reader reader = new InputStreamReader(inputStream,"UTF-8");
InputSource is = new InputSource(reader);
is.setEncoding("UTF-8");
saxParser.parse(is, handler);
If it's anything else than UTF-8, just change the encoding part for the good one.
I was getting the xml as a String and using xml.getBytes() and getting this error. Changing to xml.getBytes(Charset.forName("UTF-8")) worked for me.
I had the same problem in my JSF application which was having a comment line containing some special characters in the XMHTL page. When I compared the previous version in my eclipse it had a comment,
//Some � special characters found
Removed those characters and the page loaded fine. Mostly it is related to XML files, so please compare it with the working version.
I had this problem, but the file was in UTF-8, it was just that somehow on character had come in that was not encoded in UTF-8. To solve the problem I did what is stated in this thread, i.e. I validated the file:
How to check whether a file is valid UTF-8?
Basically you run the command:
$ iconv -f UTF-8 your_file -o /dev/null
And if there is something that is not encoded in UTF-8 it will give you the line and row numbers so that you can find it.
I happened to run into this problem because of an Ant build.
That Ant build took files and applied filterchain expandproperties to it. During this file filtering, my Windows machine's implicit default non-UTF-8 character encoding was used to generate the filtered files - therefore characters outside of its character set could not be mapped correctly.
One solution was to provide Ant with an explicit environment variable for UTF-8.
In Cygwin, before launching Ant: export ANT_OPTS="-Dfile.encoding=UTF-8".
This error comes when you are trying to load jasper report file with the extension .jasper
For Example
c://reports//EmployeeReport.jasper"
While you should load jasper report file with the extension .jrxml
For Example
c://reports//EmployeeReport.jrxml"
[See Problem Screenshot ][1] [1]: https://i.stack.imgur.com/D5SzR.png
[See Solution Screenshot][2] [2]: https://i.stack.imgur.com/VeQb9.png
I had a similar problem.
I had saved some xml in a file and when reading it into a DOM document, it failed due to special character. Then I used the following code to fix it:
String enco = new String(Files.readAllBytes(Paths.get(listPayloadPath+"/Payload.xml")), StandardCharsets.UTF_8);
Document doc = builder.parse(new ByteArrayInputStream(enco.getBytes(StandardCharsets.UTF_8)));
Let me know if it works for you.
I have met the same problem and after long investigation of my XML file I found the problem: there was few unescaped characters like « ».
Those like me who understand character encoding principles, also read Joel's article which is funny as it contains wrong characters anyway and still can't figure out what the heck (spoiler alert, I'm Mac user) then your solution can be as simple as removing your local repo and clone it again.
My code base did not change since the last time it was running OK so it made no sense to have UTF errors given the fact that our build system never complained about it....till I remembered that I accidentally unplugged my computer few days ago with IntelliJ Idea and the whole thing running (Java/Tomcat/Hibernate)
My Mac did a brilliant job as pretending nothing happened and I carried on business as usual but the underlying file system was left corrupted somehow. Wasted the whole day trying to figure this one out. I hope it helps somebody.
I had the same issue. My problem was it was missing “-Dfile.encoding=UTF8” argument under the JAVA_OPTION in statWeblogic.cmd file in WebLogic server.
You have a library that needs to be erased
Like the following library
implementation 'org.apache.maven.plugins:maven-surefire-plugin:2.4.3'
This error surprised me in production...
The error is because the char encoding is wrong, so the best solution is implement a way to auto detect the input charset.
This is one way to do it:
...
import org.xml.sax.InputSource;
...
InputSource inputSource = new InputSource(inputStream);
someReader(
inputSource.getByteStream(), inputSource.getEncoding()
);
Input sample:
<?xml version="1.0" encoding="utf-16"?>
<rss xmlns:dc="https://purl.org/dc/elements/1.1/" version="2.0">
<channel>
...
We were using JasperReports 4.6.0 and Java 1.6 to generate the PDF reports. It was working fine for parameters with text on English. If I pass the Arabic input parameter the input parameter
passed as like a junk character hence I couldn't fetch the recordset. The same was working when I use JasperReports 3.7.6 and Java 1.5
My code:
JasperPrint print = null;
Runtime run = null;
String strJasperFile = "E:/DailyWork/FEB-2013/report2.jasper";
String strOutputFile = "E:/DailyWork/FEB-2013/report2.xls";
String printtime="";
Connection con = getSqlConnection();
HashMap mpDetailSp = new HashMap();
mpDetailSp.put("parameter1", "B المهمات");
print = JasperFillManager.fillReport(strJasperFile, mpDetailSp, con);
JRExporter exporter = new JRXlsExporter();
exporter.setParameter(JRExporterParameter.JASPER_PRINT, print);
exporter.setParameter(JRExporterParameter.OUTPUT_FILE_NAME, strOutputFile);
exporter.exportReport();
Could you please advise me how to sort it out this problem?
Not being the an expert in Jasper I can suppose that the problem is in the text encoding. I performed a short search and found this resource: http://www.adp-gmbh.ch/misc/tools/jasper/java.html
Please take a look in the template example and pay attention on line <?xml version="1.0" encoding="UTF-8"?> on top of the file. Does your file report2.jasper contain such line? Check it and include it if it is missing.
Additionally add line
exporter.setParameter(JRExporterParameter.CHARACTER_ENCODING, "UTF-8");
I hope now all will work.
Finally, I found out the solution for this problem. It was encoding setting in the Java 1.6 JVM. I had set the environment variable as like below to sort it out the problem
JAVA_TOOL_OPTIONS to -Dfile.encoding=UTF8
It was resolved my problem. The input was given by the another forum post in statck overflow
https://stackoverflow.com/a/623036/770927
I thank to Edward Grech, He explained obvious reason and solution in the above post.