I have previously posted a question related to this work link but I am posting a new one because the question is not completely resolved
I am working on converting the completed code into java with php.
It is a function that reads encrypted files, decrypts them by 16 bytes, makes them into a single string, and encodes them with base 64.
php is already on the server and running, and I have to use java to produce the same result.
If you decrypt the read file,
<?xml version="1.0" encoding="utf-8" ?>
<FileInfo>
...
<TextData> (text data) </TextData>
</FileInfo>
(image data)
It is in the format, and the text data in shows php and java exactly the same.
I am trying to encode the image data part into base64, but the result is different from php.
This is the part of the php code I have that decrypts and processes the read file
$fileContentArray = array(16);
$transContentArray = array(16);
$fileRead = fread($fp,16);
for($i = 0 ; $i < strlen($fileRead); $i++){
$fileContentArray[$i] = ord($fileRead[$i]);
}
$seed->SeedDecrypt($fileContentArray,$pdwRoundKey,$transContentArray);
$transStr = call_user_func_array("pack",array_merge(array("C16"),$transContentArray));
$mergeStr .= $transStr;
}
$dataExplode = explode("<TextData>",trim($mergeStr) );
$dataExplode1 = explode("</FileInfo>",trim($dataExplode[1]) );
$dataExplode2 = explode("</TextData>",$dataExplode1[0]);
$textData = iconv("EUC-KR","utf-8",$dataExplode2[0]);
$imageData = base64_encode(trim($dataExplode1[1]));
And this is the same part of the java code I wrote
byte[] fileContentArray=new byte[n];
for(int i=0;i<fileContentArray.length;i++){
fileContentArray[i]=mergeArr[nReadCur+i];
}
seed.SeedDecrypt(fileContentArray, pdwRoundKey, outbuf);
System.arraycopy(outbuf, 0, resultArr, nReadCur, outbuf.length);
nReadCur=nReadCur+fileContentArray.length;
p=p+fileContentArray.length;
if(p>=nFileSize){
fis.close();
break;
}
}//while
mergeStr=new String(resultArr,"MS949");
String[] dataExplode=mergeStr.trim().split("<TextData>");
String[] dataExplode1=dataExplode[1].trim().split("</FileInfo>");
String[] dataExplode2=dataExplode1[0].trim().split("</TextData>");
String textData = "";
String imageData = "";
textData=dataExplode2[0];
imageData=dataExplode1[1];
Encoder encoder=Base64.getEncoder();
Decoder decoder=Base64.getDecoder();
byte[] encArr=encoder.encode(imageData.trim().getBytes("MS949"));
imageData=new String(encArr,"MS949");
As a result of encoding image data into base64
php: R0lGODlhAwNLBPcAAAAAAAAAMwAAZgAAmQAAzAAA/wArAAArMwArZgArmQArzAAr/wBVAABVMwBVZgBVmQBVzABV/wCAAACAMwCAZgCAmQCAzACA/ ... VzpYirO0le55zF0=
java: R0lGODlhAwNLBD8AAAAAAAAAMwAAZgAAPwAAPwAAPwArAAArMwArZgArPwArPwArPwBVAABVMwBVZgBVPwBVPwBVPwA/AAA/MwA/ZgA/PwA/PwA/PwA/ ... DAQEAOz9GPz8/IXY=
As you can see, the result values are output differently.
Is there anything I'm missing? What should I do to make the result of java the same as php?
Also, MergeStr, who has read the file,
java:
GIF89aK? 3 f ? ? ? + +3 +f +? +? +? U U3 Uf U? U? U? ? ?3 ?f ?? ?? ?? ? ?3 챖 첌 ぬ ? ? ?3 ?f ??
...
J뇽杞H?*]苛⒢쬝쥻쒳뎁諾X...
A?h?~?0a?2$ #삁?d?Dd??e ...
...
WC ;홃?뿿!v
php:
GIF89aK? 3 f ? ? + +3 +f +? +? + U U3 Uf U? U? U 3 f ? ? ? ? 챖 첌 ぬ ? ? ? ? ? 螂 ? 3 f ? ? ...
A??~?a?$ #삁?d?Dd?e...
...
WC ;홃??v余퍙W:X뒽킉??
Like this, there is a new line text that I didn't put in, and there's a slight difference in result. Is this a simple difference in encoding? And does this affect the base64 conversion?
I tried encoding with UTF-8 and failed again,
and I used this code to load all bytes of the file at once
FileInputStream fis = new FileInputStream(tpf);
fis.read(mergeArr);
Java uses a little bit different encoding for base64 than PHP. You may use these helper functions to make them compatible to the Java version.
function base64url_encode($data)
{
return rtrim(strtr(base64_encode($data), '+/', '-_'), '=');
}
function base64url_decode($data, $strict = false)
{
return base64_decode(strtr($data, '-_', '+/'), $strict);
}
Given a grammar (simplified version below) where I can enter arbitrary text in a section of the grammar, is it possible to format the content of the arbitrary text? I understand how to format the position of the arbitrary text in relation to the rest of the grammar, but not whether it is possible to format the content string itself?
Sample grammar
Model:
'content' content=RT
terminal RT: // (returns ecore::EString:)
'RT>>' -> '<<RT';
Sample content
content RT>>
# Some sample arbitrary text
which I would like to format
<<RT
you can add custom ITextReplacer to the region of the string.
assuming you have a grammar like
Model:
greetings+=Greeting*;
Greeting:
'Hello' name=STRING '!';
you can do something like the follow in the formatter
def dispatch void format(Greeting model, extension IFormattableDocument document) {
model.prepend[newLine]
val region = model.regionFor.feature(MyDslPackage.Literals.GREETING__NAME)
val r = new AbstractTextReplacer(document, region) {
override createReplacements(ITextReplacerContext it) {
val text = region.text
var int index = text.indexOf(SPACE);
val offset = region.offset
while (index >=0){
it.addReplacement(region.textRegionAccess.rewriter.createReplacement(offset+index, SPACE.length, "\n"))
index = text.indexOf(SPACE, index+SPACE.length()) ;
}
it
}
}
addReplacer(r)
}
this will turn this model
Hello "A B C"!
into
Hello "A
B
C"!
of course you need to come up with a more sophisticated formatter logic.
see How to define different indentation levels in the same document with Xtext formatter too
I am doing a project using libsvm and I am preparing my data to use the lib. How can I convert CSV file to LIBSVM compatible data?
CSV File:
https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/datasets/data/iris.csv
In the frequencies questions:
How to convert other data formats to LIBSVM format?
It depends on your data format. A simple way is to use libsvmwrite in the libsvm matlab/octave interface. Take a CSV (comma-separated values) file in UCI machine learning repository as an example. We download SPECTF.train. Labels are in the first column. The following steps produce a file in the libsvm format.
matlab> SPECTF = csvread('SPECTF.train'); % read a csv file
matlab> labels = SPECTF(:, 1); % labels from the 1st column
matlab> features = SPECTF(:, 2:end);
matlab> features_sparse = sparse(features); % features must be in a sparse matrix
matlab> libsvmwrite('SPECTFlibsvm.train', labels, features_sparse);
The tranformed data are stored in SPECTFlibsvm.train.
Alternatively, you can use convert.c to convert CSV format to libsvm format.
but I don't wanna use matlab, I use python.
I found this solution as well using JAVA
Can anyone recommend a way to tackle this problem ?
You can use csv2libsvm.py to convert csv to libsvm data
python csv2libsvm.py iris.csv libsvm.data 4 True
where 4 means target index, and True means csv has a header.
Finally, you can get libsvm.data as
0 1:5.1 2:3.5 3:1.4 4:0.2
0 1:4.9 2:3.0 3:1.4 4:0.2
0 1:4.7 2:3.2 3:1.3 4:0.2
0 1:4.6 2:3.1 3:1.5 4:0.2
...
from iris.csv
150,4,setosa,versicolor,virginica
5.1,3.5,1.4,0.2,0
4.9,3.0,1.4,0.2,0
4.7,3.2,1.3,0.2,0
4.6,3.1,1.5,0.2,0
...
csv2libsvm.py does not work with Python3, and also it does not support label targets (string targets), I have slightly modified it. Now It should work with Python3 as well as wıth the label targets.
I am very new to Python, so my code may do not follow the best practices, but I hope it is good enough to help someone.
#!/usr/bin/env python
"""
Convert CSV file to libsvm format. Works only with numeric variables.
Put -1 as label index (argv[3]) if there are no labels in your file.
Expecting no headers. If present, headers can be skipped with argv[4] == 1.
"""
import sys
import csv
import operator
from collections import defaultdict
def construct_line(label, line, labels_dict):
new_line = []
if label.isnumeric():
if float(label) == 0.0:
label = "0"
else:
if label in labels_dict:
new_line.append(labels_dict.get(label))
else:
label_id = str(len(labels_dict))
labels_dict[label] = label_id
new_line.append(label_id)
for i, item in enumerate(line):
if item == '' or float(item) == 0.0:
continue
elif item=='NaN':
item="0.0"
new_item = "%s:%s" % (i + 1, item)
new_line.append(new_item)
new_line = " ".join(new_line)
new_line += "\n"
return new_line
# ---
input_file = sys.argv[1]
try:
output_file = sys.argv[2]
except IndexError:
output_file = input_file+".out"
try:
label_index = int( sys.argv[3] )
except IndexError:
label_index = 0
try:
skip_headers = sys.argv[4]
except IndexError:
skip_headers = 0
i = open(input_file, 'rt')
o = open(output_file, 'wb')
reader = csv.reader(i)
if skip_headers:
headers = reader.__next__()
labels_dict = {}
for line in reader:
if label_index == -1:
label = '1'
else:
label = line.pop(label_index)
new_line = construct_line(label, line, labels_dict)
o.write(new_line.encode('utf-8'))
I'm new in Ext and I have a problem: I'm trying to fill extjs-grid with data:
Ext.onReady(function() {
var store = new Ext.data.JsonStore({
root: 'topics',
totalProperty: 'totalCount',
idProperty: 'threadid',
remoteSort: true,
autoLoad: true, ///
fields: [
'title', 'forumtitle', 'forumid', 'author',
{name: 'replycount', type: 'int'},
{name: 'lastpost', mapping: 'lastpost', type: 'date', dateFormat: 'timestamp'},
'lastposter', 'excerpt'
],
proxy: new Ext.data.ScriptTagProxy({
url:'http://10.10.10.101:8080/myproject/statusList/getJobs/2-10/search-jobname-/sort-asdf/filterjobname-123/filterusername-davs/filterstatus-completed/filtersubmdate-today',
method : 'GET'
})
});
//
var cm = new Ext.grid.ColumnModel([
{sortable:true, id : 'id', dataIndex:'id'},
{sortable:true, id : 'title', dataIndex:'title'},
{sortable:true, id : 'forumtitle', dataIndex:'forumtitle'},
{sortable:true, id : 'forumid', dataIndex:'forumid'},
{sortable:true, id : 'author', dataIndex:'author'}
]);
var grid = new Ext.grid.GridPanel({
id: 'mainGrid',
el:'mainPageGrid',
pageSize:10,
store:store,
// stripeRows: true,
cm:cm,
stateful: false, // skipSavingSortState
viewConfig:{
forceFit:true
},
// width:1000,
// height:700,
loadMask:true,
frame:false,
bbar: new Ext.PagingToolbar({
id : 'mainGridPaginator',
store:store,
hideRefresh : true,
plugins: new Ext.ux.Andrie.pPageSize({
beforeText: 'View: ',
afterText: '',
addAfter: '-',
variations: [10, 25, 50, 100, 1000]
//comboCfg: {
//id: '${ dispview_widgetId }_bbar_pageSize'
//}
}),
displayMsg: 'Displaying items {0} - {1} of {2}',
emptyMsg:'No data found',
displayInfo:true
})
});
grid.render();
});
and the Java part:
#GET
#Path("/getJobs/{startFrom}-{startTo}/search-{searchType}-{searchName:.*}/" +
"sort-{sortType}/filterjobname-{filterJobName:.*}/filterusername-{filterUsername:.*}/" +
"filterstatus-{filterStatus:.*}/filtersubmdate-{filterSubmittedDate:.*}")
#Produces({"application/json"})
#Encoded
public String getJobs(
#PathParam("startFrom") String startFrom,
#PathParam("startTo") String startTo,
#PathParam("searchType") String searchType,
#PathParam("searchName") String searchName,
#PathParam("sortType") String sortType,
#PathParam("filterJobName") String filterJobName,
#PathParam("filterUsername") String filterUsername,
#PathParam("filterStatus") String filterStatus,
#PathParam("filterSubmittedDate") String filterSubmittedDate) {
return "{totalCount:'3',topics:[{title:'XTemplate with in EditorGridPanel',threadid:'133690',username:'kpremco',userid:'272497',dateline:'1305604761',postid:'602876',forumtitle:'Ext 3x Help',forumid:'40',replycount:'2',lastpost:'1305857807',lastposter:'kpremco',excerpt:'Hi I have an EditiorGridPanel whose one column i am using XTemplate to render and another Column is Combo Box FieldWhen i render the EditorGri'}," +
"{title:'IFrame error _flyweights is undefined',threadid:'133571',username:'Daz',userid:'52119',dateline:'1305533577',postid:'602456',forumtitle:'Ext 3x Help',forumid:'40',replycount:'1',lastpost:'1305857313',lastposter:'Daz',excerpt:'For Ext 330 using Firefox 4 Firebug, the following error is often happening when our app loads e._flyweights is undefined Yetthis '}," +
"{title:'hellllllllllllllpwhy it doesnt fire cellclick event after I change the cell value',threadid:'133827',username:'aimer311',userid:'162000',dateline:'1305700219',postid:'603309',forumtitle:'Ext 3x Help',forumid:'40',replycount:'3',lastpost:'1305856996',lastposter:'aimer311',excerpt:'okI will discribe this problem as more detail as I canI look into this problem for a whole dayI set clicksToEdit1 to a EditorGridPanelso when I'}]}";
As a result I'm getting a JavaScript error:
Syntax error at line 1 while loading:
totalCount:'3',topics:[{title:'XTemplate
---------------------^
expected ';', got ':'
Although, when I'm using Proxy's URL:
URL: 'http://extjs.com/forum/topics-browse-remote.php',
which represents same information, I don't have any problems.
Where is my failure????
P.S. Comments for the first answer:
return "{\"totalCount\":\"3\",\"topics\":[{\"title\":\"XTemplate with in EditorGridPanel\",\"threadid\":\"133690\",\"username\":\"kpremco\",\"userid\":\"272497\",\"dateline\":\"1305604761\",\"postid\":\"602876\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"2\",\"lastpost\":\"1305857807\",\"lastposter\":\"kpremco\",\"excerpt\":\"Hi I have an EditiorGridPanel whose one column i am using XTemplate to render and another Column is Combo Box FieldWhen i render the EditorGri\"}," +
"{\"title\":\"IFrame error _flyweights is undefined\",\"threadid\":\"133571\",\"username\":\"Daz\",\"userid\":\"52119\",\"dateline\":\"1305533577\",\"postid\":\"602456\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"1\",\"lastpost\":\"1305857313\",\"lastposter\":\"Daz\",\"excerpt\":\"For Ext 330 using Firefox 4 Firebug, the following error is often happening when our app loads e._flyweights is undefined Yet, this \"}," +
"{\"title\":\"hellllllllllllllpwhy it doesn't fire cellclick event after I change the cell value\",\"threadid\":\"133827\",\"username\":\"aimer311\",\"userid\":\"162000\",\"dateline\":\"1305700219\",\"postid\":\"603309\",\"forumtitle\":\"Ext 3x Help\",\"forumid\":\"40\",\"replycount\":\"3\",\"lastpost\":\"1305856996\",\"lastposter\":\"aimer311\",\"excerpt\":\"okI will discribe this problem as more detail as I canI look into this problem for a whole dayI set clicksToEdit1 to a EditorGridPanelso when I\"}]}";
I've got the following error:
Syntax error at line 1 while loading:
{"totalCount":"3","topics":[{"title
-------------^
expected ';', got ':'
P.S. #2. When I've added '[' to the begining of the response string and ']' to the end , erros disappered, but grid hasn't been filled with data
You're not returning (valid) JSON. Refer to the JSON site for details, but for instance, all property keys must be in double quotes. (All strings must also be in double quotes; single quotes are not valid for JSON strings.)
So for instance, this is not valid JSON:
{totalCount:'3'}
...because the key is not in quotes, and the value is using single quotes. The correct JSON would be:
{"totalCount":"3"}
...if you really want the 3 to be a string, or:
{"totalCount":3}
...if the 3 should be a number.
People frequently confuse JSON and JavaScript's object literal notation, but they are different. Specifically, JSON is a subset of object literal notation. A lot of things that are valid in object literal notation are not valid in JSON. Any time you're in doubt, you can check at jsonlint.com, which provides a proper JSON validator.
I have found the root of issue.
As I've known Ext send to my web service function with parameter 'callback=[some_callback_name]' (e.g. callback1001). It means that Extjs wanna to get results not just in JSON format, but in format 'callback1001()'. When I've return my data in this format everything became good.
Proof links:
http://www.sencha.com/forum/showthread.php?22990-Json-Invalid-label (response #6)
http://indiandeve.wordpress.com/2009/12/02/extjs-error-invalid-label-error-while-using-scripttagproxy-for-json-data-in-paging-grid-example/
JSPX has the lovely side effect of turning:
<div class="magic"></div>
Into:
<div class="magic" />
For many browsers this causes pandemonium and mayhem with layout even though it is valid XHTML.
Consequently I have resorted to using a groovy script to find all possible bad HTML with the following regex:
def m = html =~ /<(\w+)[^>]*?><\/(\w+)>/
def bad = m.findAll { it[1] == it[2] };
Is there way to have the JSPX XML processor not to self close tags?
I've been using <div><jsp:text/></div>
AFAIK, there is no elegant solution to this (read: configurable on container level). Check "jspx script element on GlassFish v3" for possible workarounds.
You can try specifying content inside the element that won't affect how the HTML renders, but will prevent the XHTML from being serialized as a self-closing element; like a comment, processing instruction, or non-breaking white space character().
As noted by Neeme there seems to be no solution to this problem.
However I have written a Groovy script that you can have Maven call (GMaven) to check for possible XHTML tags that will get self closed.
This script probably needs better error messages and does not handle all cases but has so far mitigated the problem.
#!/usr/bin/env groovy
def srcdir = project.properties['srcdir'];
def badFiles = [];
def checkFile(badFiles, file) {
def htmlLines = file.readLines();
def found = [];
int i = 0;
for (html in htmlLines) {
++i;
//print html;
def m = html =~ /<(\w+)[^>]*?><\/(\w+)>/
def bad = m.findAll { it[1] == it[2] };
if (bad)
found.add(['bad' : bad, 'line' : i]);
}
if (found) {
badFiles << file;
println "File had bad HTML: " + file.canonicalPath;
println found;
}
}
def ant = new AntBuilder();
scanner = ant.fileScanner {
fileset(dir:srcdir) {
include(name:"**/*.jspx")
}
}
for (f in scanner) {
//println "Checking file: " + f.canonicalPath;
checkFile(badFiles, f);
}
if (badFiles) {
println "Bad files: " + badFiles;
fail('Bad files: ' + badFiles);
}