Develop Reference

How do I handle non-English characters properly?

So I'm working with last.fm API. Sometimes, the query results in tracks that contain characters like these:
Æther, é, Hṛṣṭa
or non-English characters like these:
水鏡.
When debugging in Eclipse, I see them just fine (as-is) but printing on console prints these as ??? - which is OK for me.
Now, how do I handle these? At first I though I could remove every song that has any character other than the ones in English language. I used the regex ^\\w+$ but it didn't work. I also tried \\w+. That didn't work either.
Then I thought further on how do handle these properly. Any one can help me out? I am perfectly fine with letting these tracks out of the equation, ie. I'm fine with having only English character tracks.
Another question: What is the best way to display these character of console and/or Swing GUI?

You must ensure that you use correct encoding when reading your input first.
Second ensure that the font used in Eclipse on platform you developing has ability to display all these characters. Swing must display unicode chars if you read them correctly.
You will likely want to use UTF-8 everywhere.

Is there a standard way to detect directional character?

I'm parsing a text file made from this Wikipedia article, basically I made a Ctrl+A and copy/paste all the content in a text file. (I use it as example).
I'm trying to make a list of words with their counts and for that I use a Scanner with this delimiter :
sc.useDelimiter("[\\p{javaWhitespace}\\p{Punct}]+");
It works great for my need, but analysing the result, I saw something that looks like a blank token (again...). The character is after (nynorsk)‬ in the article (funny when I copy/paste here the character disappear, in gedit I can use → and ← and the cursor don't move).
After further research I've found out that this token was actually the POP DIRECTIONAL FORMATTING (U+202C).
It's not the only directional character, looking at the Character documentation Java seems to define them.
So I'm wondering if there is a standard way to detect these characters, and if possible a way that can be easily integrated in the delimiter pattern.
I'd like to avoid to make my own list because I fear I will forgot some of them.

You could always go the other way round and use a whitelist rather than a blacklist:
sc.useDelimiter("[^\\p{L}]+");

How to reduce the number of runs returned from Uniscribe ScriptItemize

I am using a Java SWT StyledText control to display some text on Windows. However, the performance sucks because the text I am displaying has lots of commas. Here is an example of a line of text I am trying to display:
1,2,3,4,5,1,
The Java SWT code calls into the standard Windows Uniscribe ScriptItemize function to split this text into runs and gets back 12 runs! Each character is its own run! This kills the performance of the SWT drawing code to have to draw the characters one by one.
I have raised Eclipse bug 352927 to cover this issue. Does anyone have experience with Uniscribe who could comment on how to avoid these one character runs?
The settings for the ScriptItemize method come from a SCRIPT_CONTROL struct. However, the
defn of this struct used by SWT is missing the fMergeNeutralItems field. This
sounds promising to me but I haven't managed to find anything clear online
about whether this field would merge together the returned runs for my comma
heavy text.

Setting fMergeNeutralItems in the SCRIPT_CONTROL struct fixes this issue. See the SWT bugzilla bug for more details.

Word wrap in Eclipse Java? [duplicate]

This question already has answers here:
Does Eclipse have line-wrap
(12 answers)
Closed 7 years ago.
I already have word wrap for Eclipse PHP (How can I get word wrap to work in Eclipse PDT for PHP files?). What about Eclipse Java? Is there a similar tool to wrap Java code lines?

Ctrl+Shift+F will format a file in Eclipse, breaking long lines into multiple lines and nicely word-wrapping comments. You can also highlight just a section of text and format that.
I realize this is not an automatic soft/hard word wrap, but I don't think the question was asking for anything fancy.

Just like to point out that word wrapping is a terrible idea in code. I suppose everyone has their own style, but typically you should refactor or rethink blocks of code that take much more than a single average horizontal screen. (about 1280 pixels)
Of course there are always exceptions... like verbose exceptions for example, but these are not critical for code understanding.
Scanning a line that goes beyond a single horizontal screen, or having to read a thought that stacks on multiple lines will make your code much more difficult to figure out.
Additionally, line counts will be weird, as word wrapping will make a single line seem like it takes up 2, 3, or shudder the thought... 4 lines of code.
As far as comments go, line returns should not be much of a problem for anyone.
Also keep in mind, word wrapping code is not a common practice, and you are essentially saying that everyone reading your code should turn word wrapping on, which is, I'm sorry to say, an unnecessarily painful thing to make others working with your code do....
If nesting is causing you the need for word wrap... well this is also a very bad code smell.
That's my two-cents.

This is for anyone who is having the following problem with the "Ahtik" word-wrap plugin in Texlipse: the plugin installs fine, but selecting the "word-wrap" option using the context menu (or pressing ctrl+alt+w) causes the text to wrap momentarily, and then just flick back to being unwrapped (this probably includes anyone using Windows 7).
Here is a fix...straight from the developer himself! I emailed Ahti Kitsik in a last-ditch attempt to find a way round, and was amazed when he actually found a solution, and sent it to me the next day. He suggested that in return I could just make this information widely available.
So, the way to fix it as follows:
Look up the texlipse plugin location. This will probably be in the Eclipse folder, and the address will probably end: ...\eclipse\plugins\net.sourceforge.texlipse_1.5.0.
Backup the plugin.xml file there just in case, and then edit the original by deleting the following chunks of code:
-Firstly, the "action" (starting at line 843):
action
class="net.sourceforge.texlipse.actions.TexWordWrapAction"
definitionId="net.sourceforge.texlipse.commands.texWordWrap"
icon="icons/wrap.gif"
id="net.sourceforge.texlipse.actions.texWordWrap"
label="W&rap text"
menubarPath="net.sourceforge.texlipse.menus.latex/latexGroup"
style="toggle"
toolbarPath="latexGroup"
tooltip="Use word wrap"
-Secondly, the "command" (starting at line 972):
command
categoryId="net.sourceforge.texlipse.latexEditingCategory"
name="Wrap text"
id="net.sourceforge.texlipse.commands.texWordWrap"
(When you have done this, the file should be 1257 lines long, instead of 1271).
Finally, Ahti says: "Start eclipse with -clean command line argument to force reloading of plugin.xml." (Personally I had no idea how to do this, so I just removed and reinstalled his plugin, then exited Eclipse, and edited the code before I restarted it).
The function should work fine now. Hope this helps someone. And thanks again Ahti!

In the meantime, there has been an fork to Ahtik's word wrap plugin which, I must say, works very nice and fixes jumbled line numbering bug: Eclipse Word-Wrap Plug-In

Open Eclipse -> Help -> Install New Software -> Click "Add.." button and copy this line "http://dev.cdhq.de/eclipse/updatesite/luna/" into field Name and Location -> Click OK
Check -> Line Number Ruler and Word Wrap Feature -> Next -> Next -> Finish -> Restard Eclipse!

Selenium typeKeys strips out dot from the String being typed

The following instruction
Selenium.typeKeys("location", "gmail.com");
types the string gmailcom instead of gmail.com.
What's happening there?
From the comments:
I am trying to simulate autofill and the only way to do it currently on selenium is to combine type and typeKeys. eg:
selenium.type("assigned_to", split[0]+"#");
selenium.typeKeys("assigned_to", "gmail.com");
Now my question is why typeKeys doesn't type the 'dot' in between gmail.com?

Have you tried using the Native key functions and javascript char codes?
I couldn't get a 'period' character to work using them (char 190), but I got the decimal (char 110) to work just fine, and the text box shouldn't have a problem with either.
selenium.Focus("assigned_to");
selenium.Type("assigned_to", split[0]+"#");
selenium.TypeKeys("assigned_to", "gmail");
selenium.KeyPressNative("110");
selenium.TypeKeys("assigned_to", "com");

Use the type method.
From the javadoc for typekeys:
this command may or may not have any
visible effect, even in cases where
typing keys would normally have a
visible effect
...
In some cases, you may
need to use the simple "type" command
to set the value of the field and then
the "typeKeys" command to send the
keystroke events corresponding to what
you just typed.

We had similar problems using typekeys in selenium python.
One workaround we figured to resolve this issue is to use the combination of 'type' and 'type_keys'. As you might be aware, type does not have such issues.
We did this in our selenium python script and it works just fine.
For example:
If there's an email address to be entered in a text box: test.me#test.me.uk
Then do
type(locator,’test.me#test.me.’)
type_keys(locator,’uk’)
Maybe a very crude way to do, but it did the job.
Hope this helps someone else with a similar problem.

Also try to set focus on element before write on it.
selenium.focus(locator);
selenium.typeKeys(locator, value);
it did function in my case, handling a input type=password.

Suppose the string to be typed using typeKeys is "abc.xyz.efg". Then, we can use type and typeKeys commands to write the given string.
type(locator,"abc.xyz.")
typeKeys(locator,"efg")
The above two steps are useful whenever you want to select an element in drop down box, and the drop down pops down only if we use typeKeys command.

I'm also seeing this behaviour when using Selenium RC (C#), and with different characters ('y' for example which also seems to remove the character follow it from the typing action..)
For some situations it is entirely possible to get around the issue by typing out the keys with the TypeKeys or KeyPress functions, but I haven't managed to find a solution that works when you want to type text in a field, enter control characters ([Enter] or a newline for example), and then type more text.. (using the 'Type' function in this case simply overwrites the field contents...).
If anyone manages to find a reasonable solution to this issue, please add it to this question as it's starting to come up in google now and would probably help alot of people out.. (I'd start a bounty if I could..)

This is an open bug in Selenium (bug SEL-519).
After some fiddling around with it, I finally managed to enter '.' into a textarea by using JavaScript. Execute something like window.document.getElementById('assigned_to').value += '.' via storeEval or the like.

I got the same behaviour but fixed it by passing a variable to the Type command instead of a string.
string email = #"name#gmail.com";
selenium.Type(inputfield, email);
It works like a charm!