Develop Reference

"ValueError: Incompatible Language version 13. Must not be between 9 and 12" with Google Colab

I am trying to build a deep learning model with transformer model architecture. In that case when I am trying to cleaning the dataset following error occurred.
I am using Pytorch and google colab for that case & trying to clean Java methods and comment dataset.
Tested Code
import re
from fast_trees.core import FastParser
parser = FastParser('java')
def get_cmt_params(cmt: str) -> List[str]:
'''
Grabs the parameter identifier names from a JavaDoc comment
:param cmt: the comment to extract the parameter identifier names from
:returns: an array of the parameter identifier names found in the given comment
'''
params = re.findall('#param+\s+\w+', cmt)
param_names = []
for param in params:
param_names.append(param.split()[1])
return param_name
Occured Error
Downloading repo https://github.com/tree-sitter/tree-sitter-java to /usr/local/lib/python3.7/dist-packages/fast_trees/tree-sitter-java.
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-31-64f6fa6ed39b> in <module>()
3 from fast_trees.core import FastParser
4
----> 5 parser.set_language = FastParser('java')
6
7 def get_cmt_params(cmt: str) -> List[str]:
3 frames
/usr/local/lib/python3.7/dist-packages/fast_trees/core.py in FastParser(lang)
96 }
97
---> 98 return PARSERS[lang]()
/usr/local/lib/python3.7/dist-packages/fast_trees/core.py in __init__(self)
46
47 def __init__(self):
---> 48 super().__init__()
49
50 def get_method_parameters(self, mthd: str) -> List[str]:
/usr/local/lib/python3.7/dist-packages/fast_trees/core.py in __init__(self)
15 class BaseParser:
16 def __init__(self):
---> 17 self.build_parser()
18
19 def build_parser(self):
/usr/local/lib/python3.7/dist-packages/fast_trees/core.py in build_parser(self)
35 self.language = Language(build_dir, self.LANG)
36 self.parser = Parser()
---> 37 self.parser.set_language(self.language)
38
39 # Cell
ValueError: Incompatible Language version 13. Must not be between 9 and 12
an anybody help me to solve this issue?

fast_trees uses tree_sitter and according to tree_sitter repo it is an incomatibility issue. If you know the owner of fast_trees ask them to upgrade their tree_sitter version.
Or you can fork it and upgrade it yourself, but keep in mind it may not be backwards compatible if you take it upon yourself and it may not be just a simple new version install.

The fast-trees library uses the tree-sitter library and since they recommended using the 0.2.0 version of tree-sitter in order to use fast-trees. Although downgrade the tree-sitter to the 0.2.0 version will not be resolved your problem. I also tried out it by downgrading it.
So, without investing time to figure out the bug in tree-sitter it is better to move to another stable library that satisfies your requirements. So, as your requirement, you need to extract features from a given java code. So, you can use javalang library to extract features from a given java code.
javalang is a pure Python library for working with Java source code.
javalang provides a lexer and parser targeting Java 8. The
implementation is based on the Java language spec available at
http://docs.oracle.com/javase/specs/jls/se8/html/.
you can refer it from - https://pypi.org/project/javalang/0.13.0/
Since javalang is a pure library it will help go forward on your research without any bugs

H2O AI : Unsupported MOJO model 'word2vec'

I have 3 h2o models:
$ ls dataset/mojo
1. DeepLearning_model_python_1582176092021_2.zip
2. StackedEnsemble_BestOfFamily_AutoML_20200220_073620.zip
3. Word2Vec_model_python_1582176092021_1.zip
The binary models for these 3 were generated on v3.28.0.3, but I am trying to upgrade the h2o version and productionize it onto v3.30.0.5
So i converted those 3 binaries successfully to MOJO models (as listed above)
When trying to upload these mojo models using the h2o.upload_mojo, for Word2Vec alone, am getting the error:
In [15]: w2v_path = 'dataset/mojo/Word2Vec_model_python_1582176092021_1.zip'
In [16]: w2v_model = h2o.upload_mojo(w2v_path)
generic Model Build progress: | (failed) | 0%
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-16-734005ed70a8> in <module>
----> 1 w2v_model = h2o.upload_mojo(w2v_path)
~/.envs/h2o-test/lib/python3.8/site-packages/h2o/h2o.py in upload_mojo(mojo_path)
2149 frame_key = response["destination_frame"]
2150 mojo_estimator = H2OGenericEstimator(model_key = get_frame(frame_key))
-> 2151 mojo_estimator.train()
2152 print(mojo_estimator)
2153 return mojo_estimator
~/.envs/h2o-test/lib/python3.8/site-packages/h2o/estimators/estimator_base.py in train(self, x, y, training_frame, offset_column, fold_column, weights_column, validation_frame, max_runtime_secs, ignored_columns, model_id, verbose)
113 validation_frame=validation_frame, max_runtime_secs=max_runtime_secs,
114 ignored_columns=ignored_columns, model_id=model_id, verbose=verbose)
--> 115 self._train(parms, verbose=verbose)
116
117 def train_segments(self, x=None, y=None, training_frame=None, offset_column=None, fold_column=None,
~/.envs/h2o-test/lib/python3.8/site-packages/h2o/estimators/estimator_base.py in _train(self, parms, verbose)
205 return
206
--> 207 job.poll(poll_updates=self._print_model_scoring_history if verbose else None)
208 model_json = h2o.api("GET /%d/Models/%s" % (rest_ver, job.dest_key))["models"][0]
209 self._resolve_model(job.dest_key, model_json)
~/.envs/h2o-test/lib/python3.8/site-packages/h2o/job.py in poll(self, poll_updates)
75 if self.status == "FAILED":
76 if (isinstance(self.job, dict)) and ("stacktrace" in list(self.job)):
---> 77 raise EnvironmentError("Job with key {} failed with an exception: {}\nstacktrace: "
78 "\n{}".format(self.job_key, self.exception, self.job["stacktrace"]))
79 else:
OSError: Job with key $03010a64051932d4ffffffff$_8d0c64127137bd1eef16202889cf4fca failed with an exception: java.lang.IllegalArgumentException: Unsupported MOJO model 'word2vec'.
stacktrace:
java.lang.IllegalArgumentException: Unsupported MOJO model 'word2vec'.
at hex.generic.Generic$MojoDelegatingModelDriver.computeImpl(Generic.java:99)
at hex.ModelBuilder$Driver.compute2(ModelBuilder.java:248)
at hex.generic.Generic$MojoDelegatingModelDriver.compute2(Generic.java:78)
at water.H2O$H2OCountedCompleter.compute(H2O.java:1557)
at jsr166y.CountedCompleter.exec(CountedCompleter.java:468)
at jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:263)
at jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:974)
at jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1477)
at jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
The other two models succeed without any issues, and returns a valid model_id. Any idea what the issue here is, coz from docs its understood that all three model types are supported by MOJO
I tried this with a cluster of 2 pods on K8s with 2Gi/1cpu memory each, but results in same outcome as above.

Word2Vec is not currently in the list of allowed algos to import back into H2O.
The documentation is a little bit confusing and needs improvement. MOJO is a way to take H2O models into production. Those are usable outside of H2O using H2O's genmodel. Some of those MOJOs are importable back into H2O and inspected. But not all of them. The first two algorithms listed are supported. Unfortunately, Word2Vec is not.
I've created a JIRA to track this issue. We should be able to enable at least scoring.

"Java not found" in using tabula-py

everyone, I am using tabula-py in python to extract table from pdfs. I used following codes.
import tabula
table_temp = tabula.read_pdf('./example_pdf/sample1.pdf',pages=11)
However, I got the error message as pasted below, in which I was told "no such file or directory: 'java'". I have installed Java in the following folder
"/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home".
Could anyone help me on solving the problem?
Thanks.
FileNotFoundError Traceback (most recent call last)
<ipython-input-4-41c9ba6fd519> in <module>()
----> 1 table_temp = tabula.read_pdf('./example_pdf/sample1.pdf',pages=11)
/Users/Myworld/anaconda/lib/python3.5/site-packages/tabula/wrapper.py in read_pdf(input_path, **kwargs)
64
65 try:
---> 66 output = subprocess.check_output(args)
67 finally:
68 if is_url:
/Users/Myworld/anaconda/lib/python3.5/subprocess.py in check_output(timeout, *popenargs, **kwargs)
314
315 return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
--> 316 **kwargs).stdout
317
318
/Users/Myworld/anaconda/lib/python3.5/subprocess.py in run(input, timeout, check, *popenargs, **kwargs)
381 kwargs['stdin'] = PIPE
382
--> 383 with Popen(*popenargs, **kwargs) as process:
384 try:
385 stdout, stderr = process.communicate(input, timeout=timeout)
/Users/Myworld/anaconda/lib/python3.5/subprocess.py in __init__(self, args, bufsize, executable, stdin, stdout, stderr, preexec_fn, close_fds, shell, cwd, env, universal_newlines, startupinfo, creationflags, restore_signals, start_new_session, pass_fds)
674 c2pread, c2pwrite,
675 errread, errwrite,
--> 676 restore_signals, start_new_session)
677 except:
678 # Cleanup if the child failed starting.
/Users/Myworld/anaconda/lib/python3.5/subprocess.py in _execute_child(self, args, executable, preexec_fn, close_fds, pass_fds, cwd, env, startupinfo, creationflags, shell, p2cread, p2cwrite, c2pread, c2pwrite, errread, errwrite, restore_signals, start_new_session)
1280 else:
1281 err_msg += ': ' + repr(orig_executable)
-> 1282 raise child_exception_type(errno_num, err_msg)
1283 raise child_exception_type(err_msg)
1284
FileNotFoundError: [Errno 2] No such file or directory: 'java'

I had run into the same error. The line actually causing the error for me was subprocess.call('java').
Installing Java on my machine fixed the error for me.
If installing Java still doesn't solve the problem for you, try running which java, and add the output directory to your PATH environment variable.

Failed to install WWW::HtmlUnit in Windows box

I'm trying to install WWW::HTMLUnit on Windows 7. There're step that I run through:
Install Inline::Java 0.53
Install WWW::HTMLUnit 0.15
At step 2, after nmake, I type nmake test to test module but it failed. Here's output:
C:\nmake test
Microsoft (R) Program Maintenance Utility Version 9.00.30729.01
Copyright (C) Microsoft Corporation. All rights reserved.
C:\Perl\bin\perl.exe "-MExtUtils::Command::MM" "-e" "test_harness(0, 'blib\lib', 'blib\arch')" t/*.t
t/00_basic...........
t/00_basic...........NOK 1/1# Failed test 'use WWW::HtmlUnit;'
# at t/00_basic.t line 9.
# Tried to use 'WWW::HtmlUnit'.
# Error: Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
# BEGIN failed--compilation aborted at (eval 4) line 2, <GEN7> line 4.
# Looks like you failed 1 test of 1.
t/00_basic...........dubious
Test returned status 1 (wstat 256, 0x100)
DIED. FAILED test 1
Failed 1/1 tests, 0.00% okay
t/01_hello...........Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
BEGIN failed--compilation aborted at t/01_hello.t line 4, <GEN7> line 4.
t/01_hello...........dubious
Test returned status 26 (wstat 6656, 0x1a00)
t/02_hello_sweet.....dubious
Test returned status 19 (wstat 4864, 0x1300)
t/03_clickhandler....Class com.gargoylesoftware.htmlunit.WebClient not found at C:/Perl/site/lib/Inline/Java.pm line 619
BEGIN failed--compilation aborted at t/03_clickhandler.t line 6, <GEN7> line 4.
t/03_clickhandler....dubious
Test returned status 29 (wstat 7424, 0x1d00)
DIED. FAILED tests 1-8
Failed 8/8 tests, 0.00% okay
Failed Test Stat Wstat Total Fail List of Failed
-------------------------------------------------------------------------------
t/00_basic.t 1 256 1 1 1
t/01_hello.t 26 6656 ?? ?? ??
t/02_hello_sweet.t 19 4864 ?? ?? ??
t/03_clickhandler.t 29 7424 8 16 1-8
Failed 4/4 test scripts. 9/9 subtests failed.
Files=4, Tests=9, 3 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU)
Failed 4/4 test programs. 9/9 subtests failed.
NMAKE : fatal error U1077: 'C:\Perl\bin\perl.exe' : return code '0x1d'
Stop.
From above log, I could see that:
class Error: com.gargoylesoftware.htmlunit.WebClient could not be found.
I have no idea that I missed anything.
Any help would be appreciated.
Thanks.
Minh.

I found it.
There's different between path in Unix and Windows system. Unix uses ':' for a delimiter but Windows uses ';'. So what I've done is that open HTMLUnit.pm and change all of ':' to ';'.
With HTMLUnit version 0.15 I made changes at these lines below:
Line 78:
return join ';', map { "$jar_path/$_" } qw( # return join ':', map { "$jar_path/$_" } qw(
Line 127:
$custom_jars = join(';', #{$parameters{'jars'}}); # $custom_jars = join(':', #{$parameters{'jars'}});
Line 148:
CLASSPATH => collect_default_jars() . ";" . $custom_jars, # CLASSPATH => collect_default_jars() . ":" . $custom_jars,
And it works like a magic.

(it wouldn't let me comment on an existing answer)
I see your answer about ':' vs ';'. I'll try to include a fix in the next WWW::HtmlUnit release (I am the author of the perl bindings).

TInyOS 1.x Generating an error when compiling BLINK

root#everton-laptop:/opt/tinyos-1.x/apps/Blink# make pc
compiling Blink to a pc binary
ncc -o build/pc/main.exe -g -O0 -board=micasb -pthread -target=pc -Wall -Wshadow -DDEF_TOS_AM_GROUP=0x7d -Wnesc-all -fnesc-nido-tosnodes=1000 -fnesc-cfile=build/pc/app.c Blink.nc -lm
In file included from /opt/tinyos-1.x/tos/platform/pc/packet_sim.h:55,
from /opt/tinyos-1.x/tos/platform/pc/nido.h:81,
from /opt/tinyos-1.x/tos/platform/pc/hardware.h:43,
from /opt/tinyos-1.x/tos/system/tos.h:144:
/opt/tinyos-1.x/tos/types/AM.h:155: parse error before `struct'
/opt/tinyos-1.x/tos/types/AM.h:156: parse error before `struct'
/opt/tinyos-1.x/tos/types/AM.h:158: parse error before `struct'
/opt/tinyos-1.x/tos/types/AM.h: In function `TOS_MsgLength':
/opt/tinyos-1.x/tos/types/AM.h:186: parse error before `TOS_Msg'
In file included from /opt/tinyos-1.x/tos/platform/pc/hardware.h:116,
from /opt/tinyos-1.x/tos/system/tos.h:144:
/opt/tinyos-1.x/tos/platform/pc/eeprom.c: At top level:
/opt/tinyos-1.x/tos/platform/pc/eeprom.c:115: warning: declaration of `length' shadows global declaration
/opt/tinyos-1.x/tos/types/AM.h:158: warning: location of shadowed declaration
/opt/tinyos-1.x/tos/platform/pc/eeprom.c:145: warning: declaration of `length' shadows global declaration
/opt/tinyos-1.x/tos/types/AM.h:158: warning: location of shadowed declaration
make: *** [build/pc/main.exe] Error 1
Tried compiling BLink and i keep getting the above error and not sure what to do next. any help would be nice .

Looking at the CVS repository for tos/types/AM.h, it looks like it's choking on the following code:
154: enum {
155: MSG_DATA_SIZE = offsetof(struct TOS_Msg, crc) + sizeof(uint16_t), // 36 by default
156: TINYSEC_MSG_DATA_SIZE = offsetof(struct TinySec_Msg, mac) + TINYSEC_MAC_LENGTH, // 41 by default
157: DATA_LENGTH = TOSH_DATA_LENGTH,
158: LENGTH_BYTE_NUMBER = offsetof(struct TOS_Msg, length) + 1,
159: TINYSEC_NODE_ID_SIZE = sizeof(uint16_t)
160: };
The common item from lines 155, 156, and 158 are the offsetof() macro, which should be defined in stddef.h and which looks like should have been brought in by the tos.h before it causes AM.h to be included, so offsetof() should already be defined.
You might want to verify whether or not the compiler you're using has offsetof() properly defined and/or why it's not available for use in AM.h. If necessary, you can probably define it yourself using a more or less usual implementation:
#define offsetof(st, m) ((size_t) ( (char *)&((st *)(0))->m - (char *)0 )) // stolen from Wikipedia