This is in terms of code compilation and nothing else.. :)
So, I am a newbie in my company and predictably got stuck with an awesomely slow computer. And I am having a big problem with my Netbeans running out of memory/resource every time I make a build. I am compiling my JAVA files.
I was using 7.0, and even though I was getting this error, I got by it by compiling the source packages in chunks. (sometimes I had to compile the selected ones more than once)
But ever since I moved to 7.2, this problem is getting worse. I have to now compile the packages in even more smaller chunks. Sometimes package by package and file by file. Hence costing me a lot of time and even lot of hair.
I have no idea which packages to compile first. The netbeans was taking care of that. Therefore, taking resources.
Most of my colleagues have powerful computers and have no problem building the whole source base. So, I started getting the complied packages and only building the required ones.
So, is this the correct approach or building the whole source (even though I just make changes to 1% of the total code base, at any given time)?
Almost everyone in this company is building the whole code base, at least once, even though most of the changes are only in 1%.
It is far better to build the entire project and have it work as designed, then build 99% of it and it doesn't work. There's no indication that the 1% is critical or non-critical code, and as a beginner, you can't tell that just right off the bat.
I would inform your teammates/IT personnel about the slow build and ask what can be done to resolve it, instead of building the code in chunks.
Maybe you should highlight the issues with a developer having a slow machine impeding the work you are doing, when you explain the difference in lost productivity versus hardware cost, you will shortly have a new machine.
Then you can stop worrying about building "99%" and get on to real issues.
It's better to build the entire project. Try tune netbeans.conf
netbeans_default_options="-J-client -J-Xss4m -J-Xms128m -J-XX:PermSize=128m -J-XX:MaxPermSize=512m -J-Dapple.laf.useScreenMenuBar=true -J-Dapple.awt.graphics.UseQuartz=true -J-Dsun.java2d.noddraw=true-J-XX:+UseParNewGC -J-XX:+UseConcMarkSweepGC -J-XX:+CMSClassUnloadingEnabled -J-XX:+CMSPermGenSweepingEnabled"
So, is this the correct approach or building the whole source (even
though I just make changes to 1% of the total code base, at any given
time)?
I think that you can build only some parts of project only if you perfectly know all internal dependencies and can guaranty that no unexpected behaviour in nearby module happens after your modifications were made. It is my opinion. Moreover, you can change code and compile it succesful, but the entire project build can fail the same.
P.S. You should get company to buy you a new computer.
In theory you could walk through all dependencies and make yourself a dependency hierarchy map, and you should only have to compile the code you've changed plus everything that depends on it. However it's not necessarily 100% foolproof and requires A LOT of effort for very little gain. It's not something that I would expect to be a newbies responsibility to sort out, rather your superiors should get you sorted out with some appropriate kit.
Related
I am doing a project that requires me to repeatedly run a java function in python (it's like designing a learning algorithm in python but the value function was provided in java)
So what would be the practice for this scenario? Shall I use subprocess.run() to call the java function every time or shall I use the things like Py4J, Jython or JPype? What's the difference between using subprocess.run() and the others?
The efficiency should be the top concern since I need to run the same java function repeatedly.
Using subprocess has two problems. If neither one is relevant, it'll work fine.
If you're sending large amounts of data back and forth, you have to serialize it in some format to pass in via files and command-line arguments, or pipes or sockets, which can be slow.
If you're calling a whole lot of short functions instead of one occasional huge one, you'll be spending more time setting up and tearing down the JVM (and warming up the JIT) than doing actual work.
Jython has two problems. Again, if neither one affects you, it'll work fine.
It can't use many popular third-party libraries because they're built in C, for CPython.
It's out of date. The latest version implements Python 2.7, which is less than 2 years away from going out of support.
JPype has one problem, but it's a doozy. If the current fork does what you need and has no bugs blocking you, maybe it's ok anyway.
It's a vaporware project abandoned over a decade ago. It was picked up and knocked into shape by someone else a few years ago, and the current maintainer is keeping it running, and occasionally gets patches for things like working in 64-bit cygwin or updating to OS X 10.9, but it's not exactly a vibrant project with major support behind it.
Py4J has two problems.
It's incomplete. Not unusuable, and not completely moribund, but there hasn't been any visible work on it in over a year, and nobody seems interested in anything but the minimal functionality needed for Apache Spark.
It's doing the same kind of serialization you'd do with subprocess behind your back, and more beyond that for every call you make, and the FAQ justifies this by saying performance is not a priority. (Spark just ignores all of that and uses its own channels for everything.)
For more minimal use—just starting up a JVM and setting up a socket to it—it may be better than subprocess because you don't have to keep starting and tearing down a JVM, but writing a socket protocol on both sides is a little bit more work than storing files and passing filenames on the command line. (Not a huge hurdle, but a problem if you've never done this kind of thing before.)
You may also want to look at transpilers. I don't know much about any of them, but I've talked to people who are using BeeWare to compile Python 3.4 code to Java source code that they then build together with their native Java code. I'm pretty sure this won't work if you're using any C extension, but if that's not a problem for you, it might be worth considering.
I am developing an application with Android Studio, but I have a problem. I must always delete the cache of the IDE (Invalidate/Caches restart) for each compilation.
How do you configure your IDE for avoid this problem ?
Thank in advance,
It looks like that the actual problem is why you have to clean the cache on every compilation. You don't need to clean the cache on every compilation.
However, the current version of the doc tells that the cache can become overloaded and cause problems:
IntelliJ IDEA caches a great number of files, therefore the system
cache may one day become overloaded. In certain situations the caches
will never be needed again, for example, if you work with frequent
short-term projects. Also, the only way to solve some conflicts is to
clean out the cache.
PS At the same time, #yole tells that There is no such thing as the IntelliJ IDEA cache being overloaded.
This seems to be a problem with the on-the-fly code parser. As I am typing something, eclipse (latest release version) might update to reflect that I have an error. But when I finish typing the line, the error will still remain in the Problems tab and never get fixed, no matter what changes I make to that file or any other file. The only way to fix this problem is to go to Project->Clean... which of course works every time.
I use a rapid prototyping technique where I need to test my project hundreds of times throughout a day. This error happens about 10% of the time I want to run or debug a project, which is 10 times every day. Cleaning my project takes a significant number of seconds since we have hundreds of resources that need to be re-parsed. This is very frustrating and is killing my productivity. Is there any fix or patch to eclipse to fix this problem? If not, does anyone know of a workaround?
Right click your project and hit refresh. It usually works and is much faster than a full clean and build.
can you please expand the errors ?
I guess this must be due to linking error..
ensure all the required jar are added, that should resolve.
After adding Google Guava r09 to our Android project the build time increased significantly, especially the DEX generation phase. I understand that DEX generation takes all our classes + all jars we depend on and translates them to DEX format. Guava is a pretty big jar around 1.1MB
Can it be the cause for the build slowdown?
Are there anything can be done to speed this up?
P.S. Usually I build from Intellij, but I also tried building with Maven - same results.
Thanks
Alex
For what it's worth, my gut is that this isn't the cause. It's hard to take a long time doing anything with a mere 1.1MB of bytecode; I've never noticed dex taking any significant time. But let's assume it is the issue for sake of argument.
If it matters enough, you could probably slice up the Guava .jar to remove whole packages you don't use. It is composed of several pieces that aren't necessarily all inter-related.
I don't think this is going to speed things up, but maybe worth mentioning: if you run the build through Proguard (the optimizer now bundled with the SDK), it can remove unused classes before you get to DEX (and, do a bunch of other great optimization on the byte code). But of course that process probably takes longer itself than dex-ing.
I find java starts up and runs practically instantly for me - but javac takes a few seconds, and ant makes it slower again. It's only a few seconds, but my edit-compile-test loop would be smoother without it. :-)
BTW: I've already used vim's ":make" with ant.
Is there a way to speed up javac and/or ant? I'm thinking of special switches, or tricks? Or maybe an alternative java compiler (I'm using 1.6, in linux)
Eclipse does that for you ... but it's probably a bit big as a "patch" for your problem.
That aside, you can roll your own compiler plugin. There are two approaches:
Run the java compiler from within ant (instead of creating a new process). Not sure if ant already does that; if not, that will save you a bit of time. Look at the Java 6 compiler API for details.
Run javac in a server process which listens for options on a socket. In ant, send the process the command line and wait for the output to be sent back. Saves you starting a new process all the time.
Try to use the eclipse compiler. Unlike the original javac, the Eclipse compiler is pretty good at "ignoring" errors, so it can produce class files even when they contain errors. That doesn't seem to mean much but it allows you to compile all the time in the background. When you do your last save (wrapping everything up), the compiler will have been able to compile everything else and will just have to look at a single file.
Google found these two (I haven't tried either yet)
javac -J-client -J-Xms100m -J-Xmx100m <src>
JBrownie "monitors Java sourcecode and automatically recompiles any files found changed" along the lines of Aaron Digulla's (2)
I can strongly recommend using a suitable IDE with Java as the productivity increase with using an editor which knows about your program is immense. Think of "goto the line containing the defintion of the variable/class/field the cursor is on", "rename this class and all references to it", and all other kinds of nice things. If you are annoyed with the time it takes to invoke Java, you might be ready :)
You may want to have a look at JavaRebel
I point all of my output files to a ramdisk that I create with this utility. It speeds up builds a fair bit (Though not to a magical extent) since almost everything is in memory. It's most noticeable when doing a 'clean' build which becomes almost instant.