Develop Reference

How to approach writing algorithm from a complex research paper

I thought of writing a piece of software which does Alpha Compositing. I didn't wanted ready made code off from internet so I tried to find research papers and other sources to understand the mathematical algorithms, and initiated to implement.
But, I got lost very quickly. So my question is,
How should I approach these papers to extract the necessary details from it in order to write algorithm based on it. Any specific set of steps which works well?
Desired answer :
Read ...
Extract ...
Understand ...
Implement ...
Note: This question is not limited to only Alpha Compositing, so more generalised approach will be helpful. I have tagged Java and C++, because thats my desired language to implement the image processing.
What I have done so far?
This is not a homework question but it is of course better to say what I know. I have read wiki of Alpha compositing, and few closely related Image compositing research papers. But, I stuck at the next step to take in order to go from understanding to implementation.
Wikipedia
Technical Memo, Image compositing

I'd recommend reading articles with complex formulas with a pencil and paper. Work through the math involved until you have a good grasp on it. Then, you'll be ready to code.
Start with identifying the steps needed to perform your algorithm on some image data. Include all of the steps from loading the image itself into memory all the way through the complex calculations that you may need to perform. Then structure that list into pseudocode. Once you have that, it should be rather easy to code up.

Write pseudocode. Ideally, the authors of the research papers would have done this, but often they don't. Write pseudocode for some simple language like Matlab or possibly Python, and hack away at writing a working implementation based on the psuedocode.
If you understand some parts of the algorithm but not others, then implement your pseudocode into real code for the parts you understand, and leaving comments for the places you don't.
The section from The Pragmatic Programmer on "Tracer Bullets" basically describes this idea. You want to quickly hack together something that takes your data into some form of an output, and then iterate on the body of the code to get it to slowly resemble the algorithm you're trying to produce.
My answer is necessarily somewhat vague. There's no magic bullet for something like this.

Have you implemented any image processing algorithms? Maybe start with something a little simpler, like desaturation/color intensification, reversal (side to side and upside down), rotating, scaling, and compositing images through a mask.
Once you have those figured out, you will be in a very good position to do an alpha composite.
I agree that academic papers seem to go out of their way to make implementation details muddy and uncertain. I find that large amounts of simplification to what is written is needed to begin to perform a practical implementation. In their haste to be general, writers excessively parameterize every aspect. To build useful, reliable software, it is necessary to start with something simple which actually works so that it can be a framework to add features. To do that, it is necessary to throw away 80–90 percent of the academic generality. Often much can be done with a raft of symbolic constants, but abandoning generality (say for four and five dimensional images) doesn't really lose anything in practice.

My suggestion is to first write the algorithm using Matlab to make sure that you understood all the steps and then try to implement using C++ or java.

To add to the good suggestions above, try to write your pseudocode in simple module (Object oriented style ) so has to have a deep understanding of each part of your code while not loosing the big picture. Writing everything in a procedural way is good a the beginning but as the code grow, it might get become hard to keep up will all you are trying to do.

This example cites one of the seminal works on the topic: Compositing Digital Images by Porter & Duff. The class java.awt.AlphaComposite implements the same rules.

ML technique for classification with probability estimates

I want to implement a OCR system. I need my program to not make any mistakes on the letters it does choose to recognize. It doesn't matter if it cannot recognize a lot of them (i.e high precision even with a low recall is Okay).
Can someone help me choose a suitable ML algorithm for this. I've been looking around and find some confusing things. For example, I found contradicting statements about SVM. In the scikits learn docs, it was mentioned that we cannot get probability estimates for SVM. Whereas, I found another post that says it is possible to do this in WEKA.
Anyway, I am looking for a machine learning algorithm that best suites this purpose. It would be great if you could suggest a library for the algorithm as well. I prefer Python based solutions, but I am OK to work with Java as well.

It is possible to get probability estimates from SVMs in scikit-learn by simply setting probability=True when constructing the SVC object. The docs only warn that the probability estimates might not be very good.
The quintessential probabilistic classifier is logistic regression, so you might give that a try. Note that LR is a linear model though, unlike SVMs which can learn complicated non-linear decision boundaries by using kernels.

I've seen people using neural networks with good results, but that was already a few years ago. I asked an expert colleague and he said that nowadays people use things like nearest-neighbor classifiers.
I don't know scikit or WEKA, but any half-decent classification package should have at least k-nearest neighbors implemented. Or you can implement it yourself, it's ridiculously easy. Give that one a try: it will probably have lower precision than you want, however you can make a slight modification where instead of taking a simple majority vote (i.e. the most frequent class among the neighbors wins) you require larger consensus among the neighbors to assign a class (for example, at least 50% of neighbors must be of the same class). The larger the consensus you require, the larger your precision will be, at the expense of recall.

Where to code this heuristic?

I want to ask a complex question.
I have to code a heuristic for my thesis. I need followings:
Evaluate some integral functions
Minimize functions over an interval
Do this over thousand and thousand times.
So I need a faster programming language to do these jobs. Which language do you suggest? First, I started with Java, but taking integrals become a problem. And I'm not sure about speed.
Connecting Java and other softwares like MATLAB may be a good idea. Since I'm not sure, I want to take your opinions.
Thanks!

C,Java, ... are all Turing complete languages. They can calculate the same functions with the same precision.
If you want achieve performance goals use C that is a compiled and high performances language . Can decrease your computation time avoiding method calls and high level features present in an interpreted language like Java.
Anyway remember that your implementation may impact the performances more than which language you choose, because for increasing input dimension is the computational complexity that is relevant ( http://en.wikipedia.org/wiki/Computational_complexity_theory ).

It's not the programming language, it's probably your algorithm. Determine the big0 notation of your algorithm. If you use loops in loops, where you could use a search by a hash in a Map instead, your algorithm can be made n times faster.
Note: Modern JVM's (JDK 1.5 or 1.6) compile Just-In-Time natively (as in not-interpreted) to a specific OS and a specific OS version and a specific hardware architecture. You could try the -server to JIT even more aggressively (at the cost of an even longer initialization time).
Do this over thousand and thousand times.
Are you sure it's not more, something like 10^1000 instead? Try accurately calculating how many times you need to run that loop, it might surprise you. The type of problems on which heuristics are used, tend to have a really big search space.

Before you start switching languages, I'd first try to do the following things:
Find the best available algorithms.
Find available implementations of those algorithms usable from your language.
There are e.g. scientific libraries for Java. Try to use these libraries.
If they are not fast enough investigate whether there is anything to be done about it. Is your problem more specific than what the library assumes. Are you able to improve the algorithm based on that knowledge.
What is it that takes so much/memory? Is this realy related to your language? Try to avoid observing JVM start times instead of the time it performed calculation for you.
Then, I'd consider switching languages. But don't expect it to be easy to beat optimized third party java libraries in c.

Order of the algorithm
Tipically switching between languages only reduce the time required by a constant factor. Let's say you can double the speed using C, but if your algorithm is O(n^2) it will take four times to process if you double the data no matter the language.
And the JVM can optimize a lot of things getting good results.
Some posible optimizations in Java
If you have functions that are called a lot of times make them final. And the same for entire classes. The compiler will know that it can inline the method code, avoiding creating method-call stack frames for that call.

Is a Java book enough or should I have to learn algorithms first? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am new to Java and referring Head-First book.
Do I have to learn Algorithms to be able to make programs in Java?
Should I learn Algorithms first or only Java books like Effective Java, Java puzzlers etc will be enough?
I want to be a successful enterprise developer. Then what algorithms and data structures I should be well versed with? What books would you recommend me?
To be a successful Java developer, do I need to know all the advanced algorithms such as those given in CLRS?
PS: I had C and C++ in my past semesters, I got good marks in them but that was kinda mugging. I know basics of programming. I am not a newbie. Thing is that I don't have sound knowledge in any language also I want to be a developer not an algorithmist.

If you want to become a successfull developer you'll need to learn a lot of stuff. A main programming language (java) and CS basics (e.g. algorithms) are just two of them. Others are: a communication skills, testing (I'd say TDD but I don't want to start a fight), handling of databases, web stuff, OOD and many more. Also a lot of stuff that you might not use directly will still help a lot by broadening your horizon (functional programming, advanced CS concepts)
There is no fixed order in which to learn that stuff. Since learning works best when you are motivated, just pick what you are interested in most or what is helping you most. Learn a little here, then learn a little there. Keep your eyes open and practice a lot and you will be fine.
In your current situation I'd recommend to pick up an Algorithms book and implementing the algorithms there in java. It teach you java and algorithms.
Also read "Clean Code", "The Pragmatic Programmer" and the SOLID http://www.butunclebob.com/ArticleS.UncleBob.PrinciplesOfOod principles

My advice - learn a subset of Java, then learn some basic algorithms, then figure out where you want to go from there.
You don't need to know all of Java to experiment with implementing algorithms. You will need to be able to wrap a few methods (often only one) in a single class (and even that only because Java insists everything's a class). You'll need conditional and looping constructs and arrays. For data structures, you'll probably need to understand Java references, though you may be able to fake pointers using integers as array subscripts.
The point is that algorithms are important, but on a relatively small scale. Data structures typically need a few algorithms to make them work. But you don't need an understanding of larger-scale issues such as object-oriented design to experiment with algorithms.
EDIT
From comments, I see that you're already well into the "where to go from there" stage.
Don't forget algorithms (or discrete math) because you don't care about analysis, though. A broad (not necessarily in depth) understanding is a useful resource - they are great for problem-solving methods.
For example, I was recently confused in a dependency-ordering issue about how to deal with cycles. I'd forgotten about "strongly connected components" in digraphs. Once I was reminded, the problem became trivial - no point trying to order within a strongly connected component, but those components form an acyclic digraph. From there, the answer is just a topological sort away.
Knowing about topological sorts makes the last step trivial. Having forgotten about strongly connected components cost me a fair bit of time. Understanding how Tarjans strongly connected components algorithm works... Wikipedia and a few minutes with pen and paper are enough, once you know what to look for.
Actually, I should confess - "I was reminded" means I looked up an old Dr. Dobbs article on topological sorting that used the same approach.

I recommend you begin learning with Head First. functionx.com is also a good site. Most Java books describe algorithms as part of the book, so don't worry about algorithms. First of all learn Java.
Let me give you a easy way to find good books. Always look for higher rated books in amazon.com and then download them from wowebook.be.
I recommend "Data structures and algorithms in Java" by Michael T. Goodrich and Roberto Tamassia.
"Think Like a Programmer" are good algorithm-related books.

I'm with Steve314 for the most part but wanted to specify some things.
Learn enough Java (or whatever else) to be dangerous. Then the main data structures (lists, stacks, maps, trees, etc), which does include some algorithms for traversing them. Do the rest of your algorithms work after that. Here are some specific goals, if you want them, that more or less mirror the order of Data Structures & Algorithms in Java.
Understand Java and OO on a basic level. Specifically inheritance and polymorphism
(as Java define them) and how to use generics.
Have a basic understanding of Big-O
(how it's defined and why you can
drop lower order terms).
Be able to use, write, and trace recursive functions.
Code singly- and doubly-linked lists. They should implement Iterable and support adding and removing elements.
Use them to implement a stack and a queue.
A couple of things to know about once you do dive into a book like Algorithms:
MIT OCW has videos lectures up from the class that Algorithms was written for, given by one of its authors.
The purpose of a book like Algorithms is not just to show you specific algorithms, but to teach you how to analyze their running times. You need a small amount of discrete math for that, basically sums and recurrence relations. Look on here for big-O questions to see what I mean.
Speaking of videos, I think this is a way better place to start: Programming Abstractions (Stanford, Julie Zelenski). I'm very happy that those exist.
(As it happens I am in the middle of implementing a map with a binary search tree.)

#Chankey: According to me you should take both java and algorithms hand in hand. First learn some basics of Java programming language like what are classes, data types, functions etc. Then learn some basic algorithms of sorting and searching. And, now apply your java knowledge to implement these algorithms.
Cheers!!
Mukul Gupta

Function point count given source code

Given the source code of a program, how do I analyze it and count the function points within it?
Thanks!

You might find this tutorial on FPA of interest. Personally, I don't put much stock in this estimation method. From my perspective it attempts to provide a precise estimate in for things that have been shown repeatedly to not be precisely measurable. I much prefer planning poker or something similar that tries to group things within a similar order of magnitude and provide an estimate based on your previous estimations for similarly sized stories.
If you're doing this for a class, simply follow the rules given in the text book and crank out the answer. If you're really intending to try this as a software development estimation method, my advice is to simplify the process rather than make it more complex. I would imagine that members of the International Function Point User Group (yes, there is one), will disagree.

With a code analysis tool. If you want to write one yourself, you might want to start with cglib or ASM.