Develop Reference

Are there any Java Class Library "header files" containing all method descriptors in the standard library?

In order to create a valid .class file, every method has to have a full internal name and type descriptors associated with it. When procedurally creating these, is there some sort of lookup table one can use (outside of Java, where a ClassLoader can be used) to get these type descriptors from a method name? For example, how would one go from Scanner.hasNextByte to boolean java.util.Scanner.hasNextByte(int) / boolean java.util.Scanner.hasNextByte() (or even from java.util.Scanner.hasNextByte to boolean java.util.Scanner.hasNextByte(int) / boolean java.util.Scanner.hasNextByte())? The above example has overloading in it, which is another problem a human- but mostly computer-readable declarations file would hopefully address.
I've found many sources of human-readable documentation like https://docs.oracle.com/javase/8/docs/api/index.html containing uses of each method, hyperlinks to other places, etc. but never a simple text file or collection of files containing just declarations in any format. If there's no such file(s) don't worry about it, I can try and scrape some annoying HTML files, but if there is it would save a lot of time. Thanks!

The short answer is No.
There isn't a "header file" containing the class and method signatures for the Java class libraries. The Java tool chain has no need for such a thing. Nor do 3rd-party Java compilers, or compilers for other languages that rely on the Java SE class libraries.
AFAIK, there isn't a 3rd-party tool that builds such a file or an equivalent database or in-memory data structures.
You could create one though.
You could chose an existing Java parsing library, and use it to build parse trees for all of the source files in the class library, and emit the information that you need.
You could potentially create a custom Javadoc "doclet" plugin to emit the information.
Having said that, I don't understand why you would need such a mapping. Surely, your IDE does this already ... and exposes the information via some internal API. And if this is not for an IDE plugin, what it is for?
You commented:
I'm making a compiler for a JVM-based programming language ....
Ah ... so your compiler should do what other compilers do. Get the information from the ".class" file. You can either load the class using a standard or custom class loader, or you can use a library like asm or bcel or javassist ... which can read a ".class" file without loading it.
(I haven't checked, but I think the standard javac compiler uses an internal API to do this.)
Note that your proposed approaches won't work for interfacing with 3rd-party Java libraries where the source code is not available and/or the javadoc is not scrapable.

What about building it from the source files for the standard library?
The Oracle Java 8 API web pages you referenced was created by Javadoc processing of source files for the Java standard library.
If you use an IDE with a debugger, there is a good chance you already have much of the standard library source code downloaded. After all, if you set a break point, and then follow the program step-by-step with "Step into", you can trace the execution of the program into standard library methods. The source files would be part of the JDK.
However, some parts of the standard library source might not be available, due to licensing restrictions.

Java - make a library and import optional

I have a library that I'm using in an Java application - it's important for certain functionality, but it's optional. Meaning that if the JAR file is not there, the program continues on without issue. I'd like to open source my program, but I can not include this library, which is necessary to compile the source code as I have numerous import statements to use the API. I don't want to maintain two code sets. What is the best way to remove the physical jar file from open source release, but still maintain the code to support it where other people could still compile it?

the typical approach taken is to define the wrapper API (i.e. interfaces) and include those interfaces in the open sourced code, and then provide configuration options where one can specify class names of classes that implement certain interfaces.
You will import API interfaces instead of importing classes directly into your open sourced code. This way, you are open sourcing the API but not the implementation of the parts that you do not want to open source or you cannot open source.
There are many examples, but take a look at JDBC API (interfaces) and JDBC drivers (implementation classes) for starters.

I was pretty much typing the same thing as smallworld with one addition. If this API were necessary you can use a project build tool like Maven to handle the dependencies on you project. If someone checks it out from source control with the pom they can download the dependencies for themselves and you don't have to include them in a source repo.

There's probably a number of ways to fix this, here's a couple I can think of:
If you have only a couple of methods you need to invoke in the 3rd party library, you could use reflection to invoke those methods. It creates really verbose code, that is hard to read though.
If you don't have too much of the API in the 3rd party library you use, you could also create a separate JAR file, containing just a non-functional shell of the classes in the library (just types with the same names and methods with the same signatures). You can then use this JAR to distribute and compile against. At run-time you'd replace it with the real JAR if available.
The most common way is probably to just create a wrapper API in a separate module/project for the code that is dependent on the 3rd party library, and possibly distribute a pre-built JAR. This might go against your wish to not maintain two code sets, but may prove to be the best and less painful solution in the long run.

changing java's .class file without recompiling

is there any way to do the following. so i have a project.jar file, inside it i need to modify the string passed to some method of let's say classA.class. for example, let's say this classA.class has a method named
change(String a, String b)
what i all want is to do the following as the first line of the code inside this method as follows:
a = a + "hi";
i want to modify the .class file directly, without needing to recompile everything again. then after that i would update the jar file with the new class file. is it possible? if yes can anyone show an example? thanks a lot in advance!

Yes it's possible with byte code editors. Commonly you can use aspectj together with compile time weaving to modify a class file. You can also use libraries such as BCEL, cglib etc.
However, for the use case you are describing you typically don't need to edit a class file. You can just wrap your object in a proxy or modify it's behaviour using AoP style of programming (as supported by e.g., aspectj)

There are tools to manipulate byte code dynamically, such as ASM: http://asm.ow2.org/

another way could be instrumentation. when you are loading the class you could manipulate the bytecode before using it. there are some good libraries for this, for example javassist from jboss. i think aspectJ works similar.
but why you want to change the bytecode, compile it and add it to a jar file again? are you needing the source code? perhaps you could use some de-compiler to get the source code, if you need it. a good tool is http://java.decompiler.free.fr/.

Retro-actively add Java annotations to methods?

Is there a way to modify .class files in order to add Java annotations to certain methods? Basically I want to traverse methods of each class file in a jar file and annotate certain ones. Note that this is not at run-time while using the jar file. Rather, after I'm done I want to have modified class files with the annotations.
I do have access to the source code, so if there's an automatic source code modifier, that would work as well...
I'm assuming I'll need a tool such as Javassist or ASM. If so, which one should I use and how would I go about it?

Actually, this is a classic use case for AspectJ:
declare #method : public * BankAccount+.*(..) : #Secured(role="supervisor")
While I will grant you that direct byte code manipulation is more powerful, AspectJ is much more user-friendly, and it immediately gives you compiler warnings when you are doing something wrong.
Also, if you use Load Time Weaving, you can leave the original library jar unchanged, because the weaving happens at class-load time.
Reference:
Declare Annotation
AspectJ in Action (book)

Googling for an hour or so turned this article up which seems to completely answer my question: use ASM. To write class files using the changed bytecode, use ClassWriter.
Well, time to get to work then, I guess. :)

Compile Java class with missing code parts

I'm looking for some ideas on how to compile Java code with some other pieces of code missing (method calls). I am fully aware that javac will not allow you to compile Java files if cannot find all dependencies. But maybe there is some way how to bypass it, something like force compile.
My bytecode knowledge is not so good but I think some method invoke is just full package definition of class and method name with parameters. So if compiler just puts this data to class file and assume in running process dependency will be available (if not simple NoSuchMethodExp).
Only workaround so far I found is to create empty missing class files with empty methods to "cheat" compiler. Works perfectly but there should be easier way :)
Any ideas?

Use Interfaces.
Create the interfaces that have the methods you need. At runtime, inject (Spring, Guice, etc.) or generate (cglib ...) classes that implement the interface.

If you're modifying a jar, you can extract the class files you are not modifying to another directory and include that in the classpath. That way they will be available to the compiler.

Bad luck! Probably all you can do is to create mock objects for missing parts of code just to compile your code (empty methods, so the compiler can find it).
Another question - if you miss some classes, how will you execute that code?
UPDATED according to information provided:
Well, there is another option to modify classes in jar, you can use AOP, and to make it done read about AspectJ - actually for me this is the easiest option (typically you need to spend time mocking objects, writing empty methods, so I would contribute that time to study new technology, which will help you many times ;)
And btw the easiest way to implement it, if you use Eclipse, is:
install AJDT
create aspect project
create aspect which modifies code (depending on what you need to change)
add jar file you want to modify
immediately get modified code in
another already packed jar file
Sounds magically :)
In this case you don't need any dependencies in classpath, except for libraries which are needed for new code you add!

Methods aren't dependencies. They are part of the class definition. The only places the java runtime looks for method definitions is in the class def that was compiled at compile time and in its parent classes. If you're problem is that a super class is incomplete, I don't think I can help you.
If not, you could define some of these methods as abstract and than have a child class implement them.

What kind of code is missing? Normally this happens if you refer to libraries your compiler can't find. Maybe you simply need to extend the classpath the compiler is searching for classes.
If you really refer to code that is not available yet you need to implement at least those methods you refer to. But that sounds strange... maybe you can clear things up.