Load different resources when testing in Python - java

First of all let me put you in context. My main background is Java and I'm working in Python since 2 months ago. I don't know if the approach it's wrong due to my Java background and in Python has a different solution or it's just a technical ignorance problem.
In Java often you have a packaging structure like:
project
|___src
|___main
| |___java/MyClass.java
| |___resources/properties.file
|
|___test
|___java/MyClassTest.java
|___resources/properties.file
Thus, when you execute this from tests (with Maven or IDE):
this.getClass().getResourceAsStream(resourcePath);
Either Maven or IDE loads the test path in the classpath, making test resources available rather than the ones within the main package.
Conversely, when previous line is executed as main, only the resources within the main path are loaded to the classpath.
My question is: Is there any mechanism in Python to simulate this feature? Does Python have other ways to manage resources depending on the execution path?


I think mock might be what you are looking for. It allows for 'mock'ing out external functions to limit the test to strictly the unit under test.
This could mean changing some of the philosophy around some of it. For example, if you want to test that a function is reading a file correctly the filename would get passed to the method as a parameter. In your unit test for the function, pass in a different filename from your test folder.
In another test mock out the open call to the os to check that the method responds as expected when the file is not there, or cannot be opened, or whatever other mis-behaviors you want to test for.


No, Python has no deployable like Java has (WAR, JAR, etc...). You will run your code directly from the source, so, just read the file.
In the Java context, you do not have the code when it is deployed. So every resource should be package inside a file (JAR or WAR).
If you want to find the file in the current folder, look this question.


I don't know very well Python but you are right to ask yourself the question
as separating test and application code makes part of good practice to have a robust/reliable application and tests.
The pytest (a known test framework for Python) documentation explains in its best practice guide the two ways (separating and not separating the test code from the application).
Here is the part referencing the isolated layout :
Choosing a test layout / import rules
pytest supports two common test layouts:
Tests outside application code
Putting tests into an extra directory outside your actual application
code might be useful if you have many functional tests or for other
reasons want to keep tests separate from actual application code
(often a good idea): setup.py
mypkg/
init.py
app.py
view.py
tests/
test_app.py
test_view.py
...
This way your tests can run easily against an installed version of
mypkg.
Note that using this scheme your test files must have unique names,
because pytest will import them as top-level modules since there are
no packages to derive a full package name from. In other words, the
test files in the example above will be imported as test_app and
test_view top-level modules by adding tests/ to sys.path.
If you need to have test modules with the same name, you might add
init.py files to your tests folder and subfolders, changing them to packages: setup.py
mypkg/
...
tests/
init.py
foo/
init.py
test_view.py
bar/
init.py
test_view.py
Now pytest will load the modules as tests.foo.test_view and
tests.bar.test_view, allowing you to have modules with the same name.
But now this introduces a subtle problem: in order to load the test
modules from the tests directory, pytest prepends the root of the
repository to sys.path, which adds the side-effect that now mypkg is
also importable. This is problematic if you are using a tool like tox
to test your package in a virtual environment, because you want to
test the installed version of your package, not the local code from
the repository.
In this situation, it is strongly suggested to use a src layout where
application root package resides in a sub-directory of your root:
setup.py
src/
mypkg/
init.py
app.py
view.py
tests/
init.py
foo/
init.py
test_view.py
bar/
init.py
test_view.py This layout prevents a lot of common pitfalls and has many benefits, which are better explained in this
excellent blog post by Ionel Cristian Mărieș.
https://docs.pytest.org/en/latest/goodpractices.html

Related

how to include all subdirectories as gradle modules?

I'm going to have a lot of submodules in my main project directory x, like x/module1, x/module2...
can i avoid manually adding every single module into settings.gradle? can i somehow script it to find all the subdirectories and add them automatically?

As cricket_007 already mentioned, Gradle is based on the Groovy programming language (which is, like Java, executed in the JVM) and the settings.gradle file is nothing more but a Groovy script.
Whenever you use include 'project', the include method of a Settings instance is called, so for your goal, you could simply create a loop which iterates over all folders and calls include for each of them.
A more 'groovyesque' approach would be the usage of a closure for each subdirectory, provided by the Groovy SDK extension for the File class:
file('.').eachDir { sub ->
include sub.name
}
There are multiple ways to solve your problem, e.g. since the include method accepts an array of project path strings, you could also aggregate all required paths first and pass them all together. Simply get familiar with the Gradle docs and decide on your own, what solution suits your case the best.

Junit to use correct resources

When writing unit tests for the application, usually I create copy of some configurations that are in original main/resources and modify them for test purpose, but i leave the names the same. However Junit sometimes takes the one in src and sometimes the one in the test/resources.
How do we manage which one it picks up without renaming files?
For example i have some "config.json" which is in both test and main's resources, how does junit choose which one to pick when running a test...

Ok, just a quick answer:
You cannot exclude resources from src/main/resources. They are always inside your class path and thus they are part of your program/library.
But you shouldn't have to do so.
Log4j2 for example allows to have a differently named configuration file in the class path (while running the tests). Just name it log4j2-test.xml and it will be loaded prior to a non-test file.
Spring behaves the same when annotating the test classes with #ContextConfiguration.

How to make IntellijIDEA ignore work-in-progress class files?

When I'm working in IntellijIDEA how do I tell it to ignore a class file that may have problems and I want to leave dormant for a while?
It will throw errors when I compile whatever class I am working on until I fix the first "dormant" class.
I have tried adding my class to a bogus package but Intellij doesn't like that either because the path doesn't match.

Settings | Compiler | Excludes, add your WIP files there:

You could use Refactor -> Rename File..., and change the file extension.

That is set at the inspection level
Configure Current File Analysis CTRL + SHIFT + ALT + H
I have profiles with differing inspection levels setup loosely based on the phase of my project builds ... I'd suggest taking a look at Customizing Inspection Profiles.
To ignore specific files during compilation you can add files individually or recursively in via the project settings panel ...
Configure Compiler Analysis CTRL + ALT + S :: Compiler => Validation

A bit late, still
If all your files are in the same package, then right-click on the package in the Project tool window and pick "Mark directory as -> Excluded".
All the classes inside the package won't be compiled. You can cancel exclusion any time you want the same way.

Usually, only the classes that are used in the application are actually compiled.
In your case, I would guess that it's only broken unit tests that hinder the compilation (as opposed to any other Java classes in the /main folder).
The reason is this: When running all unit tests in a package or source folder, IntelliJ searches and includes all the files that appear like unit tests by default: those with Test or Suite in the class name, but also those annotated with #Test or #Suite.
So the easiest way to exclude your test is to create a third source folder, call it /ignore, and not mark it as a source folder in IntelliJ. You can then drop any file you don't want to include in your compilation there temporarily, and drag it back to its original folder when you want to continue working on it. Beware, though: You will get only limited tool support if you open and edit the file within an unmarked source folder, so it should really be used for "parking" only.
You could also change the file extension, as the other answer suggests, but then IntelliJ will also change its handling of the file in other respects, not just during compilation.
Also, if you're using JUnit 4, you can always annotate any single test method, or the entire test class, with #Ignore, and it will be skipped during the test run. This requires the class to be formally correct, though, i.e.: no compile time errors.
P.S: You need to actually move the test to a different folder, if you really want the package to change - not just edit the package declaration. Otherwise, a non-matching declaration will also be considered an error.

Import an Ant classpatch from another file

I'm exploring the awesomeness of Ant 1.8.1's import ability. Here's my situation: I have a top-level Ant file (project.xml) that turns around and calls ant on another Ant file (say, neato_project.xml) which actually does the build, or clean or whatever.
I have 12 different project files that this top-level (project.xml) file can call, so I want to put a common classpath entry into the project.xml file that I can pass to the others to use as their individual classpaths.
How do I do that? I've been trying to play with import task, but I haven't gotten that figured out. I'm open to another approach if there's a better way to approach this problem in Ant.

Import wasn't introduced in 1.8; it was enhanced in 1.8. This is good because it means people like me have a couple years experience with import.
What I do:
constants.xml - the common strings and classpaths my build uses
build.xml - the main file imports #1 and #3
helper-project-1.xml - it has a clearer name, but that's hardly the poing
helper-project-2.xml, etc
I use this approach because I want build.xml to pass the constants. I only keep them in a separate file for readability.

Java project structure explained for newbies?

I come from a .NET background and am completely new to Java and am trying to get my head around the Java project structure.
My typical .NET solution structure contains projects that denote logically distinct components, usually named using the format:
MyCompany.SomeApplication.ProjectName
The project name usually equals the root namespace for the project. I might break the namespace down further if it's a large project, but more often than not I see no need to namespace any further.
Now in Java, you have applications consisting of projects, and then you have a new logical level - the package. What is a package? What should it contain? How do you namespace within this App.Project.Package structure? Where do JARs fit into all this? Basically, can someone provide a newbies intro to Java application structure?
Thanks!
Edit: Some really cracking answers thanks guys. A couple of followup questions then:
Do .JAR files contain compiled code? Or just compressed source code files?
Is there a good reason why package names are all lower case?
Can Packages have 'circular dependencies'? In other words, can Package.A use Package.B and vice versa?
Can anyone just show the typical syntax for declaring a class as being in a package and declaring that you wish to reference another package in a class (a using statement maybe?)

"Simple" J2SE projects
As cletus explained, source directory structure is directly equivalent to package structure, and that's essentially built into Java. Everything else is a bit less clear-cut.
A lot of simple projects are organized by hand, so people get to pick a structure they feel OK with. What's often done (and this is also reflected by the structure of projects in Eclipse, a very dominant Java tool) is to have your source tree begin in a directory called src. Your package-less source files would sit directly in src, and your package hierarchy, typically starting with a com directory, would likewise be contained in src. If you CD to the src directory before firing up the javac compiler, your compiled .class files will end up in the same directory structure, with each .class file sitting in the same directory and next to its .java file.
If you have a lot of source and class files, you'll want to separate them out from each other to reduce clutter. Manual and Eclipse organization often place a bin or classes directory parallel to src so the .class files end up in a hierarchy that mirrors that of src.
If your project has a set of .jar files to deliver capability from third-party libraries, then a third directory, typically lib, is placed parallel to src and bin. Everything in lib needs to be put on the classpath for compilation and execution.
Finally, there's a bunch of this and that which is more or less optional:
docs in doc
resources in resources
data in data
configuration in conf...
You get the idea. The compiler doesn't care about these directories, they're just ways for you to organize (or confuse) yourself.
J2EE projects
J2EE is roughly equivalent to ASP.NET, it's a massive (standard) framework for organizing Web applications. While you can develop your code for J2EE projects any way you like, there is a firm standard for the structure that a Web container will expect your application delivered in. And that structure tends to reflect back a bit to the source layout as well.
Here is a page that details project structures for Java projects in general (they don't agree very much with what I wrote above) and for J2EE projects in particular:
http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html
Maven projects
Maven is a very versatile project build tool. Personally, my build needs are nicely met by ant, which roughly compares with nmake. Maven, on the other hand, is complete-lifecyle build management with dependency management bolted on. The libs and source for most of the code in the Java world is freely available in the 'net, and maven, if asked nicely, will go crawling it for you and bring home everything your project needs without you needing to even tell it to. It manages a little repository for you, too.
The downside to this highly industrious critter is the fact that it's highly fascist about project structure. You do it the Maven way or not at all. By forcing its standard down your throat, Maven manages to make projects worldwide a bit more similar in structure, easier to manage and easier to build automatically with a minimum of input.
Should you ever opt for Maven, you can stop worrying about project structure, because there can only be one. This is it: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html

A package in Java is very similar to a namespace in .Net. The name of the package essentially creates a path to the classes that live inside it. This path can be thought of as the class's namespace (in .Net terms) because it is the unique identifier for the specific class you want to use. For example if you have a package named:
org.myapp.myProject
And inside it you had a bunch of classes:
MyClass1
MyClass2
To specifically refer to those classes you would use:
org.myapp.myProject.MyClass1
org.myapp.myProject.MyClass2
The only real difference between this and .Net (that I know of) is that Java organizes its "namespaces" structurally (each package is a distinct folder) whereas .Net allows you to scope classes using the namespace keyword and ignores where the document actually lives.
A JAR file is roughly analogous to a DLL in most cases. It is a compressed file (you can open them with 7zip) that contains source code from other projects that can be added as dependencies in your application. Libraries are generally contained in JARs.
The thing to remember about Java is that is is very structural; WHERE files live is important. Of course there is more to the story then what I posted but I think this should get you started.

A package is much like a .Net namespace. The general convention in Java is to use your reversed domain name as a package prefix so if your company is example.com your packages will probably be:
com.example.projectname.etc...
It can be broken down to many levels rather than just one (projectname) but usually one is sufficient.
Inside your project structure classes are usually divided into logical areas: controllers, models, views, etc. It depends on the type of project.
There are two dominant build systems in Java: Ant and Maven.
Ant is basically a domain-specific scripting language and quite flexible but you end up writing a lot of boilerplate stuff yourself (build, deploy, test, etc tasks). It's quick and convenient though.
Maven is more modern and more complete and is worth using (imho). Maven is different to Ant in that Maven declares that this project is a "Web application project" (called an archetype). Once that is declared the directory structure is mandated once you specify your groupId (com.example) and artifactId (project name).
You get a lot of stuff for free this way. The real bonus of Maven is that it manages your project dependencies for you so with a pom.xml (Maven project file) and correctly configured Maven you can give that to someone else (with your source code) and they can build, deploy, test and run your project with libraries being downloaded automatically.
Ant gets something like this with Ivy.

Here are some notes about Java packages that should get you started:
The best practice with Java package names is to use the domain name of the organisation as the start of the package, but in reverse, e.g. if your company owns the domain "bobswidgets.com", you would start your package off with "com.bobswidgets".
The next level down will often be the application or library level, so if it's your ecommerce libraries, it could be something like "com.bobswidgets.ecommerce".
Further down than that often represents the architecture of your application. Classes and interfaces that are core to the project reside in the "root" e.g. com.bobswidgets.ecommerce.InvalidRequestException.
Using packages to subdivide functionality further is common. usually the pattern is to put interfaces and exceptions into whatever the root of the subdivision is and the implementation into sub packages e.g.
com.bobswidgets.ecommerce.payment.PaymentAuthoriser (interface)
com.bobswidgets.ecommerce.payment.PaymentException
com.bobswidgets.ecommerce.payment.paypal.PaypalPaymentAuthoriser (implementation)
This makes it pretty easy to pull the "payment" classes and packages into their own project.
Some other notes:
Java packages are tightly coupled to directory structure. So, within a project, a class with a Package of com.example.MyClass will invariably be in com/example/MyClass.java. This is because when it is packaged up into a Jar, the class file will definitely be in com/example/MyClass.class.
Java packages are loosely coupled to projects. It is quite common that projects will have their own distinct package names e.g. com.bobswidgets.ecommerce for ecommerce, com.bobswidgets.intranet for the intranet project.
Jar files will container the class files that are the result of compiling your .java code into bytecodes. They are just zip files with .jar extension. The root of the Jar file is the root of the namespace hierarchy e.g. com.bobswidgets.ecommerce will be /com/bobswidgets/ecommerce/ in the Jar file. Jar files can also container resources e.g. property files etc.

A package is a grouping of source files that lets them see each others' package-private methods and variables, so that that group of classes can access things in each other that other classes can't.
The expectation is that all java classes have a package that is used to disambiguate them. So if you open a jar file in your project, like spring, every package starts with org.springframework. The classloaders don't know about the jarfile name, they use only the package.
There's a common practice of breaking things down by type of object or function, not everybody agrees about this. Like Cletus posted here, there's a tendency to group web controllers, domain objects, services, and data access objects into their own packages. I think some Domain-Driven Design people do not think this is a good thing. It does have the advantage that typically everything in your package shares the same kind of dependencies (controllers might depend on services and domain objects, services depend on domain objects and data access objects, etc.) so that can be convenient.

Okay so in java you have three different types of access to a classes member functions and variables
public
protected
package-private
and private
All classes in the same package can see each others public, protected, and package-private elements.
Packages are not hierarchical in the system. Usually they are organized in a hierarchical way, but as far as runtime is concerned com.example.widgets is a completely different package from com.example.widgets.cogs
Packages are arranged as directories, which helps keep things organized: your file structure is always similar to your package structure.
They are planning on adding a module system to Java in JDK7 (called Project Jigsaw) and there is an existing module system called OSGi. These module systems will/can give you a lot more flexibility and power then the simple package system.
Also, package names are usually all lower case. :)

To answer the example sub-question:
package com.smotricz.goodfornaught;
import java.util.HashMap;
import javax.swing.*;
public class MyFrame extends JFrame {
private HashMap myMap = new HashMap();
public MyFrame() {
setTitle("My very own frame");
}
}

Do .JAR files contain compiled code? Or just compressed source code files?
They might contain both, or even totally different kinds of files like pictures. It's a zip archive first of all. Most often you would see JARs that contain class files, and those which contain source files (handy for debugging in your IDE if you use third party code) or those that contain javadoc (sourcecode documentatin), also handy if your IDE supports tooltipping the documentation when you access the lib's functions.
Is there a good reason why package names are all lower case?
Yes there is a good reason for package names to be written in lowercase letters: There is a guideline which says that only classnames are written with a capital letter in front.
Can Packages have 'circular dependencies'? In other words, can Package.A use Package.B and vice versa?
Packages do not use each other. Only classes do. And yes that might be possible but bad practice.
Can anyone just show the typical syntax for declaring a class as being in a package and declaring that you wish to reference another package in a class (a using statement maybe?)
Let's assume you want to use the ArrayList class from package java.util, either use
import java.util.ArrayList;
ArrayList myList = new ArrayList();
or use without import (say you use two different classes named ArrayList from different packages)
java.util.ArrayList myList = new java.util.ArrayList();
your.package.ArrayList mySecondList = new your.package.ArrayList();

From Wikipedia:
A Java package is a mechanism for
organizing Java classes into
namespaces
and
Java packages can be stored in
compressed files called JAR files
So for package a.b.c, you could have Java classes in the a, a.b, and a.b.c packages. Generally you group classes inside the same package when they represent related functionality. Functionally, the only difference between classes in the same package and classes in different package is that the default access level for members in Java is "package-protected", which means that other classes in the same package have access.
For a class a.b.c.MyClass, if you want to use MyClass in your project you would import a.b.c.MyClass or, less recommended, import a.b.c.* Also, for MyClass to reside in package a.b.c in the first place, you would declare it in the first line of MyClass.java: package a.b.c;.
To do this you could JAR up the whole package (including packages b and c and class MyClass) and put this JAR into your $CLASSPATH; this would make it accessible for your other source code to use (via the aforementioned import statement).

While it is not as easy to make circular dependent classes work, it may not be impossible. I did get it to work in one case. class A and class B depended on each other and wouldn't compile from scratch. but realizing that a part of class A didn't need class B, and that part was what class B needed to compile completely, I rem'd out that part of class A, not needed by class B, and the remaining part of class A was able to compile, then I was able to compile class B. I was then able to un-rem that section of class A that needed class B, and was able to compile the full class A. Both classes then functioned properly. While it is not typical, if the classes are tied together like this, it is kosher and at times possibly necessary. Just make sure you leave yourself special compile instructions for future updates.

Categories

Resources