Flyway: slow performance migrating a large number of scripts - java

We are using flywayDB for some years to manage a quite large database application running on oracle 12c. This works fine and very reliable.
But recently we are running into performance problems during database migration. The number of database skripts we manage in our versions-table has risen beyond 10,000. The time flyway takes to migrate a single script has increased from some originally milliseconds to currently about a second. It seems that flyway per single migration step selects the full content of the versions table to calculate at client side its installation rank. This doesnt scale well.
Is there any possiblity to speed up flyway, may be by caching the content of the versions table?
We are using the Java-API of flyway version 3.2.1.
For completeness I have written a testcase to demonstrate this behaviour.
#RunWith(Parameterized.class)
public class PerformanceTestcase {
private static Logger LOG = Logger.getLogger( PerformanceTestcase.class.getName() );
#Parameter
public int noOfScripts;
#Before
public void generateLotsOfInstallerSkripts() throws IOException {
LOG.log(Level.INFO, "generating {0} skripts", noOfScripts);
Path baseVersion = getBaseVersionPath();
generateSkripts( noOfScripts, baseVersion, BASE_SKRIPT_NAME );
}
#Test
public void testPerformance() throws IOException, SQLException {
// this one does not scale well with increasing noOfScripts
migrate();
}
private static final String SCHEMA_TABLE_NAME = "test_versions";
private static final String SKRIPT_NAME_FORMAT = "%s.%05d__test.sql";
private static final String SKRIPT_CONTENT = "select %05d from dual;";
private static final String FILESYSTEM = "filesystem:";
private static final String BASE_SKRIPT_NAME = "V00.00.00";
private static final String BASE_DIR = "/tmp/performanceTest";
private void migrate() throws SQLException {
Flyway flyway = new Flyway();
flyway.setDataSource( getDataSource() );
flyway.setLocations( FILESYSTEM + BASE_DIR );
flyway.setTable( SCHEMA_TABLE_NAME );
flyway.setBaselineVersionAsString(BASE_SKRIPT_NAME.substring(1) );
flyway.setBaselineOnMigrate(true);
flyway.setValidateOnMigrate(false);
flyway.migrate();
}
#Parameters(name="noOfScripts={0}")
public static Iterable<? extends Object> data() {
List<Integer> retval = new LinkedList<Integer>();
for ( int i=0; i<16000; i+=1000 ) {
if ( i>0 ) retval.add( Integer.valueOf(i) );
retval.add( Integer.valueOf(i+100) );
}
return retval;
}
private Path getBaseDirPath() throws IOException {
Path base = Paths.get(BASE_DIR);
if ( !Files.exists(base) ) {
Files.createDirectory(base);
}
return base;
}
private Path getBaseVersionPath() throws IOException {
Path base = getBaseDirPath();
Path baseVersion = base.resolve(BASE_SKRIPT_NAME);
if ( !Files.exists(baseVersion) ) {
Files.createDirectories(baseVersion);
}
return baseVersion;
}
private void generateSkripts( int numberOfSkripts, Path baseDir, String baseName ) throws IOException {
for (int i = 0; i < numberOfSkripts; i++) {
Path file = baseDir.resolve( String.format(SKRIPT_NAME_FORMAT, baseName, i) );
Files.write( file
, Arrays.asList( new String[] { String.format( SKRIPT_CONTENT, i ) } )
, StandardOpenOption.CREATE
, StandardOpenOption.TRUNCATE_EXISTING
);
}
}
private DataSource getDataSource() throws SQLException {
OracleDataSource ds = new OracleDataSource();
ds.setURL(CONNECTION_URL);
return ds;
}
}
Update
I just run the testcase with the current version 4.0.3 of flywayDB. It ran about half the time compared to 3.2.1, but the scaling problem still persists. Flyway is selecting the complete versions table per single migration step, which significantly slows down a migration when the version table is quite populated.
Update again
I've looked into the source-code of flywayDB version 4.0.3: In org.flywaydb.core.internal.command.DbMigrate#migrate the MigrationInfoServiceImpl is created and refreshed. This selects the full schema_versions-Table. But after that step only a single migration script is executed. I would expect migrating all pending scripts instead.
I've opened an issue on this one at github.

The issue is fixed. Today I tested flyway version 4.1.2 and the performance problem is gone. Thanks for the great work at boxfuse!

Related

Can we automatically refresh spring properties file without using actuator refresh endpoint [duplicate]

Many in-house solutions come to mind. Like having the properties in a database and poll it every N secs. Then also check the timestamp modification for a .properties file and reload it.
But I was looking in Java EE standards and spring boot docs and I can't seem to find some best way of doing it.
I need my application to read a properties file(or env. variables or DB parameters), then be able to re-read them. What is the best practice being used in production?
A correct answer will at least solve one scenario (Spring Boot or Java EE) and provide a conceptual clue on how to make it work on the other
After further research, reloading properties must be carefully considered. In Spring, for example, we can reload the 'current' values of properties without much problem. But. Special care must be taken when resources were initialized at the context initialization time based on the values that were present in the application.properties file (e.g. Datasources, connection pools, queues, etc.).
NOTE:
The abstract classes used for Spring and Java EE are not the best example of clean code. But it is easy to use and it does address this basic initial requirements:
No usage of external libraries other than Java 8 Classes.
Only one file to solve the problem (~160 lines for the Java EE version).
Usage of standard Java Properties UTF-8 encoded file available in the File System.
Support encrypted properties.
For Spring Boot
This code helps with hot-reloading application.properties file without the usage of a Spring Cloud Config server (which may be overkill for some use cases)
This abstract class you may just copy & paste (SO goodies :D ) It's a code derived from this SO answer
// imports from java/spring/lombok
public abstract class ReloadableProperties {
#Autowired
protected StandardEnvironment environment;
private long lastModTime = 0L;
private Path configPath = null;
private PropertySource<?> appConfigPropertySource = null;
#PostConstruct
private void stopIfProblemsCreatingContext() {
System.out.println("reloading");
MutablePropertySources propertySources = environment.getPropertySources();
Optional<PropertySource<?>> appConfigPsOp =
StreamSupport.stream(propertySources.spliterator(), false)
.filter(ps -> ps.getName().matches("^.*applicationConfig.*file:.*$"))
.findFirst();
if (!appConfigPsOp.isPresent()) {
// this will stop context initialization
// (i.e. kill the spring boot program before it initializes)
throw new RuntimeException("Unable to find property Source as file");
}
appConfigPropertySource = appConfigPsOp.get();
String filename = appConfigPropertySource.getName();
filename = filename
.replace("applicationConfig: [file:", "")
.replaceAll("\\]$", "");
configPath = Paths.get(filename);
}
#Scheduled(fixedRate=2000)
private void reload() throws IOException {
System.out.println("reloading...");
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
environment.getPropertySources()
.replace(
appConfigPropertySource.getName(),
new PropertiesPropertySource(
appConfigPropertySource.getName(),
properties
)
);
System.out.println("Reloaded.");
propertiesReloaded();
}
}
protected abstract void propertiesReloaded();
}
Then you make a bean class that allows retrieval of property values from applicatoin.properties that uses the abstract class
#Component
public class AppProperties extends ReloadableProperties {
public String dynamicProperty() {
return environment.getProperty("dynamic.prop");
}
public String anotherDynamicProperty() {
return environment.getProperty("another.dynamic.prop");
}
#Override
protected void propertiesReloaded() {
// do something after a change in property values was done
}
}
Make sure to add #EnableScheduling to your #SpringBootApplication
#SpringBootApplication
#EnableScheduling
public class MainApp {
public static void main(String[] args) {
SpringApplication.run(MainApp.class, args);
}
}
Now you can auto-wire the AppProperties Bean wherever you need it. Just make sure to always call the methods in it instead of saving it's value in a variable. And make sure to re-configure any resource or bean that was initialized with potentially different property values.
For now, I have only tested this with an external-and-default-found ./config/application.properties file.
For Java EE
I made a common Java SE abstract class to do the job.
You may copy & paste this:
// imports from java.* and javax.crypto.*
public abstract class ReloadableProperties {
private volatile Properties properties = null;
private volatile String propertiesPassword = null;
private volatile long lastModTimeOfFile = 0L;
private volatile long lastTimeChecked = 0L;
private volatile Path propertyFileAddress;
abstract protected void propertiesUpdated();
public class DynProp {
private final String propertyName;
public DynProp(String propertyName) {
this.propertyName = propertyName;
}
public String val() {
try {
return ReloadableProperties.this.getString(propertyName);
} catch (Exception e) {
e.printStackTrace();
throw new RuntimeException(e);
}
}
}
protected void init(Path path) {
this.propertyFileAddress = path;
initOrReloadIfNeeded();
}
private synchronized void initOrReloadIfNeeded() {
boolean firstTime = lastModTimeOfFile == 0L;
long currentTs = System.currentTimeMillis();
if ((lastTimeChecked + 3000) > currentTs)
return;
try {
File fa = propertyFileAddress.toFile();
long currModTime = fa.lastModified();
if (currModTime > lastModTimeOfFile) {
lastModTimeOfFile = currModTime;
InputStreamReader isr = new InputStreamReader(new FileInputStream(fa), StandardCharsets.UTF_8);
Properties prop = new Properties();
prop.load(isr);
properties = prop;
isr.close();
File passwordFiles = new File(fa.getAbsolutePath() + ".key");
if (passwordFiles.exists()) {
byte[] bytes = Files.readAllBytes(passwordFiles.toPath());
propertiesPassword = new String(bytes,StandardCharsets.US_ASCII);
propertiesPassword = propertiesPassword.trim();
propertiesPassword = propertiesPassword.replaceAll("(\\r|\\n)", "");
}
}
updateProperties();
if (!firstTime)
propertiesUpdated();
} catch (Exception e) {
e.printStackTrace();
}
}
private void updateProperties() {
List<DynProp> dynProps = Arrays.asList(this.getClass().getDeclaredFields())
.stream()
.filter(f -> f.getType().isAssignableFrom(DynProp.class))
.map(f-> fromField(f))
.collect(Collectors.toList());
for (DynProp dp :dynProps) {
if (!properties.containsKey(dp.propertyName)) {
System.out.println("propertyName: "+ dp.propertyName + " does not exist in property file");
}
}
for (Object key : properties.keySet()) {
if (!dynProps.stream().anyMatch(dp->dp.propertyName.equals(key.toString()))) {
System.out.println("property in file is not used in application: "+ key);
}
}
}
private DynProp fromField(Field f) {
try {
return (DynProp) f.get(this);
} catch (IllegalAccessException e) {
e.printStackTrace();
}
return null;
}
protected String getString(String param) throws Exception {
initOrReloadIfNeeded();
String value = properties.getProperty(param);
if (value.startsWith("ENC(")) {
String cipheredText = value
.replace("ENC(", "")
.replaceAll("\\)$", "");
value = decrypt(cipheredText, propertiesPassword);
}
return value;
}
public static String encrypt(String plainText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
SecureRandom secureRandom = new SecureRandom();
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
byte[] iv = new byte[12];
secureRandom.nextBytes(iv);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
GCMParameterSpec parameterSpec = new GCMParameterSpec(128, iv); //128 bit auth tag length
cipher.init(Cipher.ENCRYPT_MODE, secretKey, parameterSpec);
byte[] cipherText = cipher.doFinal(plainText.getBytes(StandardCharsets.UTF_8));
ByteBuffer byteBuffer = ByteBuffer.allocate(4 + iv.length + cipherText.length);
byteBuffer.putInt(iv.length);
byteBuffer.put(iv);
byteBuffer.put(cipherText);
byte[] cipherMessage = byteBuffer.array();
String cyphertext = Base64.getEncoder().encodeToString(cipherMessage);
return cyphertext;
}
public static String decrypt(String cypherText, String key)
throws NoSuchPaddingException, NoSuchAlgorithmException, InvalidAlgorithmParameterException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException, InvalidKeySpecException {
byte[] cipherMessage = Base64.getDecoder().decode(cypherText);
ByteBuffer byteBuffer = ByteBuffer.wrap(cipherMessage);
int ivLength = byteBuffer.getInt();
if(ivLength < 12 || ivLength >= 16) { // check input parameter
throw new IllegalArgumentException("invalid iv length");
}
byte[] iv = new byte[ivLength];
byteBuffer.get(iv);
byte[] cipherText = new byte[byteBuffer.remaining()];
byteBuffer.get(cipherText);
byte[] keyBytes = key.getBytes(StandardCharsets.US_ASCII);
final Cipher cipher = Cipher.getInstance("AES/GCM/NoPadding");
SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256");
KeySpec spec = new PBEKeySpec(key.toCharArray(), new byte[]{0,1,2,3,4,5,6,7}, 65536, 128);
SecretKey tmp = factory.generateSecret(spec);
SecretKey secretKey = new SecretKeySpec(tmp.getEncoded(), "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKey, new GCMParameterSpec(128, iv));
byte[] plainText= cipher.doFinal(cipherText);
String plain = new String(plainText, StandardCharsets.UTF_8);
return plain;
}
}
Then you can use it this way:
public class AppProperties extends ReloadableProperties {
public static final AppProperties INSTANCE; static {
INSTANCE = new AppProperties();
INSTANCE.init(Paths.get("application.properties"));
}
#Override
protected void propertiesUpdated() {
// run code every time a property is updated
}
public final DynProp wsUrl = new DynProp("ws.url");
public final DynProp hiddenText = new DynProp("hidden.text");
}
In case you want to use encoded properties you may enclose it's value inside ENC() and a password for decryption will be searched for in the same path and name of the property file with an added .key extension. In this example it will look for the password in the application.properties.key file.
application.properties ->
ws.url=http://some webside
hidden.text=ENC(AAAADCzaasd9g61MI4l5sbCXrFNaQfQrgkxygNmFa3UuB9Y+YzRuBGYj+A==)
aplication.properties.key ->
password aca
For the encryption of property values for the Java EE solution I consulted Patrick Favre-Bulle excellent article on Symmetric Encryption with AES in Java and Android. Then checked the Cipher, block mode and padding in this SO question about AES/GCM/NoPadding. And finally I made the AES bits be derived from a password from #erickson excellent answer in SO about AES Password Based Encryption. Regarding encryption of value properties in Spring I think they are integrated with Java Simplified Encryption
Wether this qualify as a best practice or not may be out of scope. This answer shows how to have reloadable properties in Spring Boot and Java EE.
This functionality can be achieved by using a Spring Cloud Config Server and a refresh scope client.
Server
Server (Spring Boot app) serves the configuration stored, for example, in a Git repository:
#SpringBootApplication
#EnableConfigServer
public class ConfigServer {
public static void main(String[] args) {
SpringApplication.run(ConfigServer.class, args);
}
}
application.yml:
spring:
cloud:
config:
server:
git:
uri: git-repository-url-which-stores-configuration.git
configuration file configuration-client.properties (in a Git repository):
configuration.value=Old
Client
Client (Spring Boot app) reads configuration from the configuration server by using #RefreshScope annotation:
#Component
#RefreshScope
public class Foo {
#Value("${configuration.value}")
private String value;
....
}
bootstrap.yml:
spring:
application:
name: configuration-client
cloud:
config:
uri: configuration-server-url
When there is a configuration change in the Git repository:
configuration.value=New
reload the configuration variable by sending a POST request to the /refresh endpoint:
$ curl -X POST http://client-url/actuator/refresh
Now you have the new value New.
Additionally Foo class can serve the value to the rest of application via RESTful API if its changed to RestController and has a corresponding endpont.
I used #David Hofmann concept and made some changes because of not all was good.
First of all, in my case I no need auto-reload, I just call the REST controller for updating properties.
The second case #David Hofmann's approach not workable for me with outside files.
Now, this code can work with application.properties file from resources(inside the app) and from an outside place. The outside file I put near jar, and I use this --spring.config.location=app.properties argument when the application starts.
#Component
public class PropertyReloader {
private final Logger logger = LoggerFactory.getLogger(getClass());
#Autowired
private StandardEnvironment environment;
private long lastModTime = 0L;
private PropertySource<?> appConfigPropertySource = null;
private Path configPath;
private static final String PROPERTY_NAME = "app.properties";
#PostConstruct
private void createContext() {
MutablePropertySources propertySources = environment.getPropertySources();
// first of all we check if application started with external file
String property = "applicationConfig: [file:" + PROPERTY_NAME + "]";
PropertySource<?> appConfigPsOp = propertySources.get(property);
configPath = Paths.get(PROPERTY_NAME).toAbsolutePath();
if (appConfigPsOp == null) {
// if not we check properties file from resources folder
property = "class path resource [" + PROPERTY_NAME + "]";
configPath = Paths.get("src/main/resources/" + PROPERTY_NAME).toAbsolutePath();
}
appConfigPsOp = propertySources.get(property);
appConfigPropertySource = appConfigPsOp;
}
// this method I call into REST cintroller for reloading all properties after change
// app.properties file
public void reload() {
try {
long currentModTs = Files.getLastModifiedTime(configPath).toMillis();
if (currentModTs > lastModTime) {
lastModTime = currentModTs;
Properties properties = new Properties();
#Cleanup InputStream inputStream = Files.newInputStream(configPath);
properties.load(inputStream);
String property = appConfigPropertySource.getName();
PropertiesPropertySource updatedProperty = new PropertiesPropertySource(property, properties);
environment.getPropertySources().replace(property, updatedProperty);
logger.info("Configs {} were reloaded", property);
}
} catch (Exception e) {
logger.error("Can't reload config file " + e);
}
}
}
I hope that my approach will help somebody
As mentioned by #Boris, Spring Cloud Config is the way to go to avoid patchy solution. To keep the setup minimum, I will suggest the Embedding the Config Server Approach with native type (file type).
To support automatic config refresh without calling the actuator endpoint manually, I have created a directory listener to detect file changes and to dispatch refresh scope event.
Proof Of Concept repo (git)
For spring boot, there's a really good article on this topic here, but for multiple property files it doesn't work perfectly.
In my case I had 2 property files, one non sensitive and one containing the passwords. I proceeded with the following:
<dependency>
<groupId>commons-configuration</groupId>
<artifactId>commons-configuration</artifactId>
<version>1.10</version>
</dependency>
Extend the spring's PropertySource so that you can add the reloadable version to the environment.
public class ReloadablePropertySource extends PropertySource {
private final PropertiesConfiguration propertiesConfiguration;
public ReloadablePropertySource(String name, String path, ConfigurationListener listener) {
super(StringUtils.hasText(name) ? name : path);
try {
this.propertiesConfiguration = getConfiguration(path, listener);
} catch (Exception e) {
throw new MissingRequiredPropertiesException();
}
}
#Override
public Object getProperty(String s) {
return propertiesConfiguration.getProperty(s);
}
private PropertiesConfiguration getConfiguration(String path, ConfigurationListener listener) throws ConfigurationException {
PropertiesConfiguration configuration = new PropertiesConfiguration(path);
FileChangedReloadingStrategy reloadingStrategy = new FileChangedReloadingStrategy();
reloadingStrategy.setRefreshDelay(5000);
configuration.setReloadingStrategy(reloadingStrategy);
configuration.addConfigurationListener(listener);
return configuration;
}
}
Now add all of your properties files (now reloadable) inside the spring's env
#Configuration
public class ReloadablePropertySourceConfig {
private final ConfigurableEnvironment env;
#Value("${spring.config.location}")
private String appConfigPath;
#Value("${spring.config.additional-location}")
private String vaultConfigPath;
public ReloadablePropertySourceConfig(ConfigurableEnvironment env) {
this.env = env;
}
#Bean
#ConditionalOnProperty(name = "spring.config.location")
public ReloadablePropertySource getAppConfigReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicNonSensitive", appConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
#Bean
#ConditionalOnProperty(name = "spring.config.additional-location")
public ReloadablePropertySource getVaultReloadablePropertySource(){
ReloadablePropertySource rps = new ReloadablePropertySource("dynamicVault", vaultConfigPath, new PropertiesChangeListener());
MutablePropertySources sources = env.getPropertySources();
sources.addFirst(rps);
return rps;
}
private static class PropertiesChangeListener implements ConfigurationListener{
#Override
public void configurationChanged(ConfigurationEvent event) {
if (!event.isBeforeUpdate()){
System.out.println("config refreshed!");
}
}
}
}
From the article
We've added the new property source as the first item because we want it to override any existing property with the same key
In our case, we have 2 "reloadable" property sources and both will be looked up first.
Finally create one more class from which we can access the env's properties
#Component
public class ConfigProperties {
private final Environment environment;
public ConfigProperties(Environment environment) {
this.environment = environment;
}
public String getProperty(String name){
return environment.getProperty(name);
}
}
Now you can autowire ConfigProperties and always get the latest property in the files without requiring to restart the application.
#RestController
#Slf4j
public class TestController {
#Autowired
private ConfigProperties env;
#GetMapping("/refresh")
public String test2() {
log.info("hit");
String updatedProperty = env.getProperty("test.property");
String password = env.getProperty("db.password");
return updatedProperty + "\n" + password;
}
}
where test.property is coming from 1st file and db.password is coming from another.
If you want to change the properties at realtime and don't want to restart the server then follow the below steps:
1). Application.properties
app.name= xyz
management.endpoints.web.exposure.include=*
2). Add below dependencies in pom.xml
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-rest</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-context</artifactId>
<version>2.1.1.RELEASE</version>
</dependency>
3).Place application.properties in /target/config folder. Create the jar in /target folder
4).add a classas below ApplcationProperties.java
#Component
#RefreshScope
#ConfigurationProperties(prefix = "app")
public class ApplicationProperties {
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
5). Write Controller.java and inject ApplcationProperties
#RestController
public class TestController {
#Autowired
private ApplicationProperties applcationProperties;
#GetMapping("/test")
public String getString() {
return applcationProperties.getName();
}
}
6).Run the spring boot application
Call localhost:XXXX/test from your browser
Output : xyz
7). Change the value in application.properties from xyz to abc
8). Using postman send a POST request to localhost:XXXX/actuator/refresh
response: ["app.name"]
9). Call localhost:XXXX/find from your browser
Output : abc

How to get #tag from an excel file to CucumberOptions in Selenium java

Normally I would write a cucumber option as below:
#CucumberOptions(
features = "src\\main\\java\\feature"
, glue= "stepdefination",
plugin= {"com.cucumber.listener.ExtentCucumberFormatter:Report/Report.html"}
tags="#tag, #tag1, #sort"
)
public class TestRunner extends TestFunction {
#Test
public void runcukes( ) {
new TestNGCucumberRunner(getClass()).runCukes();
}
#BeforeClass
public void tags() {
}
#AfterClass
public void writeExtentReport() {
Reporter.loadXMLConfig("extent-config.xml");
}
}
My question is: How can I fetch #tag, #tag1, #sort from an excel file to #cucmberoptions and run the program in Selenium Java?
I am not sure about using cucumber options but by using cucumber RuntimeOptions class you can achieve it. The below method needs to be called in a loop that means you have 10 tags to execute then call this method in for loop.
public static boolean cucumberRun(String tag) {
try {
ClassLoader classLoader = CustomClass.class.getClassLoader();
ResourceLoader resourceLoader = new MultiLoader(classLoader);
ClassFinder classFinder = new ResourceLoaderClassFinder(resourceLoader, classLoader);
/* Adding cucumber plugins */
List<String> pluginList = new ArrayList<String>();
pluginList.add("--plugin");
pluginList.add("html:target/cucumber-html-report");
pluginList.add("--plugin");
pluginList.add("json:target/cucumber.json");
pluginList.add("--plugin");
pluginList.add("com.cucumber.listener.ExtentCucumberFormatter:");
pluginList.add("--plugin");
pluginList.add("rerun:target/failedScenarios.txt");
/* Location for BDD extent report. */
ExtentProperties extentProperties = ExtentProperties.INSTANCE;
extentProperties.setReportPath("location/Extent_report.html");
/*
* Adding cucumberOptions.
*
* You can get the tag value from excel and pass it here. We need to add # before tag value.
*/
RuntimeOptions ro = new RuntimeOptions(pluginList);
ro.getFilters().add("#"+tag);
ro.getFeaturePaths().add("location of feature files");
ro.getGlue().add("location of glue code");
/* Loads all the resources into Cucumber RunTime class and execute. */
cucumber.runtime.Runtime runtime = new cucumber.runtime.Runtime(resourceLoader, classFinder, classLoader,
ro);
runtime.run();
}catch(Exception e){
// Handle exception
}

Creating .accdb database using Java

I am trying to automatically create an .accdb database but I get a compile error using the create() function:
I use the code from this SO answer:
Create an Access database file (.mdb or .accdb) using Java
public class JackcessLibrary {
private static Database createDatabase(String databaseName) throws IOException {
return Database.create(new File(databaseName));
}
private static TableBuilder createTable(String tableName) {
return new TableBuilder(tableName);
}
public static void addColumn(Database database, TableBuilder tableName, String columnName, Types sqlType) throws SQLException, IOException {
tableName.addColumn(new ColumnBuilder(columnName).setSQLType(Types.INTEGER).toColumn()).toTable(database);
}
public static void startDatabaseProcess() throws IOException, SQLException {
String databaseName = "C:/Users/abdulwhab/Desktop/database/db.accdb"; // Creating an MS Access database
Database database = createDatabase(databaseName);
String tableName = "Employee"; // Creating table
Table table = createTable(tableName)
.addColumn(new ColumnBuilder("Emp_Id").setSQLType(Types.INTEGER).toColumn())
.addColumn(new ColumnBuilder("Emp_Name").setSQLType(Types.VARCHAR).toColumn())
.addColumn(new ColumnBuilder("Emp_Employer").setSQLType(Types.VARCHAR).toColumn())
.toTable(database);
table.addRow(122875, "Sarath Kumar Sivan","Infosys Limited.");//Inserting values into the table
}
public static void main(String[] args) throws IOException, SQLException {
JackcessLibrary.startDatabaseProcess();
}
}
You use the brand new version 2.1.3 of Jackcess, wheras the five-year old answer you linked uses 1.2.6. The API of Jackcess underwent several changes when introducing version 2.
In the version you use a database is created by using a builder:
DatabaseBuilder.create(FileFormat, File)
For more on how to use the Jackcess API, see http://jackcess.sourceforge.net/cookbook.html.
for others clarity and easiness in future just do following changes in the Database creating function
File file = new File("C:/Users/abdulwhab/Desktop/database/test.accdb");
Database db = new DatabaseBuilder(file).setFileFormat(Database.FileFormat.V2000).create();
return db;
Note: i didn't chnage the jackess jar file version to previous one.
No chnages were made except the code :)

java.lang.IllegalAccessError: tried to access method net.sourceforge.tess4j.Tesseract.<init>()V from class Tess4jTest.TestTess

I did a Java OCR project with Tesseract in the Mirth.When I run the jar file from the Mirth,I get this error.When I search it,I found that there is a init() method and also it is a protected void in Tesseract.java.I think that maybe it is the reason for that error.
What should I do?Thank you so much for your helps.
package Tess4jTest;
import java.io.File;
import java.io.IOException;
import net.sourceforge.tess4j.*;
public class TestTess {
public static String Tc;
public static String phone;
public static String date;
public static void main(String[] args) {
//System.out.println(returnText("C:\\Users\\Nevzat\\Desktop\\deneme.pdf"));
}
public static String returnText(String fileName){
File imageFile = new File(fileName);
if(imageFile.exists()){
Tesseract instance = new Tesseract();
instance.setDatapath("C:\\imageRAD\\Onam\\tessdata");
String result = null;
try {
result = instance.doOCR(imageFile);
} catch (TesseractException e) {
System.err.println(e.getMessage());
}
if(result!=null){
int i=result.indexOf("Numarasn: ");
int j=result.indexOf("Tel No:");
int k=result.indexOf("Bilgllendirme Tarihl:");
Tc = result.substring(i+10, i+21);
phone = result.substring(j+8,j+23);
date = result.substring(k+22,k+32);
//System.out.println(result);
}else{
return "Null Error!";
}
}else{
return "Does not found a file!";
}
return Tc+","+phone+","+date;
}
public static String returnTC() throws IOException{
return Tc;
}
public static String returnPhone() throws IOException{
return phone;
}
public static String returnDate() throws IOException{
return date;
}
}
The error you got occurs when you try to create an object with a private constructor. (<init>() is the name of a constructor with no parameters)
Looking at the tess4j source I found a method with the following documentation:
#deprecated As of Release 2.0, use default constructor instead.
Looking at the source before 2.0 reveals the default constructor was private.
This means your problem is most likely that you are compiling against a version newer than 2.0, but your environment is running one older than 2.0.
Either update your environment or downgrade the library you build against to fix it.
I solved the error and have finished the project.I mention step by step
1.You have to use right jar files for tess4j.
2.Add java project all of in the tess4j-3.2.1.zip except tess4j-3.2.1.jar via Build Path.
3.Add tess4j-1.5.jar from this
4.Add tessdata folder,ghost4j-0.5.1.jar,jna-4.1.jar,tess4j.jar and jar file of your java project.

Running Wicket the first time

I am having issues with the Wicket GAE sample application,
The issues is that some import cannot be resolved:
import org.apache.wicket.page.PersistentPageManager; // This one
import org.apache.wicket.util.io.IObjectStreamFactory; // and this cannot be resolved
#Override
public void init()
{
super.init();
getResourceSettings().setResourcePollFrequency(null);
WicketObjects.setObjectStreamFactory(new IObjectStreamFactory() {
#Override
public ObjectInputStream newObjectInputStream(InputStream in)
throws IOException {
return new ObjectInputStream(in);
}
#Override
public ObjectOutputStream newObjectOutputStream(OutputStream out)
throws IOException {
return new ObjectOutputStream(out);
}
});
setPageManagerProvider(new DefaultPageManagerProvider(this) {
public IPageManager get(IPageManagerContext pageManagerContext)
{
IDataStore dataStore = new HttpSessionDataStore(pageManagerContext, new PageNumberEvictionStrategy(10));
IPageStore pageStore = new DefaultPageStore(getName(), dataStore,
getCacheSize());
return new PersistentPageManager(getName(), pageStore, pageManagerContext);
}
});
// add your configuration here
}
Also, the getCacheSize() method cannot be found, I am using Wicket version, 1.5-SNAPSHOT as used in the demo app, but I tried other later versions too but still cannot be resolved, how can this be fixed?
This classes and methods were introduced only in RC versions (or earlier release versions) and removed from 1.5 release
So, you can download one of the RC packages or migrate to 1.5 version.
To migrate, you should:
Remove code WicketObjects.setObjectStreamFactory.... This will be replaced by defining serializer for pageStore. See this migration section.
Override DefaultPageManagerProvider's get method as follows:
#Override
public IPageManager get(IPageManagerContext context)
{
IDataStore dataStore = new HttpSessionDataStore(context,
new PageNumberEvictionStrategy(10));
IPageStore pageStore = new DefaultPageStore (
new JavaSerializer ( getName() ), dataStore,
getStoreSettings().getInmemoryCacheSize());
return new PageStoreManager(application.getName(), pageStore,
context);
}
For further migrations you should check wicket migration guides.

Categories

Resources