Is page cache a part of Java Off heap memory? - java

JVM has on heap and off heap memory.
If a java proc write a file to the disk(not in Direct_IO mode), the os will cache the file in memory as page cache, I'm curious if the page cache is a part of JVM offheap memory?
The background is I'm running a scala Spark job in K8S . It get OOM killed frequently even if I set the memory overhead to 0.2. According to the Spark UI, the peak on-heap mem + off-heap mem is much smaller than the container memory limit. But according to dmesg the java proc take all the memory of the container at the time it was killed.
So I'm wondering if it's page cache who takes the rest of memory and cause the java proc being killed?
According to the SparkUI, the dead executor1 on take 21.1 GiB (on heap) + 142.2 MiB (off heap) memory.
spark ui
But according to dmesg, at the time it was killed, it takes much more memory as SparkUI said.
[Mon Feb 20 03:36:35 2023] memory: usage 25907200kB, limit 25907200kB, failcnt 1911352
[Mon Feb 20 03:36:35 2023] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0
[Mon Feb 20 03:36:35 2023] kmem: usage 106724kB, limit 9007199254740988kB, failcnt 0
[Mon Feb 20 03:36:35 2023] Memory cgroup stats for /kubepods/pod320ce7f2-aa63-4f3f-9fbe-2ae0be48736c:
[Mon Feb 20 03:36:35 2023] anon 25102856192
file 1315708928
kernel_stack 1069056
slab 52314112
sock 0
shmem 0
file_mapped 0
file_dirty 1316130816
file_writeback 0
anon_thp 20558381056
inactive_anon 0
active_anon 25102508032
inactive_file 328622080
active_file 987414528
unevictable 0
slab_reclaimable 43982848
slab_unreclaimable 8331264
pgfault 11556633
pgmajfault 0
workingset_refault 814704
workingset_activate 165
workingset_nodereclaim 0
pgrefill 12258140
pgscan 15669889
pgsteal 2556626
pgactivate 13111197
pgdeactivate 12258004
pglazyfree 0
pglazyfreed 0
thp_fault_alloc 11253
thp_collapse_alloc 0
[Mon Feb 20 03:36:35 2023] Tasks state (memory values in pages):
[Mon Feb 20 03:36:35 2023] [ pid ] uid tgid total_vm rss pgtables_bytes swapents oom_score_adj name
[Mon Feb 20 03:36:35 2023] [ 168097] 65535 168097 241 1 28672 0 -998 pause
[Mon Feb 20 03:36:35 2023] [ 168142] 0 168142 571 192 49152 0 -997 tini
[Mon Feb 20 03:36:35 2023] [ 168217] 0 168217 7542445 6135546 50069504 0 -997 java
Call Trace
[Mon Feb 20 03:36:55 2023] Call Trace:
[Mon Feb 20 03:36:55 2023] dump_stack+0x57/0x6d
[Mon Feb 20 03:36:55 2023] dump_header+0x4f/0x200
[Mon Feb 20 03:36:55 2023] oom_kill_process+0xec/0x140
[Mon Feb 20 03:36:55 2023] out_of_memory+0x117/0x570
[Mon Feb 20 03:36:55 2023] mem_cgroup_out_of_memory+0xbb/0xd0
[Mon Feb 20 03:36:55 2023] try_charge+0x762/0x7c0
[Mon Feb 20 03:36:55 2023] ? ext4_do_update_inode+0x4ff/0x7b0
[Mon Feb 20 03:36:55 2023] mem_cgroup_try_charge+0x75/0x190
[Mon Feb 20 03:36:55 2023] __add_to_page_cache_locked+0x21a/0x3d0
[Mon Feb 20 03:36:55 2023] ? scan_shadow_nodes+0x30/0x30
[Mon Feb 20 03:36:55 2023] add_to_page_cache_lru+0x4f/0xd0
[Mon Feb 20 03:36:55 2023] pagecache_get_page+0xea/0x2c0
[Mon Feb 20 03:36:55 2023] grab_cache_page_write_begin+0x23/0x40
[Mon Feb 20 03:36:55 2023] ext4_da_write_begin+0x11a/0x450
[Mon Feb 20 03:36:55 2023] generic_perform_write+0xb3/0x1b0
[Mon Feb 20 03:36:55 2023] __generic_file_write_iter+0x1aa/0x1d0
[Mon Feb 20 03:36:55 2023] ? generic_write_checks+0x68/0xc0
[Mon Feb 20 03:36:55 2023] ext4_file_write_iter+0xbd/0x360
[Mon Feb 20 03:36:55 2023] ? futex_wake+0x8f/0x180
[Mon Feb 20 03:36:55 2023] new_sync_write+0x125/0x1c0
[Mon Feb 20 03:36:55 2023] __vfs_write+0x29/0x40
[Mon Feb 20 03:36:55 2023] vfs_write+0xb1/0x1a0
[Mon Feb 20 03:36:55 2023] ksys_write+0xa7/0xe0
[Mon Feb 20 03:36:55 2023] __x64_sys_write+0x1a/0x20
[Mon Feb 20 03:36:55 2023] do_syscall_64+0x5e/0x200
[Mon Feb 20 03:36:55 2023] entry_SYSCALL_64_after_hwframe+0x44/0xa9

Related

GraalVM Library is not linking in linux mint

I have built a native image application using (GraalVM 22.1.0) in Mac OS environment and it worked, but when trying to build the same application on linux the gluonfx:links fails, with an error of
undefined reference to Java_com_sun_security_auth_module_UnixSystem_getUnixInfo
undefined reference to JNI_OnLoad_jaas
I thought the problem was the libjaas.a not linking as it was not presented in the log file, so i added <arg>-ljaas</arg> but gave me another error which was /usr/bin/ld: cannot find -ljaas
Here is the POM file:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<junit.version>5.8.1</junit.version>
</properties>
<dependencies>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-controls</artifactId>
<version>17.0.1</version>
</dependency>
<dependency>
<groupId>org.openjfx</groupId>
<artifactId>javafx-fxml</artifactId>
<version>17.0.1</version>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-api</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>${junit.version}</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>2.1.214</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.8.1</version>
<configuration>
<source>17</source>
<target>17</target>
</configuration>
</plugin>
<plugin>
<groupId>com.gluonhq</groupId>
<artifactId>gluonfx-maven-plugin</artifactId>
<version>1.0.15</version>
<configuration>
<mainClass>com.example.demo4.Driver</mainClass>
<linkerArgs>
<arg>-ljaas</arg>
</linkerArgs>
</configuration>
</plugin>
<plugin>
<groupId>org.openjfx</groupId>
<artifactId>javafx-maven-plugin</artifactId>
<version>0.0.8</version>
<executions>
<execution>
<!-- Default configuration for running with: mvn clean javafx:run -->
<id>default-cli</id>
<configuration>
<mainClass>com.example.demo4/com.example.demo4.HelloApplication</mainClass>
<launcher>app</launcher>
<jlinkZipName>app</jlinkZipName>
<jlinkImageName>app</jlinkImageName>
<noManPages>true</noManPages>
<stripDebug>true</stripDebug>
<noHeaderFiles>true</noHeaderFiles>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Main Class:
package com.example.demo4;
import javafx.application.Application;
import javafx.scene.Scene;
import javafx.scene.control.Label;
import javafx.stage.Stage;
import java.sql.*;
public class Driver extends Application {
#Override
public void start(Stage stage) throws Exception {
Class.forName("org.h2.Driver");
Connection connection =DriverManager.getConnection("","","");
String str = connection.toString();
connection.close();
Scene scene = new Scene(new Label(str),500,500);
stage.setScene(scene);
stage.show();
}
public static void main(String[] args) {
launch(args);
}
}
Error gluonfx:link
[Mon Sep 05 11:37:26 EEST 2022][INFO] We will now compile your code for x86_64-linux-linux. This may take some time.
[Mon Sep 05 11:37:27 EEST 2022][INFO] [SUB] Warning: Ignoring server-mode native-image argument --no-server.
[Mon Sep 05 11:37:29 EEST 2022][INFO] [SUB] ========================================================================================================================
[Mon Sep 05 11:37:29 EEST 2022][INFO] [SUB] GraalVM Native Image: Generating 'com.example.demo4.driver' (shared library)...
[Mon Sep 05 11:37:29 EEST 2022][INFO] [SUB] ========================================================================================================================
[Mon Sep 05 11:37:29 EEST 2022][INFO] [SUB] [GluonFeature] enabled for config com.oracle.svm.hosted.FeatureImpl$IsInConfigurationAccessImpl#7cc586a8
[Mon Sep 05 11:37:30 EEST 2022][INFO] [SUB] GluonFeature enabled in setup com.oracle.svm.hosted.FeatureImpl$DuringSetupAccessImpl#65f58c6e
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] [1/7] Initializing... (3.7s # 0.20GB)
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] Version info: 'GraalVM 22.1.0 Java 17 CE'
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] C compiler: gcc (linux, x86_64, 9.3.0)
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] Garbage collector: Serial GC
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] 1 user-provided feature(s)
[Mon Sep 05 11:37:31 EEST 2022][INFO] [SUB] - com.gluonhq.substrate.feature.GluonFeature
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] [2/7] Performing analysis... [***********] (45.6s # 3.08GB)
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] 13,299 (91.34%) of 14,560 classes reachable
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] 25,151 (67.63%) of 37,189 fields reachable
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] 70,391 (66.36%) of 106,070 methods reachable
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] 663 classes, 185 fields, and 2,147 methods registered for reflection
[Mon Sep 05 11:38:17 EEST 2022][INFO] [SUB] 114 classes, 144 fields, and 182 methods registered for JNI access
[Mon Sep 05 11:38:20 EEST 2022][INFO] [SUB] [3/7] Building universe... (3.2s # 2.60GB)
[Mon Sep 05 11:38:23 EEST 2022][INFO] [SUB] [4/7] Parsing methods... [**] (3.0s # 3.00GB)
[Mon Sep 05 11:38:30 EEST 2022][INFO] [SUB] [5/7] Inlining methods... [****] (6.2s # 2.27GB)
[Mon Sep 05 11:39:03 EEST 2022][INFO] [SUB] [6/7] Compiling methods... [******] (32.8s # 3.51GB)
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB]
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] ------------------------------------------------------------------------------------------------------------------------
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] 10.0s (9.6% of total time) in 59 GCs | Peak RSS: 5.27GB | CPU load: 6.42
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] ------------------------------------------------------------------------------------------------------------------------
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] Produced artifacts:
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/graal_isolate.h (header)
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/com.example.demo4.driver.h (header)
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/graal_isolate_dynamic.h (header)
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/com.example.demo4.driver_dynamic.h (header)
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/com.example.demo4.driver.build_artifacts.txt
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] ========================================================================================================================
[Mon Sep 05 11:39:11 EEST 2022][INFO] [SUB] Finished generating 'com.example.demo4.driver' in 1m 43s.
[INFO]
[INFO] --- gluonfx-maven-plugin:1.0.15:link (default-cli) # demo4 ---
[Mon Sep 05 11:39:12 EEST 2022][INFO] Substrate is tested with the Gluon's GraalVM build which you can find at https://github.com/gluonhq/graal/releases.
While you can still use other GraalVM builds, there is no guarantee that these will work properly with Substrate
[Mon Sep 05 11:39:12 EEST 2022][INFO] ==================== LINK TASK ====================
[Mon Sep 05 11:39:13 EEST 2022][INFO] [SUB] /usr/bin/ld: /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/tmp/SVM-1662367049997/com.example.demo4.driver.o:(.data+0x6d8): undefined reference to `JNI_OnLoad_jaas'
[Mon Sep 05 11:39:13 EEST 2022][INFO] [SUB] /usr/bin/ld: /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/tmp/SVM-1662367049997/com.example.demo4.driver.o:(.data+0x1050): undefined reference to `Java_com_sun_security_auth_module_UnixSystem_getUnixInfo'
[Mon Sep 05 11:39:13 EEST 2022][INFO] [SUB] collect2: error: ld returned 1 exit status
[Mon Sep 05 11:39:13 EEST 2022][SEVERE] Process link failed with result: 1
Check the log files under /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/log
And please check https://docs.gluonhq.com/ for more information.
[Mon Sep 05 11:39:13 EEST 2022][INFO] Logging process [link] to file: /home/yousef66/Desktop/demo4/target/gluonfx/log/process-link-1662367153279.log
[Mon Sep 05 11:39:13 EEST 2022][SEVERE] Linking failed.
Check the log files under /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/log
And please check https://docs.gluonhq.com/ for more information.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:49 min
[INFO] Finished at: 2022-09-05T11:39:13+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.gluonhq:gluonfx-maven-plugin:1.0.15:link (default-cli) on project demo4: Linking failed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 01:50 min
[INFO] Finished at: 2022-09-05T11:39:13+03:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal com.gluonhq:gluonfx-maven-plugin:1.0.15:build (default-cli) on project demo4: Error, gluonfx:build failed -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
Log file
[Mon Sep 05 12:46:34 EEST 2022][INFO] ==================== LINK TASK ====================
[Mon Sep 05 12:46:34 EEST 2022][FINE] Looking for resource: /native/linux/launcher.c
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for compile-additional-sources: gcc -c -DSUBSTRATE -I/home/yousef66/Desktop/graalvm-ce-java17-22.1.0/include -I/home/yousef66/Desktop/graalvm-ce-java17-22.1.0/include/linux -DGVM_17 -I/home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4 launcher.c
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process compile-additional-sources...
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for compile-additional-sources: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for libavcodec: /usr/bin/pkg-config --libs libavcodec
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for libavcodec...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lavcodec
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for libavcodec: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg libavcodec provided flags: [-lavcodec]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for libavformat: /usr/bin/pkg-config --libs libavformat
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for libavformat...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lavformat
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for libavformat: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg libavformat provided flags: [-lavformat]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for libavutil: /usr/bin/pkg-config --libs libavutil
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for libavutil...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lavutil
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for libavutil: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg libavutil provided flags: [-lavutil]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for alsa: /usr/bin/pkg-config --libs alsa
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for alsa...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lasound
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for alsa: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg alsa provided flags: [-lasound]
[Mon Sep 05 12:46:34 EEST 2022][FINE] All flags: [-lgstreamer-lite, -lavcodec, -lavformat, -lavutil, -lasound]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for gl: /usr/bin/pkg-config --libs gl
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for gl...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lGL
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for gl: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg gl provided flags: [-lGL]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for x11: /usr/bin/pkg-config --libs x11
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for x11...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lX11
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for x11: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg x11 provided flags: [-lX11]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for gtk+-x11-3.0: /usr/bin/pkg-config --libs gtk+-x11-3.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for gtk+-x11-3.0...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for gtk+-x11-3.0: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg gtk+-x11-3.0 provided flags: [-lgtk-3, -lgdk-3, -lpangocairo-1.0, -lpango-1.0, -lharfbuzz, -latk-1.0, -lcairo-gobject, -lcairo, -lgdk_pixbuf-2.0, -lgio-2.0, -lgobject-2.0, -lglib-2.0]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for freetype2: /usr/bin/pkg-config --libs freetype2
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for freetype2...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lfreetype
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for freetype2: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg freetype2 provided flags: [-lfreetype]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for pangoft2: /usr/bin/pkg-config --libs pangoft2
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for pangoft2...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lharfbuzz -lfontconfig -lfreetype
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for pangoft2: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg pangoft2 provided flags: [-lpangoft2-1.0, -lpango-1.0, -lgobject-2.0, -lglib-2.0, -lharfbuzz, -lfontconfig, -lfreetype]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for gthread-2.0: /usr/bin/pkg-config --libs gthread-2.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for gthread-2.0...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lgthread-2.0 -pthread -lglib-2.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for gthread-2.0: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg gthread-2.0 provided flags: [-lgthread-2.0, -pthread, -lglib-2.0]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for zlib: /usr/bin/pkg-config --libs zlib
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for zlib...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lz
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for zlib: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg zlib provided flags: [-lz]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for xtst: /usr/bin/pkg-config --libs xtst
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for xtst...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lXtst
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for xtst: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg xtst provided flags: [-lXtst]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for Get config for gmodule-no-export-2.0: /usr/bin/pkg-config --libs gmodule-no-export-2.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process Get config for gmodule-no-export-2.0...
[Mon Sep 05 12:46:34 EEST 2022][FINE] [SUB] -lgmodule-2.0 -pthread -lglib-2.0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for Get config for gmodule-no-export-2.0: 0
[Mon Sep 05 12:46:34 EEST 2022][FINE] Pkg gmodule-no-export-2.0 provided flags: [-lgmodule-2.0, -pthread, -lglib-2.0]
[Mon Sep 05 12:46:34 EEST 2022][FINE] All flags: [-Wl,--no-whole-archive, -lGL, -lX11, -lgtk-3, -lgdk-3, -lpangocairo-1.0, -lpango-1.0, -lharfbuzz, -latk-1.0, -lcairo-gobject, -lcairo, -lgdk_pixbuf-2.0, -lgio-2.0, -lgobject-2.0, -lglib-2.0, -lfreetype, -lpangoft2-1.0, -lpango-1.0, -lgobject-2.0, -lglib-2.0, -lharfbuzz, -lfontconfig, -lfreetype, -lgthread-2.0, -pthread, -lglib-2.0, -lstdc++, -lz, -lXtst, -lm, -lgmodule-2.0, -pthread, -lglib-2.0]
[Mon Sep 05 12:46:34 EEST 2022][FINE] PB Command for link: gcc /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/demo4/launcher.o /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/tmp/SVM-1662371091887/com.example.demo4.driver.o /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libjava.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libnio.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libzip.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libnet.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libprefs.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libj2pkcs11.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libextnet.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libfdlibm.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libfontmanager.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libjavajpeg.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/liblcms.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libawt_headless.a /home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/static/linux-amd64/glibc/libawt.a -l:libjvm.a -l:liblibchelper.a -lz -ldl -lstdc++ -lpthread -Wl,--wrap=pow -rdynamic -lprism_es2 -lglass -lglassgtk3 -ljavafx_font -ljavafx_font_freetype -ljavafx_font_pango -ljavafx_iio -lgstreamer-lite -lavcodec -lavformat -lavutil -lasound -Wl,--no-whole-archive -lGL -lX11 -lgtk-3 -lgdk-3 -lpangocairo-1.0 -lpango-1.0 -lharfbuzz -latk-1.0 -lcairo-gobject -lcairo -lgdk_pixbuf-2.0 -lgio-2.0 -lgobject-2.0 -lglib-2.0 -lfreetype -lpangoft2-1.0 -lpango-1.0 -lgobject-2.0 -lglib-2.0 -lharfbuzz -lfontconfig -lfreetype -lgthread-2.0 -pthread -lglib-2.0 -lstdc++ -lz -lXtst -lm -lgmodule-2.0 -pthread -lglib-2.0 -lm -ldl -o /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/demo4 -L/home/yousef66/Desktop/graalvm-ce-java17-22.1.0/lib/svm/clibraries/linux-amd64 -L/home/yousef66/.gluon/substrate/javafxStaticSdk/19-ea+8/linux-x86_64/sdk/lib -ljaas
[Mon Sep 05 12:46:34 EEST 2022][FINE] Start process link...
[Mon Sep 05 12:46:34 EEST 2022][INFO] [SUB] /usr/bin/ld: cannot find -ljaas
[Mon Sep 05 12:46:34 EEST 2022][INFO] [SUB] collect2: error: ld returned 1 exit status
[Mon Sep 05 12:46:34 EEST 2022][FINE] Result for link: 1
[Mon Sep 05 12:46:34 EEST 2022][SEVERE] Process link failed with result: 1
Check the log files under /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/log
And please check https://docs.gluonhq.com/ for more information.
[Mon Sep 05 12:46:34 EEST 2022][INFO] Logging process [link] to file: /home/yousef66/Desktop/demo4/target/gluonfx/log/process-link-1662371194695.log
[Mon Sep 05 12:46:34 EEST 2022][SEVERE] Linking failed.
Check the log files under /home/yousef66/Desktop/demo4/target/gluonfx/x86_64-linux/gvm/log
And please check https://docs.gluonhq.com/ for more information.

kafka connect to Google BigQuery throws error java.lang.NoClassDefFoundError: org/apache/kafka/common/config/ConfigDef$CaseInsensitiveValidString

I am trying to stream from a Kafka topic to Google BigQuery. My connect-standalone.properties file is as follows:
bootstrap.servers=rhes75:9092,rhes75:9093,rhes75:9094,rhes564:9092,rhes564:9093,rhes564:9094,rhes76:9092,rhes76:9093,rhes76:9094
key.converter=org.apache.kafka.connect.storage.StringConverter
##value.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
##internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter=org.apache.kafka.connect.storage.StringConverter
##internal.value.converter=org.apache.kafka.connect.storage.StringConverter
internal.key.converter.schemas.enable=false
internal.value.converter.schemas.enable=false
offset.storage.file.filename=/tmp/connect_bq.offsets
offset.flush.interval.ms=10000
#plugin.path=/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins
plugin.path=/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins/wepay-kafka-connect-bigquery-2.1.0
and my bigquery-sink.properties is as below
name=bigquery-sink
connector.type=bigquery-connector
connector.class=com.wepay.kafka.connect.bigquery.BigQuerySinkConnector
defaultDataset=test
project=axial-glow-224522
topics=md
autoCreateTables=false
gcsBucketName=tmp_storage_bucket
queueSize=-1
bigQueryRetry=0
bigQueryRetryWait=1000
bigQueryMessageTimePartitioning=false
bigQueryPartitionDecorator=true
timePartitioningType=DAY
keySource=FILE
keyfile=/home/hduser/xyz.json
sanitizeTopics=false
schemaRetriever=com.wepay.kafka.connect.bigquery.retrieve.IdentitySchemaRetriever
threadPoolSize=10
allBQFieldsNullable=false
avroDataCacheSize=100
batchLoadIntervalSec=120
convertDoubleSpecialValues=false
enableBatchLoad=false
upsertEnabled=false
deleteEnabled=false
mergeIntervalMs=60_000L
mergeRecordsThreshold=-1
autoCreateBucket=true
allowNewBigQueryFields=false
allowBigQueryRequiredFieldRelaxation=false
allowSchemaUnionization=false
kafkaDataFieldName=null
kafkaKeyFieldName=null
I run the code as below
#!/bin/ksh
unset CLASSPATH
export CLASSPATH=/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins/wepay-kafka-connect-bigquery-2.1.0/lib:$KAFKA_HOME/libs
export KAFKA_DEBUG=y
$KAFKA_HOME/bin/connect-standalone.sh \
/d4T/hduser/bigquery-kafka-connect-sink/etc/connect-standalone.properties \
/d4T/hduser/bigquery-kafka-connect-sink/etc/bigquery-sink.properties
It comes back with this error
[2021-03-13 18:17:38,499] INFO Started o.e.j.s.ServletContextHandler#2ecf5915{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler:744)
[2021-03-13 18:17:38,503] INFO Started http_8083#6badba10{HTTP/1.1}{0.0.0.0:8083} (org.eclipse.jetty.server.ServerConnector:266)
[2021-03-13 18:17:38,503] INFO Started #1981ms (org.eclipse.jetty.server.Server:379)
[2021-03-13 18:17:38,503] INFO Advertised URI: http://50.140.197.220:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:248)
[2021-03-13 18:17:38,503] INFO REST server listening at http://50.140.197.220:8083/, advertising URL http://50.140.197.220:8083/ (org.apache.kafka.connect.runtime.rest.RestServer:207)
[2021-03-13 18:17:38,503] INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect:55)
[2021-03-13 18:17:38,518] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:113)
java.lang.NoClassDefFoundError: org/apache/kafka/common/config/ConfigDef$CaseInsensitiveValidString
at com.wepay.kafka.connect.bigquery.config.BigQuerySinkConfig.getConfig(BigQuerySinkConfig.java:505)
at com.wepay.kafka.connect.bigquery.BigQuerySinkConnector.config(BigQuerySinkConnector.java:79)
at org.apache.kafka.connect.connector.Connector.validate(Connector.java:132)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:277)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:164)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:107)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.config.ConfigDef$CaseInsensitiveValidString
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
It complains about not finding this class
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.config.ConfigDef$CaseInsensitiveValidString
However, under directory
/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins/wepay-kafka-connect-bigquery-2.1.0/lib
it is in kafka-clients-1.1.0.jar
jar tvf kafka-clients-1.1.0.jar|grep org.apache.kafka.common.config.ConfigDef$CaseInsensitiveValidString
2051 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$1.class
1175 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$2.class
2876 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$3.class
1132 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$4.class
1801 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$CompositeValidator.class
3274 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$ConfigKey.class
1255 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Importance.class
1104 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$NonEmptyString.class
2144 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$NonEmptyStringWithoutControlChars.class
922 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$NonNullValidator.class
2072 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Range.class
583 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Recommender.class
1513 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Type.class
293 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Validator.class
1943 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$ValidList.class
2196 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$ValidString.class
1269 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef$Width.class
34225 Fri Mar 23 22:54:26 GMT 2018 org/apache/kafka/common/config/ConfigDef.class
So I have no idea what is missing in here?
The jars loaded are shown below
jvm.classpath = /d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins/wepay-kafka-connect-bigquery-2.1.0/lib:/data6/hduser/kafka_2.12-1.1.0/libs:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/aopalliance-repackaged-2.5.0-b32.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/argparse4j-0.7.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/commons-lang3-3.5.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/connect-api-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/connect-file-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/connect-json-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/connect-runtime-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/connect-transforms-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/guava-20.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/hk2-api-2.5.0-b32.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/hk2-locator-2.5.0-b32.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/hk2-utils-2.5.0-b32.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-annotations-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-core-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-databind-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-jaxrs-base-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-jaxrs-json-provider-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jackson-module-jaxb-annotations-2.9.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javassist-3.20.0-GA.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javassist-3.21.0-GA.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javax.annotation-api-1.2.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javax.inject-1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javax.inject-2.5.0-b32.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javax.servlet-api-3.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/javax.ws.rs-api-2.0.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-client-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-common-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-container-servlet-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-container-servlet-core-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-guava-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-media-jaxb-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jersey-server-2.25.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-client-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-continuation-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-http-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-io-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-security-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-server-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-servlet-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-servlets-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jetty-util-9.2.24.v20180105.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/jopt-simple-5.0.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-clients-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-log4j-appender-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-streams-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-streams-examples-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-streams-test-utils-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka-tools-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka_2.12-1.1.0-sources.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka_2.12-1.1.0-test-sources.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/kafka_2.12-1.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/log4j-1.2.17.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/lz4-java-1.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/maven-artifact-3.5.2.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/metrics-core-2.2.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/osgi-resource-locator-1.0.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/plexus-utils-3.1.0.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/reflections-0.9.11.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/rocksdbjni-5.7.3.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/scala-library-2.12.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/scala-logging_2.12-3.7.2.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/scala-reflect-2.12.4.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/slf4j-api-1.7.25.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/slf4j-log4j12-1.7.25.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/snappy-java-1.1.7.1.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/validation-api-1.1.0.Final.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/zkclient-0.10.jar:/data6/hduser/kafka_2.12-1.1.0/bin/../libs/zookeeper-3.4.10.jar
os.spec = Linux, amd64, 3.10.0-1062.9.1.el7.x86_64
os.vcpus = 6

Thanks all.
I was using an older Kafka version.
I upgraded Kafka in the cluster from kafka_2.12-1.1.0 to the latest stable version kafka_2.12-2.7.0. I also upgraded zookeeper from zookeeper-3.4.6 to apache-zookeeper-3.6.2-bin version.
In addition in the run file I added the following:
#!/bin/ksh
unset CLASSPATH
export CLASSPATH=/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins:$KAFKA_HOME/libs
$KAFKA_HOME/bin/connect-standalone.sh \
/d4T/hduser/bigquery-kafka-connect-sink/etc/connect-standalone.properties \
/d4T/hduser/bigquery-kafka-connect-sink/etc/bigquery-sink.properties
where the first part of CLASSPATH comes from plugins
plugin.path=/d4T/hduser/bigquery-kafka-connect-sink/share/kafka/plugins
This solved the issue

Why do I get "null" and "0" when I print out a text file I just read

I am supposed to print out the information that I just read from a text file and what I actually print is all "null" and zeros.
Here is what my Text File look like
1995 Jun 987 65 Allison
1995 Jul 973 85 Erin
1995 Aug 929 120 Felix
1995 Aug 968 95 Humberto
1995 Aug 965 95 Iris
1995 Aug 945 130 Luis
1995 Sep 949 100 Marilyn
1995 Sep 987 65 Noel
1995 Sep 916 130 Opal
1995 Oct 956 100 Roxanne
1995 Oct 972 75 Tanya
1996 Jul 960 100 Bertha
1996 Jul 985 75 Cesar
1996 Aug 989 70 Dolly
1996 Aug 933 125 Edouard
1996 Aug 946 105 Fran
1996 Sep 935 120 Hortense
1996 Sep 960 100 Isidore
1996 Oct 960 100 Lili
1996 Nov 983 65 Marco
1997 Jul 986 65 Bill
1997 Jul 984 70 Danny
1997 Sep 946 110 Erika
1998 Aug 954 100 Bonnie
1998 Aug 960 70 Danielle
1998 Aug 985 80 Earl
1998 Sep 937 135 Georges
1998 Sep 975 80 Ivan
1998 Sep 969 90 Jeanne
1998 Sep 970 90 Karl
1998 Oct 995 65 Lisa
1998 Oct 905 155 Mitch
1998 Nov 979 75 Nicole
1999 Aug 944 125 Bret
1999 Aug 942 120 Cindy
1999 Aug 962 90 Dennis
1999 Sep 921 135 Floyd
1999 Sep 930 130 Gert
1999 Oct 958 95 Irene
1999 Oct 979 85 Jose
1999 Nov 933 135 Lenny
2000 Aug 950 110 Alberto
2000 Aug 991 75 Debby
2000 Sep 985 70 Florence
2000 Sep 981 70 Gordon
2000 Sep 943 120 Isaac
2000 Sep 975 80 Joyce
2000 Sep 939 120 Keith
2000 Oct 965 85 Michael
2001 Sep 968 105 Erin
2001 Sep 962 100 Felix
2001 Sep 975 70 Gabrielle
2001 Sep 970 90 Humberto
2001 Oct 948 125 Iris
2001 Oct 982 70 Karen
2001 Oct 933 120 Michelle
2001 Nov 986 65 Noel
2001 Nov 973 80 Olga
2002 Sep 960 80 Gustav
2002 Sep 934 110 Isidore
2002 Sep 980 75 Kyle
2002 Sep 938 125 Lili
2003 Jul 979 80 Claudette
2003 Jul 1000 65 Danny
2003 Aug 986 65 Erika
2003 Aug 939 125 Fabian
2003 Sep 915 145 Isabel
2003 Sep 969 90 Juan
2003 Sep 952 110 Kate
2004 Jul 957 105 Alex
2004 Aug 941 130 Charley
2004 Aug 964 95 Danielle
2004 Aug 935 125 Frances
2004 Aug 985 65 Gaston
2004 Sep 910 145 Ivan
2004 Sep 950 105 Jeanne
2004 Sep 938 125 Karl
2004 Sep 987 65 Lisa
2005 Jul 991 65 Cindy
2005 Jul 930 130 Dennis
2005 Jul 929 140 Emily
2005 Aug 970 90 Irene
2005 Aug 902 150 Katrina
2005 Sep 962 100 Maria
2005 Sep 979 80 Nate
2005 Sep 976 75 Ophelia
2005 Sep 985 70 Philippe
2005 Sep 895 155 Rita
2005 Oct 977 70 Stan
2005 Oct 988 65 Vince
2005 Oct 882 160 Wilma
2005 Oct 962 100 Beta
2005 Nov 981 75 Epsilon
2006 Aug 992 65 Ernesto
2006 Sep 974 80 Florence
2006 Sep 955 105 Gordon
2006 Sep 955 105 Helene
2006 Sep 985 75 Isaac
2007 Aug 905 150 Dean
2007 Aug 929 150 Felix
2007 Sep 985 80 Humberto
2007 Sep 988 65 Karen
2007 Sep 990 70 Lorenzo
2007 Oct 980 70 Noel
2008 Jul 952 110 Bertha
2008 Jul 963 85 Dolly
2008 Aug 941 135 Gustav
2008 Aug 977 75 Hanna
2008 Sep 935 125 Ike
2008 Sep 984 70 Kyle
2008 Oct 958 115 Omar
2008 Nov 970 85 Paloma
2009 Aug 943 115 Bill
2009 Sep 958 105 Fred
2009 Nov 975 90 Ida
2010 Jun 946 95 Alex
2010 Aug 942 115 Danielle
2010 Aug 927 125 Earl
2010 Sep 924 135 Igor
2010 Sep 948 120 Julia
2010 Sep 956 110 Karl
2010 Sep 982 75 Lisa
2010 Oct 976 75 Otto
2010 Oct 981 90 Paula
2010 Oct 977 85 Richard
2010 Oct 989 65 Shary
2010 Oct 985 85 Tomas
2011 Aug 942 105 Irene
2011 Aug 942 120 Katia
2011 Sep 983 70 Maria
2011 Sep 994 65 Nate
2011 Sep 940 120 Ophelia
2011 Sep 976 80 Philippe
2011 Oct 966 100 Rina
2012 Jun 974 75 Chris
2012 Aug 973 85 Ernesto
2012 Aug 965 95 Gordon
2012 Aug 965 70 Isaac
2012 Aug 970 90 Kirk
2012 Aug 968 70 Leslie
2012 Sep 964 100 Michael
2012 Sep 978 80 Nadine
2012 Oct 969 80 Rafael
2012 Oct 940 100 Sandy
2013 Sep 979 80 Humberto
2013 Sep 983 75 Ingrid
2014 Jul 973 85 Arthur
2014 Aug 998 70 Bertha
2014 Aug 965 75 Cristobal
2014 Sep 955 105 Edouard
2014 Oct 983 70 Fay
2014 Oct 940 125 Gonzalo
2015 Aug 960 110 Danny
2015 Aug 986 75 Fred
2015 Sep 931 135 Joaquin
2015 Nov 980 75 Kate
The expected output
Year Name Pressure wind Speed
1995 Allison 987 65
1995 Erin, 973 85
1995 Felix 929 120
1995 Humberto 968 95
1995 Iris 965 95
1995 Luis 945 130
public class HurricaneSelector
{
public static void main(String[] args) throws IOException
{
Scanner in = new Scanner (System.in);
File fileName = new File ("hurricanedata.txt");
Scanner inFile = new Scanner(fileName);
int counter = 0;
while (inFile.hasNextLine())
{
inFile.nextLine();
counter++;
}
int [] year = new int[counter];
String [] month = new String[counter];
String [] name = new String[counter];
int [] pressure = new int[counter];
int [] windSpeed = new int[counter];
int index = 0;
while (inFile.hasNext())
{
year[index] = inFile.nextInt();
month[index] = inFile.nextLine();
pressure[index] = inFile.nextInt();
windSpeed[index] = inFile.nextInt();
name[index] = inFile.nextLine();
index++;
}
for (int n = 0; n < year.length; n++)
{
System.out.println(year[n] + " " + month[n] + " " + name[n] + " " + windSpeed[n]);
}
}
}

After while (inFile.hasNextLine()) the file stream is exhausted and will need to be re-read from the beginning again.
You should close the file first and then re-open it.
inFile.close ();
inFile = new Scanner(fileName);

how solve that exception in java creating bar chart? [duplicate]

Anyone can tell me where can I find the full version of JFreeChart? I downloaded the JCommon and JFreeChart as well (from the sourceforge page), but the JFreeChart contains only the jfreechart-demo.jar, so even if I correctly added the libraries in Eclipse, it doesn't work either...
Many Thanks!

When unpacked, the required JAR files are in the lib directory.
$ ls -ln lib
total 5800
-rw-r--r--# 1 501 20 45024 Jul 31 2014 hamcrest-core-1.3.jar
-rw-r--r--# 1 501 20 330246 Jul 31 2014 jcommon-1.0.23.jar
-rw-r--r--# 1 501 20 13796 Jul 31 2014 jfreechart-1.0.19-experimental.jar
-rw-r--r--# 1 501 20 79793 Jul 31 2014 jfreechart-1.0.19-swt.jar
-rw-r--r--# 1 501 20 1570157 Jul 31 2014 jfreechart-1.0.19.jar
-rw-r--r--# 1 501 20 50769 Jul 31 2014 jfreesvg-2.0.jar
-rw-r--r--# 1 501 20 245039 Jul 31 2014 junit-4.11.jar
-rw-r--r--# 1 501 20 445025 Jul 31 2014 orsoncharts-1.4-eval-nofx.jar
-rw-r--r--# 1 501 20 72444 Jul 31 2014 orsonpdf-1.6-eval.jar
-rw-r--r--# 1 501 20 80054 Jul 31 2014 servlet.jar
-rw-r--r--# 1 501 20 17492 Jul 31 2014 swtgraphics2d.jar
For example, the following command line runs TimeSeriesChartDemo1 on Unix; separate class path entries with a semicolon (;) on Windows.
$ java -cp .:lib/* org.jfree.chart.demo.TimeSeriesChartDemo1

Reading txt file from jar fails but reading image works

I have an issue which has been bothering me for days... I checked similar questions but couldn't find a solution.
I use NetBeans IDE. I build the project jar file i.e "Clock.jar" which contains a "clock" named folder in which some images, a text file and all project classes are found.
The following code for creating an image icon works
return new ImageIcon(getClass().getResource("/clock/button_close.png"));
But the following code for reading the text file fails
InputStream name = getClass().getResourceAsStream("/clock/input.txt");
BufferedReader reader = new BufferedReader(new InputStreamReader(name));
As you may have guessed the NullPointer Exception is thrown meaning probably it couln't locate the file.
But how come the image icon is constructed successfully (by passing it the URL returned from getResource) but the txt file cannot be found (by passing it an input stream from getResourceAsStream).
Thanks in advance, for any answer ( I mean it :) )
jar -tvf Clock.jar
0 Wed May 15 14:44:36 EEST 2013 META-INF/
202 Wed May 15 14:44:34 EEST 2013 META-INF/MANIFEST.MF
0 Wed May 15 14:44:36 EEST 2013 clock/
649 Wed May 15 14:44:36 EEST 2013 clock/Clock$1$1.class
789 Wed May 15 14:44:36 EEST 2013 clock/Clock$1.class
2026 Wed May 15 14:44:36 EEST 2013 clock/Clock.class
709 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$1.class
830 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$2.class
750 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$3.class
713 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$4.class
741 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$5.class
708 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$6.class
1081 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$7.class
981 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog$8.class
9640 Wed May 15 14:44:36 EEST 2013 clock/ClockDialog.class
702 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$1.class
708 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$2.class
734 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$3.class
743 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$4.class
531 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$5.class
1046 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame$6.class
9464 Wed May 15 14:44:36 EEST 2013 clock/ClockFrame.class
782 Wed May 15 14:44:36 EEST 2013 clock/ErrorReporter.class
1826 Wed May 15 14:44:36 EEST 2013 clock/IconButton.class
2693 Wed May 15 14:44:36 EEST 2013 clock/MessagePool.class
2824 Wed May 15 14:44:36 EEST 2013 clock/SystemInfo.class
2212 Wed May 15 14:44:36 EEST 2013 clock/button_close.png
6540 Wed May 15 14:44:36 EEST 2013 clock/button_close_highlighted.png
5668 Wed May 15 14:44:36 EEST 2013 clock/input.txt

Looking at the source again closely, try..
URL url = getClass().getResource("/clock/input.txt");
InputStream name = url.openStream();
I recall that the Class::getResourceAsStream variant deals with paths slightly differently than simply Class::getResource - I am sure it should work for the latter one.

Categories

Resources