I initialize the logger like this:
public static void init() {
ConsoleHandler handler = new ConsoleHandler();
handler.setFormatter(new LogFormatter());
Logger.getLogger(TrackerConfig.LOGGER_NAME).setUseParentHandlers(false);
Logger.getLogger(TrackerConfig.LOGGER_NAME).addHandler(handler);
}
The LogFormatter's format function:
#Override
public String format(LogRecord record) {
StringBuilder sb = new StringBuilder();
sb.append(new SimpleDateFormat("yyyy-MM-dd HH:mm:ss Z").format(new Date(record.getMillis())))
.append(" ")
.append(record.getLevel().getLocalizedName()).append(": ")
.append(formatMessage(record)).append(LINE_SEPARATOR);
return sb.toString();
}
To use the Log I use the following method:
private static void log(Level level, String message) {
Logger.getLogger(TrackerConfig.LOGGER_NAME).log(level, message);
if (level.intValue() >= TrackerConfig.DB_LOGGER_LEVEL.intValue()) {
DBLog.getInstance().log(level, message);
}
}
The DBLog.log method:
public void log(Level level, String message) {
try {
this.logBatch.setTimestamp(1, new Timestamp(Calendar.getInstance().getTime().getTime()));
this.logBatch.setString(2, level.getName());
this.logBatch.setString(3, message);
this.logBatch.addBatch();
} catch (SQLException ex) {
Log.logError("SQL error: " + ex.getMessage()); // if this happens the code will exit anyways so it will not cause a loop
}
}
Now a normal Log output looks like that:
2013-04-20 18:00:59 +0200 INFO: Starting up Tracker
It works for some time but the LogFormatter seems to be reset for whatever reason.
Sometimes only one Log entry is displayed correctly and after that the Log entries are displayed like:
Apr 20, 2013 6:01:01 PM package.util.Log log INFO:
Loaded 33266 database entries.
again.
What I tried:
For debugging purposes I added a thread that outputs the memory usage of the jvm every x seconds.
The output worked with the right Log Format until the reserved memory value changed (the free memory value change did not reset the log format) like this:
2013-04-20 18:16:24 +0200 WARNING: Memory usage: 23 / 74 / 227 MiB
2013-04-20 18:16:25 +0200 WARNING: Memory usage: 20 / 74 / 227 MiB
2013-04-20 18:16:26 +0200 WARNING: Memory usage: 18 / 74 / 227 MiB
Apr 20, 2013 6:16:27 PM package.util.Log log WARNING:
Memory usage: 69 / 96 / 227 MiB
Apr 20, 2013 6:16:27 PM package.util.Log log INFO:
Scheduler running
Apr 20, 2013 6:16:27 PM package.Log log WARNING:
Memory usage: 67 / 96 / 227 MiB
Also note that the log level seems to be reset from warning to info here.
Where the problem seems to be:
When I comment out the database log function like this:
private static void log(Level level, String message) {
Logger.getLogger(TrackerConfig.LOGGER_NAME).log(level, message);
if (level.intValue() >= TrackerConfig.DB_LOGGER_LEVEL.intValue()) {
// DBLog.getInstance().log(level, message);
}
}
the log is formatted properly.
Any ideas what could be wrong with the DBLog's log function or why the log suddenly resets?
I would not really call this a solution but it works now.
The cause seemed to be the memory calculation itself.
Even if I just calculated it without logging it, the log format was reset.
I have no idea why it worked when I just commented out the DBLog usage.
int mb = 1024 * 1024;
long freeMemory = Runtime.getRuntime().freeMemory() / mb;
long reservedMemory = Runtime.getRuntime().totalMemory() / mb;
long maxMemory = Runtime.getRuntime().maxMemory() / mb;
String memoryUsage = "Memory usage: " + freeMemory + " / " + reservedMemory + " / " + maxMemory + " MiB";
This is the code I used. As soon as I commented it out the log format did not reset anymore and now everything works as expected.
Related
I tried to compress Inputmask.js file. Here is my code to work
public class JSFileMinifyTest {
public static void main(String[] args) throws Exception {
String sourceFileName = "D:\\temp\\jquery.inputmask.bundle.js";
String outputFilename = "D:\\temp\\combined.min.js";
com.google.javascript.jscomp.Compiler.setLoggingLevel(Level.INFO);
com.google.javascript.jscomp.Compiler compiler = new com.google.javascript.jscomp.Compiler();
CompilerOptions options = new CompilerOptions();
CompilationLevel.WHITESPACE_ONLY.setOptionsForCompilationLevel(options);
options.setAggressiveVarCheck(CheckLevel.OFF);
options.setRuntimeTypeCheck(false);
options.setCheckRequires(CheckLevel.OFF);
options.setCheckProvides(CheckLevel.OFF);
options.setReserveRawExports(false);
WarningLevel.VERBOSE.setOptionsForWarningLevel(options);
// To get the complete set of externs, the logic in
// CompilerRunner.getDefaultExterns() should be used here.
SourceFile extern = SourceFile.fromCode("externs.js", "function alert(x) {}");
SourceFile jsFile = SourceFile.fromFile(sourceFileName);
compiler.compile(extern, jsFile, options);
for (JSError message : compiler.getWarnings()) {
System.err.println("Warning message: " + message.toString());
}
for (JSError message : compiler.getErrors()) {
System.err.println("Error message: " + message.toString());
}
FileWriter outputFile = new FileWriter(outputFilename);
outputFile.write(compiler.toSource());
outputFile.close();
}
}
But error occured while compressing this js file.
Apr 09, 2018 11:29:18 AM com.google.javascript.jscomp.parsing.ParserRunner parse
INFO: Error parsing D:\temp\jquery.inputmask.bundle.js: Compilation produced 3 syntax errors. (D:\temp\jquery.inputmask.bundle.js#1)
Apr 09, 2018 11:29:18 AM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: D:\temp\jquery.inputmask.bundle.js:1002: ERROR - Parse error. identifier is a reserved word
static || null !== test.fn && void 0 !== testPos.input ? static && null !== test.fn && void 0 !== testPos.input && (static = !1,
^
Apr 09, 2018 11:29:18 AM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: D:\temp\jquery.inputmask.bundle.js:1003: ERROR - Parse error. syntax error
maskTemplate += "</span>") : (static = !0, maskTemplate += "<span class='im-static''>");
^
Apr 09, 2018 11:29:18 AM com.google.javascript.jscomp.LoggerErrorManager println
SEVERE: D:\temp\jquery.inputmask.bundle.js:1010: ERROR - Parse error. missing variable name
var maskTemplate = "", static = !1;
^
Apr 09, 2018 11:29:18 AM com.google.javascript.jscomp.LoggerErrorManager printSummary
WARNING: 3 error(s), 0 warning(s)
Error message: JSC_PARSE_ERROR. Parse error. identifier is a reserved word at D:\temp\jquery.inputmask.bundle.js line 1002 : 16
Error message: JSC_PARSE_ERROR. Parse error. syntax error at D:\temp\jquery.inputmask.bundle.js line 1003 : 29
Error message: JSC_PARSE_ERROR. Parse error. missing variable name at D:\temp\jquery.inputmask.bundle.js line 1010 : 39
Has there anyways to skip syntax validations or ignore errors and continue to compress ?
The compiler by default, parses code as 'use strict'. 'static' is a reserved word in strict mode. You can either change the language mode or simply renamed the variable from 'strict' to something else.
The strict mode reserve words are documented here:
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Strict_mode#Paving_the_way_for_future_ECMAScript_versions
I have Java web application with REST calls using SPRING.
I want to control the number of threads the application is opening for the requests.
So I added Thread config:
package myPackage;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.TaskExecutor;
import org.springframework.scheduling.concurrent.ThreadPoolTaskExecutor;
#Configuration
public class ThreadConfig {
#Bean
public TaskExecutor threadPoolTaskExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(1);
executor.setMaxPoolSize(1);
executor.initialize();
return executor;
}
}
I'm using Sync service not Async, I tested it and it doesn't limit the threads handling the requests, it handles them all at the same time.
What I was expecting is when I send 2 requests at a time - either the 2nd request will be thrown or it will wait until the 1st request will finish.
I'm not implementing Thread in my application at all.
This is the relevant code from my controller:
#RestController
public class Module1Controller {
#RequestMapping(method = RequestMethod.GET, path = "/module1")
InterruptedException {
public Module1 Module1() throws InterruptedException {
Date startDate = new Date();
System.out.println("Thread #: " + Thread.currentThread().getId() + " Request received at: " + startDate);
Thread.sleep(10000);
Date endDate = new Date();
long diff = endDate.getTime() - startDate.getTime();
long seconds = TimeUnit.MILLISECONDS.toSeconds(diff);
System.out.println("Thread #: " + Thread.currentThread().getId() + " thread released at: " + endDate + ", total seconds: " + seconds);
return new Module1(new Clock());
}
This is the console result:
Thread #: 34 Request received at: Sun Dec 17 10:16:20 IST 2017
Thread #: 35 Request received at: Sun Dec 17 10:16:21 IST 2017
Thread #: 34 thread released at: Sun Dec 17 10:16:30 IST 2017, total seconds: 10
Thread #: 35 thread released at: Sun Dec 17 10:16:31 IST 2017, total seconds: 10
What am I missing here?
The problem is that the creation of a TaskExecutor in a configuration bean has no effect on your RestController.
The easiest way to make your RestController process only 1 request at a time is to make the handling method synchronized, e.g. like this:
#RequestMapping(method = RequestMethod.GET, path = "/module1")
public synchronized Module1 getModule1() throws InterruptedException {
If you want a certain maximum number of requests to be processed simultaneously you can use a FixedThreadPool, e.g. like this:
// allow only 2 requests at a time, more requests are automatically placed in a queue
private final ExecutorService es = Executors.newFixedThreadPool(2);
#RequestMapping(method = RequestMethod.GET, path = "/module1")
public Module1 getModule1() throws ExecutionException, InterruptedException {
Future<Module1> result = es.submit(new Callable<Module1>() {
#Override
public String call() throws Exception {
try {
//.... do your work here....
return Module1()
} catch (InterruptedException e) {
return null;
}
}
});
return result.get();
}
I'm not sure why you would want to do this. Limiting the number of requests will result in bad performance and users are not going to like this.
You can not control the threads of request in application instead in container. Maybe you want to run some tasks in limited threads in application. You can do like this:
#RestController
public class ThreadController {
#Autowired
private TaskExecutor taskExecutor;
#RequestMapping(method = RequestMethod.GET, path = "/thread")
public void Module1() {
taskExecutor.execute(new Runnable() {
#Override
public void run() {
Date startDate = new Date();
System.out.println("Thread #: " + Thread.currentThread().getId() +
" Request received at: " + startDate);
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
Date endDate = new Date();
long diff = endDate.getTime() - startDate.getTime();
long seconds = TimeUnit.MILLISECONDS.toSeconds(diff);
System.out.println("Thread #: " + Thread.currentThread().getId() +
" thread released at: " + endDate + ", total seconds: " + seconds);
}
});
}
}
The result:
Thread #: 55 Request received at: Sun Dec 17 22:40:57 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:07 CST 2017, total seconds: 10
Thread #: 55 Request received at: Sun Dec 17 22:41:16 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:26 CST 2017, total seconds: 10
Thread #: 55 Request received at: Sun Dec 17 22:41:32 CST 2017
Thread #: 55 thread released at: Sun Dec 17 22:41:42 CST 2017, total seconds: 10
Can we store the value returned from omdb function searchOneMovie(moviename) so that it can be displayed as output in JSP page when button is clicked? I am developing a Web-Application where on clicking the button I should get the details of movie as searched by user but by using searchOneMovie() I am getting the output in console not in web page. Please tell how to display searchOneMovie() value in JSP page.
CODE:
public void search(HttpServletResponse response, String moviename) throws OmdbSyntaxErrorException, OmdbConnectionErrorException, OmdbMovieNotFoundException, IOException {
Omdb o = new Omdb();
try {
PrintWriter out = response.getWriter();
out.print(o.searchOneMovie(moviename));
} catch (Exception ignore) {
}
}
OUTPUT: This output is coming on console but i wan't it in webpage please help.
Mar 13, 2017 8:13:51 PM com.omdbapi.RestClient execute
INFO: executing GET http://www.omdbapi.com/?t=star+wars HTTP/1.1
Mar 13, 2017 8:13:52 PM com.omdbapi.Omdb resultToJson
INFO: received {"Title":"Star Wars: Episode IV - A New Hope","Year":"1977","Rated":"PG","Released":"25 May 1977","Runtime":"121 min","Genre":"Action, Adventure, Fantasy","Director":"George Lucas","Writer":"George Lucas","Actors":"Mark Hamill, Harrison Ford, Carrie Fisher, Peter Cushing","Plot":"Luke Skywalker joins forces with a Jedi Knight, a cocky pilot, a wookiee and two droids to save the galaxy from the Empire's world-destroying battle-station, while also attempting to rescue Princess Leia from the evil Darth Vader.","Language":"English","Country":"USA","Awards":"Won 6 Oscars. Another 50 wins & 28 nominations.","Poster":"https://images-na.ssl-images-amazon.com/images/M/MV5BYzQ2OTk4N2QtOGQwNy00MmI3LWEwNmEtOTk0OTY3NDk2MGJkL2ltYWdlL2ltYWdlXkEyXkFqcGdeQXVyNjc1NTYyMjg#._V1_SX300.jpg","Metascore":"92","imdbRating":"8.7","imdbVotes":"963,318","imdbID":"tt0076759","Type":"movie","Response":"True"}
I am encountering with a senerior like this:
My project has a servlet to catch a request from perl. The request is to download a file. The request is a multipartRequest.
#RequestMapping(value = "/*", method = RequestMethod.POST)
public void tdRequest(#RequestHeader("Authorization") String authenticate,
HttpServletResponse response,
HttpServletRequest request) throws Exception
{
if (ServletFileUpload.isMultipartContent(request))
{
ServletFileUpload sfu = new ServletFileUpload();
FileItemIterator items = sfu.getItemIterator(request);
while (items.hasNext())
{
FileItemStream item = items.next();
if (("action").equals(item.getFieldName()))
{
InputStream stream = item.openStream();
String value = Streams.asString(stream);
if (("upload").equals(value))
{
uploadRequest(items, response);
return;
}
else if (("download").equals(value))
{
downloadRequest(items, response);
return;
}
The problem is not here, it appears on the downloadRequest() function.
void downloadRequest(FileItemIterator items,
HttpServletResponse response) throws Exception
{
log.info("Start downloadRequest.......");
OutputStream os = response.getOutputStream();
File file = new File("D:\\clip.mp4");
FileInputStream fileIn = new FileInputStream(file);
//while ((datablock = dataOutputStreamServiceImpl.readBlock()) != null)
byte[] outputByte = new byte[ONE_MEGABYE];
while (fileIn.read(outputByte) != -1)
{
System.out.println("--------" + (i = i + 1) + "--------");
System.out.println(new Date());
//dataContent = datablock.getContent();
System.out.println("Start write " + new Date());
os.write(outputByte, 0,outputByte.length);
System.out.println("End write " + new Date());
//System.out.println("----------------------");
}
os.close();
}
}
I try to read and write blocks of 1MB from the file. However, it takes too long for downloading the whole file. ( my case is 20mins for file of 100MB)
I try to sysout and I saw a result like this:
The first few blocks can read, write data realy fast:
--------1--------
Mon Dec 07 16:24:20 ICT 2015
Start write Mon Dec 07 16:24:20 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
--------2--------
Mon Dec 07 16:24:21 ICT 2015
Start write Mon Dec 07 16:24:21 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
--------3--------
Mon Dec 07 16:24:21 ICT 2015
Start write Mon Dec 07 16:24:21 ICT 2015
End write Mon Dec 07 16:24:21 ICT 2015
But the next block is slower than the previous
--------72--------
Mon Dec 07 16:29:22 ICT 2015
Start write Mon Dec 07 16:29:22 ICT 2015
End write Mon Dec 07 16:29:29 ICT 2015
--------73--------
Mon Dec 07 16:29:29 ICT 2015
Start write Mon Dec 07 16:29:29 ICT 2015
End write Mon Dec 07 16:29:37 ICT 2015
--------124--------
Mon Dec 07 16:38:22 ICT 2015
Start write Mon Dec 07 16:38:22 ICT 2015
End write Mon Dec 07 16:38:35 ICT 2015
--------125--------
Mon Dec 07 16:38:35 ICT 2015
Start write Mon Dec 07 16:38:35 ICT 2015
End write Mon Dec 07 16:38:48 ICT 2015
The problem is in the os.write()
I realy cannot understand how the outputStream write, why it take such a long time like that? or I made some mistakes?
Sorry for my bad english. I realy need your support. Thank in advance!
This is the perl code from the client side
# ----- get connected to download the file
#
$Response = $ua->request(POST $remoteHost ,
Content_Type => 'form-data',
Authorization => $Authorization,
'Proxy-Authorization' => $Proxy_Authorization ,
Content => [ DOS => 1 ,
action => 'download' ,
first_run => 0 ,
dl_filename => $dl_filename ,
delivery_dir => $delivery_dir ,
verbose => $Verbose ,
debug => $debug ,
version => $VERSION
]
);
unless ($Response->is_success) {
my $Msg = $Response->error_as_HTML;
# Remove HTML tags - we're in a DOS shell!
$Msg =~ s/<[^>]+>//g;
print "ERROR! SERVER RESPONSE:\n$Msg\n";
print "$remoteHost\n\n" if $Options{'v'};
Error "Could not connect to " . $remoteHost ;
}
my $Result2 = $Response->content();
Error "Abnormal termination...\n$Result2" if $Result2 =~ /_APP_ERROR_/;
open(F, ">$dl_filename") or Error "Could not open '$dl_filename'!";
binmode F; # unless $dl_filename =~ /\.txt$|\.htm$/;
print F $Result2;
close F;
print "received.\n";
}
One problem is that fileIn.read(outputByte) can read random number of bytes, not only full outputByte. You read few KB, then you store full 1MB, and very fast you are running out of space on disk. Try this, notice the "readed" parameter.
void downloadRequest(FileItemIterator items,
HttpServletResponse response) throws Exception
{
log.info("Start downloadRequest.......");
OutputStream os = response.getOutputStream();
File file = new File("D:\\clip.mp4");
FileInputStream fileIn = new FileInputStream(file);
//while ((datablock = dataOutputStreamServiceImpl.readBlock()) != null)
byte[] outputByte = new byte[ONE_MEGABYE];
int readed =0;
while ((readed =fileIn.read(outputByte)) != -1)
{
System.out.println("--------" + (i = i + 1) + "--------");
System.out.println(new Date());
//dataContent = datablock.getContent();
System.out.println("Start write " + new Date());
os.write(outputByte, 0,readed );
System.out.println("End write " + new Date());
//System.out.println("----------------------");
}
os.close();
}
}
It looks like your download performance gets slower and slower, the further you are getting into the download. You start out at one or less seconds per block, by block 72 it is 7+ seconds per block and by block 128 it is 13 seconds per block.
There is nothing on the server side to explain this. Rather, it has the "smell" of the client side doing something wrong. My guess is that the client side is reading the data from the socket into an in-memory data structure, and that data structure (maybe just a String or StringBuffer or StringBuilder) is getting larger and larger. Either the time take to expand it is getting larger, or your memory footprint is growing and the GC is taking longer and longer. (Or both.)
If you showed us the client-side code .....
UPDATE
As I suspected, this line of code will be reading the entire content into the Perl equivalent of a string builder before turning it into a string.
my $Result2 = $Response->content();
Depending on how it is implemented under the hood, this will lead to repeated copying of the data as the builder runs out of buffer space and needs to be expanded. Depending on the buffer expansion strategy that Perl employs for this, it could give O(N^2) behavior, where N is the size of the file you are transferring. (The evidence is that you are not getting O(N) behavior ...)
If you want a faster downloads, you need to stream the data on the client side. Read the response content in chunks and write them to the output file. (I'm not a Perl expert, so I can't offer you code.) This will also reduce the memory footprint on the client side ... which could be important if your file sizes increase.
I was trying a example code with spring. And a part of code is like below;
private List<Point> points;
long timeTakeninMilis = System.currentTimeMillis();
public List<Point> getPoints() {
return points;
}
public void setPoints(List<Point> points) {
this.points = points;
}
public void drawJava8() {
points.stream().forEachOrdered(
point -> System.out.println("Point : (" + point.getX() + ", "
+ point.getY() + ")"));
System.out.println("Total Time Taken drawJava8(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
public void draw() {
for (Point point : points) {
System.out.println("Point = (" + point.getX() + ", " + point.getY()
+ " )");
}
System.out.println("Total Time Taken draw(): "
+ (System.currentTimeMillis() - timeTakeninMilis)
+ " miliseconds");
}
The OUTPUT,
Jun 30, 2015 11:30:53 AM org.springframework.context.support.ClassPathXmlApplicationContext prepareRefresh
INFO: Refreshing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Jun 30, 2015 11:30:53 AM org.springframework.beans.factory.xml.XmlBeanDefinitionReader loadBeanDefinitions
INFO: Loading XML bean definitions from class path resource [spring.xml]
Point = (0, 0 )
Point = (-50, 0 )
Point = (0, 50 )
Total Time Taken draw(): 70 miliseconds
Point : (0, 0)
Point : (-50, 0)
Point : (0, 50)
Total Time Taken drawJava8(): 124 miliseconds
Jun 30, 2015 11:30:54 AM org.springframework.context.support.ClassPathXmlApplicationContext doClose
INFO: Closing org.springframework.context.support.ClassPathXmlApplicationContext#7daf6ecc: startup date [Tue Jun 30 11:30:53 IST 2015]; root of context hierarchy
Why it is taking more time?
Or i am doing something wrong?
I was expecting it to be faster or of similar speed...
Please help me understand what is the benefit of the Lambda Expressions?
INFO: I did it in two different programs. Times are taken from those. I merged them here to make it short.
Adding this as an analysis per original poster's request.
We can not really predict the sophisticated analysis and transformation that the modern JIT compiler performs on running code. Hence while benchmarking items such as these, you should not conclude it just by running two method calls.
Instead, create various sample input sets (boundary cases) and check the perofmance by repeatedly calling your test cases without shutting down JVM. In this case for example :
for (int i=0;i<100;i++){draw(); drawJava8();}
Once you have the results, find out average execution and you can safely ignore first execution result as it might not have had optimizations.
So the conclusion you have drawn from your tests is not completely correct.