I'm working on scheduling project for classes (Teachers, Lessons, time). I'm using optaplanner as part of spring-boot application, the test code is compiling and running correctly however the result contain empty solution, in the log output I see this message:
rted: time spent (11), best score (0hard/0soft), environment mode (REPRODUCIBLE), move thread count (NONE), random (JDK with seed 0).
2021-09-28 22:39:26.619 INFO 2579 --- [pool-1-thread-1] o.o.core.impl.solver.DefaultSolver : Skipped all phases (2): out of 0 planning entities, none are movable (non-pinned).
2021-09-28 22:39:26.620 INFO 2579 --- [pool-1-thread-1] o.o.core.impl.solver.DefaultSolver : Solving ended: time spent (16), best score (0hard/0soft), score calculation speed (62/sec), phase total (2), environment mode (REPRODUCIBLE), move thread count (NONE).
The problem is in the test calculator I wrote I'm trying to loop on the possible solution and actually decrease the cost a bit sometimes or even increase it, but it doesn't taking effect because I'm looping and trying to log the objects but nothing is being logged, this is the code of the calculator:
public class ScheduleScoreCalculator implements EasyScoreCalculator<ScheduleTable, HardSoftScore>
{
#Override
public HardSoftScore calculateScore(ScheduleTable scheduleTable) {
int hardScore = 0;
int softScore = 0;
List<ScheduledClass> scheduledClassList = scheduleTable.getScheduledClasses();
System.out.println(scheduleTable);
System.out.println("Hmmmmm ---"); // this is logged but the score is not changing
for(ScheduledClass a: scheduledClassList) {
for (ScheduledClass b : scheduledClassList) {
if (a.getTeacher().getTeacherId() > 17000L) {
hardScore+=18;
}
if (a.getTimeslot() != null && a.getTimeslot().equals(b.getTimeslot())
&& a.getId() < b.getId()) {
if (a.getTeacher() != null && a.getTeacher().equals(b.getTeacher())) {
hardScore--;
}
if (a.getTeacher().equals(b.getTeacher())) {
hardScore--;
}
} else {
hardScore++;
softScore+=2;
}
}
}
return HardSoftScore.of(hardScore, softScore);
}
}
So Please any idea why optaplanner might skip creating possible solutions?
The issue was simpler than I thought, The Solution class annotated with "PlanningSolution" has property "scheduledClasses" annotated with "PlanningEntityCollectionProperty" my mistake that this property was initialized with empty List (ArrayList), the solution was to initialize a solution class! In retrospect I think the documentation is to be blamed on this, the provided example didn't mention that we need to have this, so it should not be null (otherwise and exception will be raised) and it shouldn't be empty List. You need to initialize it with class without setting any value for the movable properties (annotated with "PlanningVariable").
Thanks for #Lukáš Petrovický as his comment helped me do the correct investigation!
Related
I have a logic that does something like this that I want to test out:
public void doSomething(int num) {
var list = service.method1(num);
if (!list.isEmpty()) {
// Flow 1
LOG.info("List exists for {}", num);
doAnotherThing(num);
} else {
// Flow 2
LOG.info("No list found for {}", num);
}
}
public void doAnotherThing(int num) {
Optional<Foo> optionalFoo = anotherService.get(num);
optionalFoo.ifPresentOrElse(
foo -> {
if (!foo.type().equals("no")) {
// Flow 3
anotherService.filter(foo.getFilter());
} else {
// Flow 4
LOG.info("Foo is type {} - skipping", foo.type());
}
},
// Flow 5
() -> LOG.info("No foo found for {} - skipping", num));
}
For each test that'll test out different flow, my first thought was to Mockito.verify() to see if they were called or not. So to test out Flow 1, I would verify that anotherService.get() was called inside doAnotherThing(). And to test out Flow 2, I would verify that anotherService.get() was never called. This would've been fine except for Flow 4 and Flow 5. They would both invoke anotherService.get() once but not anything else.
Because of that, I've created a class to capture logs in tests. This would check to see if certain logs were logged and I would be able to see by it which flow it landed on. But I wanted to ask: is this a bad practice? I would combine this with verify() so that on flows that can be reached by verify() will take that as higher precedence.
One downside to this would be that the tests would rely on the log messages being correct so it would be a bit unstable. To account for that issue, I thought about taking some of these log messages out as a protected static variable that the tests can also use so the message would remain the same between the methods and the respective tests. This way, only the flow would be tested.
If the answer is that it is a bad practice, I would appreciate any tips on how to test out Flow 4 and Flow 5.
Log statements are usually not part of the logic to test but just a tool for ops. They should be adjusted to optimize ops (not too much info, not too few), so that you can quickly find out if and where something went wrong, if something went wrong. The exact text, the log levels and the number of log statements should not be considered as something stable to rely your tests on. Otherwise it will make it harder to change the logging concept.
I'm trying to use Optaplanner to replace myself in scheduling our work planning.
The system is having a MySQL database containing the necessary information and relationships. For this issue I'll only use the three tables I need:
Employees --> Have Skills
Jobs --> Have Skills
Skills
In Drools I have the rule
rule 'required Skills'
when
Job(employee != null, missingSkillCount > 0, $missingSkillCount : missingSkillCount)
then
scoreHolder.addHardConstraintMatch(kcontext, -10 * $missingSkillCount);
end
In Class Job I have a function missingSkillCount():
public int getMissingSkillCount() {
if (this.employee == null) {
return 0;
}
int count = 0;
for (Skill skill : this.reqskills) {
if(!this.employee.getSkills().contains(skill)) {
count++;
}
}
return count;
}
When I run my program, Optaplanner returns that none of my workers have any skills...
However, when I manually use this function (adapted to accept an Employee as parameter): public int getMissingSkillCount(Employee employee), it does return the correct values.
I'm puzzled! I somehow understand that containsis checking for the same object, instead of the content of the object. But then I don't understand how to do this efficiently...
1) Are your Jobs in the Drools working memory? I presume they are your #PlanningEntity and the instances are in #PlanningEntityCollectionProperty on your #PlanningSolution, so they will be. You can verify this by just matching a rule on Job() and doing a System.out.println.
2) Try writing the constraint as a ConstraintStream (see docs) and putting a debug breakpoint in the getMissingSkillCount() > 0 lambda to see what's going on.
3) Temporarily turn on FULL_ASSERT to validate there is no score corruption.
4) Turn on DEBUG and then TRACE logging for optaplanner, to see what's going on inside.
Still wondering what makes the difference between letting Optaplanner run getMissingSkillCount() and using it "manually".
I fixed it by overriding equals(), that should have been my first clue!
Every once in a while, a server or database error causes thousands of the same stack trace in the server log files. It might be a different error/stacktrace today than a month ago. But it causes the log files to rotate completely, and I no longer have visibility into what happened before. (Alternately, I don't want to run out of disk space, which for reasons outside my control right now is limited--I'm addressing that issue separately). At any rate, I don't need thousands of copies of the same stack trace--just a dozen or so should be enough.
I would like it if I could have log4j/log4j2/another system automatically collapse repetitive errors, so that they don't fill up the log files. For example, a threshold of maybe 10 or 100 exceptions from the same place might trigger log4j to just start counting, and wait until they stop coming, then output a count of how many more times they appeared.
What pre-made solutions exist (a quick survey with links is best)? If this is something I should implement myself, what is a good pattern to start with and what should I watch out for?
Thanks!
Will the BurstFilter do what you want? If not, please create a Jira issue with the algorithm that would work for you and the Log4j team would be happy to consider it. Better yet, if you can provide a patch it would be much more likely to be incorporated.
Log4j's BurstFilter will certainly help prevent you filling your disks. Remember to configure it so that it applies in as limited a section of code as you can, or you'll filter out messages you might want to keep (that is, don't use it on your appender, but on a particular logger that you isolate in your code).
I wrote a simple utility class at one point that wrapped a logger and filtered based on n messages within a given Duration. I used instances of it around most of my warning and error logs to protect the off chance that I'd run into problems like you did. It worked pretty well for my situation, especially because it was easier to quickly adapt for different situations.
Something like:
...
public DurationThrottledLogger(Logger logger, Duration throttleDuration, int maxMessagesInPeriod) {
...
}
public void info(String msg) {
getMsgAddendumIfNotThrottled().ifPresent(addendum->logger.info(msg + addendum));
}
private synchronized Optional<String> getMsgAddendumIfNotThrottled() {
LocalDateTime now = LocalDateTime.now();
String msgAddendum;
if (throttleDuration.compareTo(Duration.between(lastInvocationTime, now)) <= 0) {
// last one was sent longer than throttleDuration ago - send it and reset everything
if (throttledInDurationCount == 0) {
msgAddendum = " [will throttle future msgs within throttle period]";
} else {
msgAddendum = String.format(" [previously throttled %d msgs received before %s]",
throttledInDurationCount, lastInvocationTime.plus(throttleDuration).format(formatter));
}
totalMessageCount++;
throttledInDurationCount = 0;
numMessagesSentInCurrentPeriod = 1;
lastInvocationTime = now;
return Optional.of(msgAddendum);
} else if (numMessagesSentInCurrentPeriod < maxMessagesInPeriod) {
msgAddendum = String.format(" [message %d of %d within throttle period]", numMessagesSentInCurrentPeriod + 1, maxMessagesInPeriod);
// within throttle period, but haven't sent max messages yet - send it
totalMessageCount++;
numMessagesSentInCurrentPeriod++;
return Optional.of(msgAddendum);
} else {
// throttle it
totalMessageCount++;
throttledInDurationCount++;
return emptyOptional;
}
}
I'm pulling this from an old version of the code, unfortunately, but the gist is there. I wrote a bunch of static factory methods that I mainly used because they let me write a single line of code to create one of these for that one log message:
} catch (IOException e) {
DurationThrottledLogger.error(logger, Duration.ofSeconds(1), "Received IO Exception. Exiting current reader loop iteration.", e);
}
This probably won't be as important in your case; for us, we were using a somewhat underpowered graylog instance that we could hose down fairly easily.
I am currently trying to get my grip on OptaPlanner as it seems to be the perfect solution for a problem I have.
Basically the Project job scheduling example is what I am going for, but as I only know my Java basics this is way to complex to start with. So I am trying to start with a very limited example and work my way up from there:
I have tasks with a duration and one defined predecessor.
The planning entity is the time each task starts.
I have a hard score that punishes tasks starting before starttime+duration of its predecessor. I also have a soft score that tries to reduce gaps, keeping the overall process as short as possible.
public HardSoftScore calculateScore(Schedule schedule) {
int hardScore = 0;
int softScore = 0;
for (Task task : schedule.getTaskList()) {
int endTime = task.getAllocation().getStartTime() + task.getDuration();
softScore = -endTime;
for (Task task2 : schedule.getTaskList()) {
if(task.getId()!=task2.getId()){
if( task2.getPredecessorId()==task.getId()) {
if (endTime > task2.getAllocation().getStartTime()) {
hardScore += task2.getAllocation().getStartTime() - endTime;
}
}
}
}
}
return HardSoftScore.valueOf(hardScore, softScore);
}
This is the solver config:
<?xml version="1.0" encoding="UTF-8"?>
<solver>
<!--<environmentMode>FAST_ASSERT</environmentMode>-->
<!-- Domain model configuration -->
<solutionClass>com.foo.scheduler.domain.Schedule</solutionClass>
<planningEntityClass>com.foo.scheduler.domain.Task</planningEntityClass>
<!-- Score configuration -->
<scoreDirectorFactory>
<scoreDefinitionType>HARD_SOFT</scoreDefinitionType>
<simpleScoreCalculatorClass>com.foo.scheduler.solver.score.SchedulingSimpleScoreCalculator</simpleScoreCalculatorClass>
</scoreDirectorFactory>
<!-- Optimization algorithms configuration -->
<termination>
<maximumSecondsSpend>100</maximumSecondsSpend>
</termination>
<constructionHeuristic>
<constructionHeuristicType>FIRST_FIT</constructionHeuristicType>
</constructionHeuristic>
<localSearch>
<acceptor>
<entityTabuSize>7</entityTabuSize>
</acceptor>
<forager>
<acceptedCountLimit>1000</acceptedCountLimit>
</forager>
</localSearch>
</solver>
The problem is that this works great as long as I only have the hard score. But of course it has gaps. As soon as I add the soft score everything gets stuck after about 10 steps. Why?
[...]
2014-05-03 20:01:31,966 [main] DEBUG Step index (10), time spend (495), score (-35hard/-66soft), best score (-34hard/-68soft), accepted/selected move count (1000/19884) for picked step (com.foo.scheduler.domain.Task#35480096 => com.foo.scheduler.domain.Allocation#f9a4520).
2014-05-03 20:03:11,471 [main] DEBUG Step index (11), time spend (100000), score (-35hard/-65soft), best score (-34hard/-68soft), accepted/selected move count (0/105934687) for picked step (com.foo.scheduler.domain.Task#7050c91f => com.foo.scheduler.domain.Allocation#47c44bd4).
A selected move count of 105934687 at step 11 clearly indicates that no move is being accepted. I don't see how a soft score could trigger that though. There is only 1 explanation:
The EntityTabuAcceptor doesn't accept any move, because they are all tabu, which means every move's planning entities are in the tabu list. This is possible if you have very small dataset (14 or less planning entities). Turn on TRACE logging and the log will confirm this.
Each of these workaround should fix that:
Use Late Acceptance
<acceptor>
<lateAcceptanceSize>400</lateAcceptanceSize>
</acceptor>
<forager>
<acceptedCountLimit>1</acceptedCountLimit>
</forager>
Use <entityTabuRatio> instead of <entityTabuSize>
Mess with the <entityTabuSize> based on the dataset size with SolverFactory.getSolverConfig(). Not recommended!
Why less than 14 planning entities?
Because by default you get a <changeMoveSelector> and a <swapMoveSelector>. The <swapMoveSelector> swaps 2 entities, making both tabu if it wins a step. The tabu list size is the number of steps, so if 7 swap moves win the steps in a row, there can be 14 entities in the tabu list.
I have multiple Jasper Reports (with sub-reports) throughout my application. For some reason, one report (that also contains sub-reports) isn't working anymore. After debugging more than 1 day, I found out that it enters an infinite loop and keeps creating Threads for sub-report filling.
Debugger keeps looping between:
JRSubReportRunnable.java
public void run()
{
running = true;
error = null;
try
{
fillSubreport.fillSubreport();
}
catch (JRFillInterruptedException e)
{
//If the subreport filler was interrupted, we should remain silent
}
// we have to catch Throwable, because it is difficult to say what would happen with the master
// filler thread in case we don't
catch (Throwable t) //NOPMD
{
error = t;
}
running = false;
}
The above method starts a Thread in order to fill a sub-report. Once done, sets running = false and the debugger gets to:
JRThreadSubreportRunner.java
public void run()
{
super.run();
if (log.isDebugEnabled())
{
log.debug("Fill " + subreportFiller.fillerId + ": notifying of completion");
}
synchronized (subreportFiller)
{
//main filler notified that the subreport has finished
subreportFiller.notifyAll();
}
}
Once the thread finishes, it gets to the above's method subreportFiller.notifyAll(); line. Then, the debugger goes back to JRSubreportRunnable.java, and so on.
Theoretically, if I have 5 sub-reports, it should create 5 threads (works for me for other reports). Unfortunately, for this case, it keeps creating threads, and my debugger gets "stuck" between the 2 methods mentioned above (FYI: the classes are from the jasperreports-3.7.6-sources.jar).
Also tried:
I found a similar StackOverflow question, but the answer proposed there did not work for me. Neither did any of the proposed solutions from this thread on the JasperSoft Community.
I really cannot figure why this issue appears. I am sure it is something minor as it used to work. Hopefully someone else stumbled upon this and might have a solution. Thanks in advance for any answer. (I know I haven't provided really much info about the content of my sub-reports, but it is pretty private; nevertheless, I can assure you that the contents of the report and associated sub-reports did not change - checked with Git)
I had the exact same problem and solved it by changing the isPrintWhenDetailOverflows property of my subreport from true to false
as suggested here:
http://community.jaspersoft.com/questions/527078/infinite-loop-subreport-fill
hope it helps