Im begineer in this concept and what I have tried to learn for a feed-forward type neural network(topology of 2x2x1 ):
Bias and weight range of each neuron_____________Outputs for XOR test inputs
[-1,1] 1,1 ----> 0,9
1,0 ----> 0,8
0,1 ---->-0.1
0,0 ----> 0.1
[-10,10] 1,1 ----> 0,24
1,0 ----> 0,67
0,1 ---->-0.54
0,0 ----> 0.10
[-4,4] 1,1 ----> -0,02
1,0 ----> 0,80
0,1 ----> 0.87
0,0 ----> -0.09
So, the range of [-4,4] seems to be better than other.
Question: Is there a way to find the proper limits of weigths and biases compared to temperature limits and temperature decrease rate?
Note: Im trying two ways here. First is randomizing all weights and biases at once for each trial. Second is randomizing only single weight and a single bias at each trial. (50 iterations before decreasing temperature). Single weight change gives worse results.
(n+1) is next value, (n) is the value before
TempMax=2.0
TempMin=0.1 ----->approaching to zero, error of XOR output approaches to zero too
Temp(n+1)=Temp(n)/1.001
Weight update:
w(n+1)=w(n)+(float)(Math.random()*t*2.0f-t*1.0f)); // t is temperature
(same for bias update)
Iterations per temperature=50
Using java's Math.random() method(Spectral property is appropriate for annealing?)
Transition probability:
(1.0f/(1.0f+Math.exp(((candidate state error)-(old error))/temp)))
Neuron activation function: Math.tanh()
Tried many times and results are nearly same. Is reannealing the only solution to evade deeper local minimums?
I need a suitable weight/bias range/limit according to total neuron number and layer number and starting/enging temperature. 3x6x5x6x1 can count 3-bit input and gives outpu, can approximate step function, but I need to play with ranges always.
For this training data set, output error is too big(193 data points, 2 inputs, 1 output):
193 2 1
0.499995 0.653846
1
0.544418 0.481604
1
0.620200 0.320118
1
0.595191 0.404816
0
0.404809 0.595184
1
0.171310 0.636142
0
0.014323 0.403392
0
0.617884 0.476556
0
0.391548 0.478424
1
0.455912 0.721618
0
0.615385 0.500005
0
0.268835 0.268827
0
0.812761 0.187243
0
0.076923 0.499997
1
0.769231 0.500006
0
0.650862 0.864223
0
0.799812 0.299678
1
0.328106 0.614848
0
0.591985 0.722088
0
0.692308 0.500005
1
0.899757 0.334418
0
0.484058 0.419839
1
0.200188 0.700322
0
0.863769 0.256940
0
0.384615 0.499995
1
0.457562 0.508439
0
0.515942 0.580161
0
0.844219 0.431535
1
0.456027 0.529379
0
0.235571 0.104252
0
0.260149 0.400644
1
0.500003 0.423077
1
0.544088 0.278382
1
0.597716 0.540480
0
0.562549 0.651021
1
0.574101 0.127491
1
0.545953 0.731052
0
0.649585 0.350424
1
0.607934 0.427886
0
0.499995 0.807692
1
0.437451 0.348979
0
0.382116 0.523444
1
1 0.500000
1
0.731165 0.731173
1
0.500002 0.038462
0
0.683896 0.536585
1
0.910232 0.581604
0
0.499998 0.961538
1
0.903742 0.769772
1
0.543973 0.470621
1
0.593481 0.639914
1
0.240659 0.448408
1
0.425899 0.872509
0
0 0.500000
0
0.500006 0.269231
1
0.155781 0.568465
0
0.096258 0.230228
0
0.583945 0.556095
0
0.550746 0.575954
0
0.680302 0.935290
1
0.693329 0.461550
1
0.500005 0.192308
0
0.230769 0.499994
1
0.721691 0.831791
0
0.621423 0.793156
1
0.735853 0.342415
0
0.402284 0.459520
1
0.589105 0.052045
0
0.189081 0.371208
0
0.533114 0.579952
0
0.251594 0.871762
1
0.764429 0.895748
1
0.499994 0.730769
0
0.415362 0.704317
0
0.422537 0.615923
1
0.337064 0.743842
1
0.560960 0.806496
1
0.810919 0.628792
1
0.319698 0.064710
0
0.757622 0.393295
0
0.577463 0.384077
0
0.349138 0.135777
1
0.165214 0.433402
0
0.241631 0.758362
0
0.118012 0.341772
1
0.514072 0.429271
1
0.676772 0.676781
0
0.294328 0.807801
0
0.153846 0.499995
0
0.500005 0.346154
0
0.307692 0.499995
0
0.615487 0.452168
0
0.466886 0.420048
1
0.440905 0.797064
1
0.485928 0.570729
0
0.470919 0.646174
1
0.224179 0.315696
0
0.439040 0.193504
0
0.408015 0.277912
1
0.316104 0.463415
0
0.278309 0.168209
1
0.214440 0.214435
1
0.089768 0.418396
1
0.678953 0.767832
1
0.080336 0.583473
1
0.363783 0.296127
1
0.474240 0.562183
0
0.313445 0.577267
0
0.416055 0.443905
1
0.529081 0.353826
0
0.953056 0.687662
1
0.534725 0.448035
1
0.469053 0.344394
0
0.759341 0.551592
0
0.705672 0.192199
1
0.385925 0.775385
1
0.590978 0.957385
1
0.406519 0.360086
0
0.409022 0.042615
0
0.264147 0.657585
1
0.758369 0.241638
1
0.622380 0.622388
1
0.321047 0.232168
0
0.739851 0.599356
0
0.555199 0.366750
0
0.608452 0.521576
0
0.352098 0.401168
0
0.530947 0.655606
1
0.160045 0.160044
0
0.455582 0.518396
0
0.881988 0.658228
0
0.643511 0.153547
1
0.499997 0.576923
0
0.575968 0.881942
0
0.923077 0.500003
0
0.449254 0.424046
1
0.839782 0.727039
0
0.647902 0.598832
1
0.444801 0.633250
1
0.392066 0.572114
1
0.242378 0.606705
1
0.136231 0.743060
1
0.711862 0.641568
0
0.834786 0.566598
1
0.846154 0.500005
1
0.538462 0.500002
1
0.379800 0.679882
0
0.584638 0.295683
1
0.459204 0.540793
0
0.331216 0.430082
0
0.672945 0.082478
0
0.671894 0.385152
1
0.046944 0.312338
0
0.499995 0.884615
0
0.542438 0.491561
1
0.540796 0.459207
1
0.828690 0.363858
1
0.785560 0.785565
0
0.686555 0.422733
1
0.231226 0.553456
1
0.465275 0.551965
0
0.378577 0.206844
0
0.567988 0.567994
0
0.668784 0.569918
1
0.384513 0.547832
1
0.288138 0.358432
1
0.432012 0.432006
1
0.424032 0.118058
1
0.296023 0.703969
1
0.525760 0.437817
1
0.748406 0.128238
0
0.775821 0.684304
1
0.919664 0.416527
0
0.327055 0.917522
1
0.985677 0.596608
1
0.356489 0.846453
0
0.500005 0.115385
1
0.377620 0.377612
0
0.559095 0.202936
0
0.410895 0.947955
1
0.187239 0.812757
1
0.768774 0.446544
0
0.614075 0.224615
0
0.350415 0.649576
0
0.160218 0.272961
1
0.454047 0.268948
1
0.306671 0.538450
0
0.323228 0.323219
1
0.839955 0.839956
1
0.636217 0.703873
0
0.703977 0.296031
0
0.662936 0.256158
0
0.100243 0.665582
1
I highly doubt that any strict rules exist for your problem. First of all, limits/bounds of weights are strictly dependant on your input data representation, activation functions, neurons number and output function. what you can rely on here are rules of the thumb in the best possible scenario.
First, lets consider the initial weights values in classical algorithms. Some basic idea of the weights scale are to use them in the range of [-1,1] for small layers, and for large ones divide it by the square root of the number of units in the large layer. More sophisticated methods are described by Bishop (1995). With such rule of the thumb we could deduce, that a resonable range (which is simply row of magniture bigger then the initial guess) would be something in the form of [-10,10]/sqrt(neurons_count_in_the_lower_layer).
Unfortunately, to my best knowledge, temperature choice is much more complex, as it is rather a data dependant factor, not just topology based one. In some papers there have been suggestions for some values for some specific time series prediction, but nothing general. In simmulated annleaing "in general" (not just applied to NN training), there have been proposed many heuristic choices, ie.
If we know the maximum distance (cost function difference) between one
neighbour and another then we can use this information to calculate a
starting temperature. Another method, suggested in (13. Rayward-Smith, V.J., Osman, I.H., Reeves, C.R., Smith, G.D. 1996. Modern Heuristic Search Methods. John Wiley & Sons.), is to start with a very high temperature and cool it rapidly
until about 60% of worst solutions are being accepted. This forms the
real starting temperature and it can now be cooled more slowly. A
similar idea, suggested in (5. Dowsland, K.A. 1995. Simulated Annealing. In Modern Heuristic Techniques for Combinatorial Problems (ed. Reeves, C.R.), McGraw-Hill, 1995), is to rapidly heat the
system until a certain proportion of worse solutions are accepted and
then slow cooling can start. This can be seen to be similar to how
physical annealing works in that the material is heated until it is
liquid and then cooling begins (i.e. once the material is a liquid it
is pointless carrying on heating it). [from notes from University of Nottingham]
But the choice of the best for your application has to be based on numerous tests, as most of the things in the machine learning. If you are dealing with the problem, where you are really concerned about well trained neural network, it seems resonable to interest in Extreme Machine Learning, and Extreme Learning Machines (ELM), where the neural network training is conducted in the global optimization procedure, which guarantees the best possible solution (under used regularized cost function). Simulated annleaing, as a interative, greedy process (as well as back propagation) cannot guarantee anything, there are only heuristics and rules of thumb.
Related
How to calculate grade wise counts by semester and grade range.
I am having tables, fields and data as follows:
SEMETSER
SEMISTER_NO
NAME
1
FIRST
2
SECOND
STUDENT_SEMETSER
STUDENT_NO
SEMESTER_NO
SCORE
1
1
9.8
2
1
9.2
3
1
8.8
4
1
7.8
5
1
7.5
6
1
7.2
7
2
8.8
8
2
8.2
9
2
8.8
10
2
6.8
11
2
6.5
12
2
6.2
RESULTS Expected:
SEMESTER_NO
SCORE 10-9
SCORE 9-8
8-7
7-6
6-5
5-4
1
2
1
3
0
0
0
2
0
3
0
2
0
0
Using CASE-WHEN and PIVOT is one of the options:
SELECT *
FROM
(
SELECT semester_no,
CASE
WHEN SCORE>= 9 and SCORE<=10 THEN 'Score 10-9'
WHEN SCORE>= 8 and SCORE<9 THEN 'Score 9-8'
WHEN SCORE>= 7 and SCORE<8 THEN 'Score 8-7'
WHEN SCORE>= 6 and SCORE<7 THEN 'Score 7-6'
WHEN SCORE>= 5 and SCORE<6 THEN 'Score 6-5'
WHEN SCORE>= 4 and SCORE<5 THEN 'Score 5-4'
END AS score_cat
FROM student_semester
)
PIVOT
(
COUNT(score_cat)
FOR score_cat IN ('Score 10-9','Score 9-8','Score 8-7','Score 7-6','Score 6-5','Score 5-4')
)
ORDER BY semester_no
The CASE statement will categorize the score into different buckets (Modify as per your need) and PIVOT statement will transpose the data into desired level.
DEMO
I have to write a program but I have no idea where to start. Can anyone help me with an outline of how I should go about it? please excuse my novice level at programming. I have provided the input and output of the program.
The trouble that I'm facing is how do I handle the input text? How should I store the input text to extract the data that I need to produce the output commands? Any guidance would be so helpful.
A little explanation of the input:
The output will start with APPLE1: CT= (whatever number is there for CT in line 4)
The following lines of the output will begin with "APPLES:"
I must include and extract the values for CR, PLANTING and RW in the output.
Wherever there is a non-zero or not null in the DATA portion, it will appear in the output.
When the program reads END, "APP;APPLER:CT=(whatever number);" will be the last two commands
INPUT:
<apple:ct=12;
FARM DATA
INPUT DATA
CT CH CR PLANTING RW DATA
12 YES PG -0 FA=1 R=CODE1 MM2 COA COB CI COC COD
0 0 1 0
COE RN COF COG COH
4 00 0
COI COJ D
0
FA=2 R=CODE2 112 COA COB CI COC COD
0 0 0 0
COE RN COF COG COH
4 00 0
COI COJ D
7
END
OUPUT:
APPLE1:CT=12;
APPLES:CR=PG-0,FA=1,R=CODE1,RW=MM2,COC=1,COE=4;
APPLES:FA=2,R=CODE2,RW=112,COE=4,COI=7;
APP;
APPLER:CT=12;
This question already has answers here:
Using the non final loop variable inside a lambda expression
(3 answers)
Closed 3 years ago.
I have a list of ints like the following one:
1318 1065 0
1392 1109 0
1522 1114 2
1764 1134 0
1643 1172 0
1611 1141 0
1608 1142 4
1689 1180 0
1546 1144 0
1811 1121 1
1682 1144 0
1687 1203 0
1751 1138 0
1702 1227 0
My goal is to keep only entries where the third element is a 0.
I've tried a few things like:
for( int i=0; i<data_fin.size();i++) {
data_fin.removeIf(s -> !((data_fin.get(i)[2]) == 0));
}
I get the error 'Local variable i defined in an enclosing scope must be final or effectively final'.
Can anyone help me understand what I'm doing wrong? I'm quite new to Java and I'm probably using removeif wrong, so I would appreciate the help!
My goal is to keep only entries where the third element is a 0
You don't need a loop in this, all you need is :
data_fin.removeIf(a -> a[2] != 0);
I am new to regx, but seems to be the only way I can solve the problem i have.
I was able to extract a text file from a CISCO switch using show vlan. The file looks like below. I just want to extact the vlan number and Name on array of strings. Where to begin.
VLAN Name Status Ports
---- -------------------------------- --------- -------------------------------
1 default active Gi0/1, Gi0/2, Gi0/4, Gi0/5
Gi0/6, Gi0/7, Gi0/8, Gi0/9
Gi0/10, Gi0/11, Gi0/12, Gi0/13
Gi0/14, Gi0/15, Gi0/16, Gi0/17
Gi0/18, Gi0/19, Gi0/20, Gi0/21
Gi0/22, Gi0/23, Gi0/24, Gi0/25
Gi0/26, Gi0/27, Gi0/28, Gi0/29
Gi0/30, Gi0/31, Gi0/32, Gi0/33
Gi0/34, Gi0/35, Gi0/37, Gi0/38
Gi0/39, Gi0/40, Gi0/41, Gi0/42
Gi0/43, Gi0/44, Gi0/45, Gi0/46
Gi0/47, Gi0/48, Gi0/49, Gi0/50
Gi0/51, Gi0/52
2 Test active Gi0/3
3 Internet active Gi0/36
1002 fddi-default act/unsup
1003 token-ring-default act/unsup
1004 fddinet-default act/unsup
1005 trnet-default act/unsup
VLAN Type SAID MTU Parent RingNo BridgeNo Stp BrdgMode Trans1 Trans2
---- ----- ---------- ----- ------ ------ -------- ---- -------- ------ ------
1 enet 100001 1500 - - - - - 0 0
2 enet 100002 1500 - - - - - 0 0
3 enet 100003 1500 - - - - - 0 0
1002 fddi 101002 1500 - - - - - 0 0
1003 tr 101003 1500 - - - - - 0 0
1004 fdnet 101004 1500 - - - ieee - 0 0
1005 trnet 101005 1500 - - - ibm - 0 0
Remote SPAN VLANs
------------------------------------------------------------------------------
Primary Secondary Type Ports
------- --------- ----------------- ------------------------------------------
^(\d+)\s+([^ ]+)
And then you will match VLAN number and type OR VLAN numer and name.
The explanation is in the first comment below this question.
The only thing I've changed is that the name or type may contain any characters except a blank space.
I think Java doesn't provide much in their API about getting processes, is there a way that you could get a parent's process PID/ID in Java?
If you're running on Linux you can check procfs using /proc/self/stat.
Following from #Imz's answer, on Linux, grab the output of /proc/self/stat (it's a one-line file, so just read it like a normal file)
43732 (java) S 43725 43725 11210 34822 43725 4202496 127791 387073
4055 0 3188 79 4597 253 20 0 53 0 16217706 39231705088 188764
18446744073709551615 4194304 4196452 140735605394256 140735605376816
274479481597 0 0 2 16800973 18446744073709551615 0 0 17 13 0 0 0 0 0
The 4th field (in bold above) is your parent process id