Reproducing jittering results

Forums

RSS (Opens New Window)

Alessandro Orio, modified 4 Years ago.

Reproducing jittering results

Youngling Posts: 20 Join Date: 5/13/19 Recent Posts

Hi,

I have tried to use the jittering on my assessment and I have found some runs with a lower LL compared to my base model.

I have then tried to replicate one of those runs by changing all the initial values in my control file with the ones reported in the report but I don't manage to get the same results. I have also tried to change all the initial values with the final ones but again nothing seem to change.

Am I approaching this in the wrong way? I really can't understand why I can't replicate those results.

Hope someone can help me understanding this a bit more.

Cheers,

Ale

Kelli Faye Johnson, modified 4 Years ago.

RE: Reproducing jittering results

Youngling Posts: 28 Join Date: 6/13/17 Recent Posts

The jitter runs are produced using starting values from the .par file. You should be able to rename the .par file from the jitter run that you want to reproduce as ss.par and run to reproduce the model results with the lower likelihood value. The starter file will dictate whether SS is reading input parameter values from the .par or the control file. Also, it could be that some of the jitter runs are not stable even though they lead to a lower likelihood value. You will want to check to make sure that parameters are not going to bounds and that the model produces a positive definite hessian.

Santiago Cervino, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 11 Join Date: 3/26/21 Recent Posts

HI, I'm trying to do something similar than what Alexandro asked. I'd like to use the jitter outputs to continue searching for a new better result. I've save the jitter par file as ss.par and I modified the starter to use this starter pareams instead of the control file. If I run >ss -stopph 0 -nohess; then the model got the same likelihood (2340). However if I run >ss; then the model start the searching in a huge likelihood (650469). I expected ss should start in 2340 and try to find something from there. Am I misundertanding something? Thanks!

Richard Methot, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 219 Join Date: 11/24/14 Recent Posts

Santiago,

It is difficult to be sure about what is happening in your model run without seeing more information about the setup. The only clue I can offer is with regard to the F method. If you are using FMethod=2 and not starting the F estimation until a phase later than phase 1, then SS3 will use hybrid F approach in phase 1. I have seen this cause a difference in some situations.

Rick

Santiago Cervino, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 11 Join Date: 3/26/21 Recent Posts

Hi Rick. Not, my F method is 4. What I'd like to do is getting a jitter model that got to a better results than the case base and make it as the new case base. Is there any other option to do it?

Santi.

Ian Taylor, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 117 Join Date: 12/8/14 Recent Posts

Hi Santiago,

I would try to turn on the ParmTrace option using this setting in the starter file:

0 # write parm values to ParmTrace.sso (0=no,1=good,active; 2=good,all; 3=every_iter,all_parms; 4=every,active)
(I personally like option 3 to get all parameter from every iteration).

The ParmTrace.sso file will help you see the parameters and likelihood associated with each step in the jitter and may again help you when you run the optimization.

-Ian

Santiago Cervino, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 11 Join Date: 3/26/21 Recent Posts

HI Ian. thats a gret trick. However what I'd like to do is to get one of the jitters, that is better than my base case and use it to run a new model. I rename the jitter param file as ss.par and modified the starter to use it as initial parameters. However when I run a optimization model with this configuration the model do not start in this space of parameters and it finish in a worst likelihood.

Ian Taylor, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 117 Join Date: 12/8/14 Recent Posts

I wasn't sure which of the jitters you were wishing to use. I was thinking that the ss.par file in the jitter folder only represents the final jittered model, but I now remember that the r4ss::SS_RunJitter() function saves the par file from each jitter, so you can pick and choose. Instead of using the ParmTrace.sso file to diagnose the difference between the ss -stopph 0 -nohess and the optimization, you might try comparing the Report files created by ss -stopph 0 -nohess with one created by ss -phase 99 -nohess -maxfn 0 which will start the model in the final phase but not do any estimation.

Santiago Cervino, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 11 Join Date: 3/26/21 Recent Posts

Hi, Problem solved. I had turned the line to 1 in the starter file to read the ss.par file but I did not realise that I had the jitter option set to 0.1, that was causing starting in a different parameters space. Now it was set to 0 and the optimization is starting in the true likelihood. Begginer's mistake, sorry!. SS has too many options ...

Santiago Cervino, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 11 Join Date: 3/26/21 Recent Posts

Hi @ian.taylor ,

the model I'm working with is quite instable. I'd like to get the hessian from the jitter run I mentioned. Whe I run ss -stopph 0 surprisingly I got a model where hessian is not positive definite. However, when I run ss -phase 99 -maxfn 0 the model provides the same hessian than the original one. Any idea about why this two approaches provides quite different results?

Richard Methot, modified 2 Years ago.

RE: Reproducing jittering results

Youngling Posts: 219 Join Date: 11/24/14 Recent Posts

With -stopph 0, no parameters are being estimated so it is impossible to get a hessian calculated. This SS3 feature is designed to simply calculate and output the model for the current set of parameters.

With the other approach, all parameters designated to be active in the control file are immediately active, so their variance can be calculated.