SS user manual

RE: Discussion regarding how Catch likelihood is calculated and the impact of using different Fmethods when Catch CVs are elevated

2022-09-01T18:58:43Z

Aloha Rick,

Meg, Felipe, and I are currently working on SS models for American Samoa bottomfishes. We've been playing around using both Fmethod 2 (Baranov) and Fmethod 3 (hybrid) and running R0 profiles, among other things.

We've noticed that the Catch component appears in the R0 likelihood profile when using F2, which makes sense since under this SS setup, catches can be adjusted based on their CV (so we have expected and observed catches, with an error distribution, allowing SS to calculate likelihood).

However, we've also noticed the Catch component appearing in some R0 profiles when using F3. This brought us to wonder how the Catch likelihood is calculated in SS under this approach? Isn't Catch fixed and therefore there is no observed vs expected Catch to calculate likelihood on? Also, we are not using a catch multiplier.

We looked at recent and old SS manuals but couldn't find a reference for how Catch likelihood is calculated in SS. Is there a reference available?

Hopefully this makes sense.

--------------------------------------------------

Rick's reply:

Hi Marc et al.

This is sort of a perennial question so I clearly need to make it more clear in the output and manual. Copying Chantel to help in that.

Catch logL is calculated the same way for method 2 and 3 and by the same code. The only difference is in how the F that calculates the expected catch is calculated.

With method 2, F is a parameter ( or set of parameters for all the fleets). SS3 uses that parameter(s) to go straight to calculating expected catch for each fleet, then to the logL.

Method 2 includes option to do method 3 in early phases and then transition to method 2 to finish (which can be faster in high F, many fleet situations)

With method 3, SS3 does a Pope's (method 1) to get each F in right ballpark and then adjusts it through a few iterations to get closer to matching the observed catch. It then finishes through the method 2 code to calculate the final iteration of expected catch and then logL. At high F and/or with many fleets it may not get to an exact match to the observed catch.

New'ish method 4 provides more flexibility in that some fleets can do hybrid and other fleets (particularly high F fleets) can do parameters starting at a fleet-specific phase. Method 4 is clear winner in my mind.

----------------------------------------------------

Marc et al.'s reply:

Aloha Rick,

If I understand correctly, method 2 focuses on estimating F as independent parameters first. I imagine these estimates are heavily informed by size structure data and CPUE trends (and the catch?). It then simply uses these F estimates to calculate expected Catch, as you said. On the other hand, method 3 is more focused on estimating the F values that match the observed catch. Is this roughly correct?

This would match our observation that catch is very minimally adjusted when using Method 3 vs. Method 2, for certain species.

For method 4, if we only have a single fleet, would there be any difference here? I thought the only advantage to this method is that you can assign F methods to different fleets, correct?

------------------------------------------------------

Rick's reply:

Correct on your last point regarding method 4, but someday I might deprecate methods 2 and 3 which really are just more restrictive versions of method 4. All use the same code.

Your first point regarding other data is not really an issue. Method 2 uses parameters and method 3 uses internal coefficients. Both have one parameter/coefficient for each catch observation. Further, method 2 can start with coefficients in early phases while temporarily using method 3, then during BETWEEN_PHASES copy the coefficients into parameters and proceed. I have run tests showing that whether you do method 2 with hundreds of F parameters or method 3 with the same number of coefficients has nil impact on the estimated variance of other model derived quantities, like ending year spawning biomass.

We have found one situation in which method 2 vs. 3 matters: when there is discard. This has come up in SEFSC assessments.

The F (whether parameter or coeff) produces a total catch, then the retention fxn splits it into retained and discarded catch, then the logL for catch compares the retained catch to the observed retained catch, and the logL for discard compares the estimated discard catch to the observed discard.

So with method 3, the coeff adjustment algorithm tunes F to match the retained catch, regardless of the catch se value. The resultant F may not match the discard well and that could result in the model changing other parameters (recruitment, selectivity, etc. ) in order to also fit the discard reasonably well.

However with method 2 the F as parameter will initially not fit either catch or discard well in early phases and the ADMB gradient algorithms slowly will adjust the F to get a better fit to both retained and discarded catch; with the catch se relative to the discard se influencing the relative fit. In other words, if you have discard and use method 2, you should not expect an exact fit to the catches unless the have a very tight se.

The other difference between method 2 and method 3 is model speed. It is a tortoise and hare situation.

method 2 is the tortoise: F starts off being far off and gradually gets better during the model run, but each model iteration is quick because there are no fancy hybrid loops embedded in each ADMB iteration

method 3 is the hare and uses those several internal hybrid loops to get the F coefficients very close beginning with the first ADMB iteration. So each ADMB iteration is slightly slower.

At low F situations, method 3 seems to get to final solution faster, but at high F situations the internal dynamics get very nonlinear as the model constantly adjusts the F coefficients to match the observed catch. Consequently each ADMB iteration tends to only move all the non-F parameters only slowly towards the solution and method 3 loses the race because it needs more ADMB iterations.

Hence method 4 which makes it easy to use hybrid in early phases to get good starting values for F parameters; then polish the run with F as parameters.

--------------------------------------------------------

Marc et al.'s reply:

Thank you for those extra details, very interesting to hear that Fmethod 2 vs 3 can generate a difference in that special edge case you described. More specifically, you mentionned "The resultant F may not match the discard well and that could result in the model changing other parameters (recruitment, selectivity, etc. ) in order to also fit the discard reasonably well. ".

We think this can also happen in a situation where the Catch data have very high CVs and there is some data conflict with length data. We are running very simple models (one fleet, catch data back to 1967 with CVs between 20-50%, size data from 2006-2021, CPUE data from 2016-2021, no rec devs). We find that when there is a conflict between the length and catch data, the SS models with Fmethod 2 will sometimes aggressively adjust the Catch and F estimates while models under Fmethod3 won't adjust catch at all (it seems), but play with other parameters (i.e. selectivity, leading to worst size data fits).

I have put together the attached example comparing model outputs for 4 scenarios: Fmethod 3 with Catch CVs from 10%-50% (i.e. our original estimated CVs), Fmethod 2 with Catch CV=10%, Fmethod2 with Catch CV=15%, and Fmethod2 witch CVs from 10% to 50%. You can see in the attached word doc how Fmethod 3 doesn't adjust Catch much in the model despite the high CVs, while Fmethod2 adjust catch more and more aggressively with higher CVs. You can also see how the SS model under Fmethod3 tries to resolve the conflict by playing with the selectivity parameters (since it's not adjusting the catch), as in your example above.

Note that this is a mild example of catch adjustment. We have worked on this model to reduce the catch vs length conflict by adjusting the growth parameters. The catch adjustments used to be much greater originally (the catch estimates in the mid-1980s would be almost doubled).

Please let us know what you think and what you would advise us to stick with.

--------------------------------------------------------

Rick's reply:

I am sure you are correct about the case with high catch CV. With Fmethod 2, catch is just data like any other data and data with high CV will not be matched exactly. I have seen the same thing with the TMB model SAM.

--------------------------------------------------------

Marc et al.'s reply:

Ok, thanks Rick. I think this all makes sense now.

To recap:

1) Both Fmethods generate an expected catch and that's where the likelihood component for "Catch" comes from. It's simply hard to see the difference between Exp. and Obs. catches when catch CVs are low or when Fmethod3 is used.

2) For the vast majority of models, which have low-to-moderate Catch CVs, using either methods generate the same F and Catch estimates (and it is recommended to use Fmethod 4 nowadays).

3) With very high Catch CVs and Fmethod 2, there can be large Catch adjustments if it is conflicting with other data types, since this data is treated like any other data types in the model.

I hope I captured this correctly.

Again, thanks Rick, your help is very much appreciated!

-------------------------------------------------------

Rick's reply:

Perfect! Now wishing that we had had this conversation on the VLAB forum for all to benefit from.

--------------------------------------------------------

End thread.

Discussion regarding how Catch likelihood is calculated and the impact of using different Fmethods when Catch CVs are elevated

2022-09-01T18:51:23Z

Rick suggested we post this email discussion here, to share. It's an issue that comes up often and it may be relevant to updating the SS manual with a better description of how Catch likelihood is calculated and the impact of using Fmethod 2 vs. 3 under high Catch CVs. Please see below

RE: CPUE plot

2020-05-29T17:38:58Z

Thanks Yevhen Leonchyk for pointing out the lack of information on the standardization approach for that figure. And thanks Ian S. and Kathryn for the explanations.

I just expanded the caption slightly so it now reads "Standardized indices overlaid. Each index is rescaled to have mean observation = 1.0."

The revised caption is in the development branch of r4ss available at

remotes::install_github('r4ss/r4ss', ref = 'development')

and will be merged into the main branch eventually.

I should also note that Allan Hicks made good progress a few years ago on a revised version of this figure which adds uncertainty intervals and (optionally) model expectations as shown at https://github.com/r4ss/r4ss/issues/67. This conversation can serve as inspiration to get back to that effort as time allows. Feel free to make further suggestions on that github issue or here on VLab.

-Ian

On Fri, May 29, 2020 at 10:18 AM Yevhen Leonchyk <VLab.Notifications@noaa.gov> wrote:

I use z-scores for scaling in Cluster analysis. But the mean-centered method is "one-size-fits-all".
Thank you for the explanation.

--
Yevhen Leonchyk Stock Synthesis Virtual Lab Forum https://vlab.noaa.gov/web/stock-synthesis/public-forums/-/message_boards/view_message/10298233 VLab.Notifications@noaa.gov

RE: CPUE plot

2020-05-29T17:18:26Z

I use z-scores for scaling in Cluster analysis. But the mean-centered method is "one-size-fits-all".
Thank you for the explanation.

RE: CPUE plot

2020-05-29T16:05:35Z

Hi Yevhen,

Sorry, I missed your response about z-scoring indices instead of dividing by the mean to standardize. I can see why that would be useful. Is there any particular reason that plot would be more useful than the current one (or in addition to the current one)? r4ss welcomes contributions and it is always possible to add/change plots depending on the needs of the r4ss users.

Thanks,

Kathryn

RE: CPUE plot

2020-05-29T16:02:52Z

Hi Yevhen,

The initial intention of for this figure was to be able to identify consistent trends across indices on a single figure, hence the scaling around 1.0 . This allows indices that have very different scales in relative abundance to be easily comparable. The alternative calculation you suggested could be a useful additional figure for r4ss to include that could be complimentary to the current figure.

RE: CPUE plot

2020-05-29T16:02:04Z

Dear Kathryn,

thank you for the links!

RE: CPUE plot

2020-05-29T15:46:40Z

Hi Yevhen,

To follow up on Ian's response, the place in the r4ss development branch code this standardization occurs: https://github.com/r4ss/r4ss/blob/development/R/SSplotIndices.R#L564

And most of the code for making this plot is also in the same function: https://github.com/r4ss/r4ss/blob/development/R/SSplotIndices.R#L779-L811

Thanks,

Kathryn

RE: CPUE plot

2020-05-29T15:45:34Z

Thank you for the reply.

How about such method

(Index-mean(Index))/sd(Index),

sd – standard deviation.

Might it be better for a wide variation case? But it seems to be "bad" when SD is close to zero.

RE: CPUE plot

2020-05-29T15:39:12Z

Hello Yevhen,

Yes, this plot just uses the simple mean of the series to standardize each index and is created in the r4ss code rather than in SS directly.

RE: CPUE plot

2020-05-29T15:20:21Z

It seems to me that ss3 uses mean value, doesn't it?

CPUE plot

2020-05-29T15:14:50Z

Dear SS3 team,

Please could you tell me how SS3 normalizes different abundance indices for building that stand_cpue_all plot? Is it explained in the tech manual?

Thank you in advance.

SS user manual (3.30.15) now can be cited

2020-05-14T21:44:44Z

The Stock Synthesis user manual (v 3.30.15) now can be cited, as it has been published as a NOAA processed report. Please cite as:

Methot et al. 2020

Methot, R. D., Jr., C. R. Wetzel, I. G. Taylor, and K. Doering. 2020. Stock Synthesis User Manual Version 3.30.15. U.S. Department of Commerce, NOAA Processed Report NMFS-NWFSC-PR-2020-05.

https://doi.org/10.25923/5wpn-qt71

Thanks to Chantel Wetzel and Al Brown for their hard work in putting it together.

Reference Points described

2020-01-21T18:11:07Z

We just developed a F0.1 option for fishing mortality reference points. In adding it to the manual, I took the opportunity to expand the general description of the benchmark (referrence point) module. We will be moving this into the user manual soon, but I thought it useful to give your a preview below:

Rick

--------------------------------------

Benchmark (reference point) calculations. This feature of SS is designed to calculate an equilibrium fishing rate intended to serve as a proxy for the fishing rate that would provide maximum sustainable yield (MSY). Then in the forecast module these fishing rates can be used in the projections.

Four reference points can be calculated by SS:

F_MSY– Search for the F that produces maximum equilibrium yield (e.g. dead catch), or set F_MSYequal to one of the other 3 options

F_SPR – Search for the F that produces spawning biomass per recruit that is a specified fraction, termed SPRtarget, of spawning biomass per recruit under unfished conditions. Note that this is in relative terms so does not take into account the spawner-recruit relationship.

F_BTGT – Search for the F that produces an absolute spawning biomass that is a specified fraction, termed relative biomass target, of the unfished spawning biomass. Note that this is in absolute terms so takes into account the spawner-recruit relationship.

F_0.1 – Search for the F that produces a slope in yield per recruit, dY/dF, that is 10% of the slope at the origin. Note that this option is mutually exclusive with F_BTGT_.Only one will be calculated and the one that is calculated can serve as the proxy for Fmsy and forecasting.

Estimation: Each of the potential reference points is calculated by searching across a range of F multiplier levels, calculating equilibrium biomass and catch at that F, using Newton-Raphson method to calculate a better F multiplier value, and iterating a fixed number of times to achieve convergence on the desired level.

Calculation: The calculation of equilibrium biomass and catch uses the same code that is used to calculate the virgin conditions and the initial equilibrium conditions. This equilibrium calculation code takes into account all morph, timing, biology, selectivity, and movement conditions as they apply while doing the time series calculations. You can verify this by running SS to calculate Fmsy then hardwire initial F to equal this value, use the F_method approach 2 so each annual F is equal to Fmsy and then set forecast F to be the same Fmsy. Then run SS without estimation and no recruitment deviations. You should see that the population has an initial equilibrium abundance equal to Bmsy and stays at this level during the time series and forecast.

Catch Units: For each fleet, SS always calculates catch in terms of biomass (mt) and numbers (1000s) for encountered (selected) catch, dead catch, and retained catch. These three categories differ only when some fleets have discarding or are designated as a bycatch fleet. SS uses total dead catch biomass as the quantity that is principally reported and the quantity that is optimized when searching for F_MSY.The quantity “dead catch” may occasionally be referred to as “yield”.

Biomass Units: The principle measure of fish abundance, for the purpose of reference point calculation, is female reproductive output. This is referred to as SSB (spawning stock biomass) and sometimes just “B” because the typical user settings have one unit of reproductive output (fecundity) per kg of mature female biomass. So when the output label says Bmsy, this is actually the female reproductive output at the proxy for Fmsy.

Fleet allocation: An important concept for the reference point calculation is the allocation of fishing rate among fleets. Internally, this is Bmark_relF(f,s) and it is the fraction of the F multiplier assigned to each fleet, f and season, s. The value, F_multiplier * Bmark_relF(f,s), is the F level for a particular fleet in a particular season and for the age that has a selectivity of 1.0. Other ages will have different F values according to their selectivity.

The Bmark_relF values can be calculated by SS from a range of years specified in the input for Benchmark Years or it can be set to be the same as the Forecast_RelF, which in turn can be based on a range of years or can be input as a set of fixed values.
Note that for Bycatch Fleets, the F’s calculated by application of Bmark_relF for a bycatch fleet can be overridden by a F value calculated from a range of years or a fixed F value that is input by the user. If such an override is selected for a bycatch fleet, that F value is not adjusted by changes to the F multiplier. This allows the user to treat a bycatch fleet as a constant background F while the optimal F for other fleets is sought. Also for bycatch fleets, there is user control for whether or not the dead catch from the bycatch fleet is included in the total dead catch that is optimized when searching for Fmsy.

Virgin vs. Unfished: The concept of unfished spawning biomass, SSB_unf, is important to the reference points calculations. Unfished spawning biomass can be potentially different than virgin spawning biomass, SSB_virgin.

Virgin spawning biomass is calculated from the parameter values associated with the start year of the model configuration and it serves as the basis from which the population model starts and the basis for calculation of stock depletion.
Unfished spawning biomass can be calculated for any year or range of years, so can change over time as R0, steepness, or biological parameters change.
In the reference points calculation, the Benchmark Years input specifies the range of time over which various quantities are averaged to calculate the reference points. For biology, selectivity, F’s, and movement the values being averaged are the year-specific derived quantities. But for the SRparms (R0 and steepness), the parameter values themselves are averaged over time.
During the time series or forecast, the current year’s SSB_unf is used as the basis for the spawner-recruitment curve against which deviations from the spawner-recruitment curve are applied. So if R0 is made time-varying, then the spawner-recruit curve itself is changed. However, if the regime shift parameter is time-varying, then this is an offset from the spawner-recruitment curve and not a change in the curve itself. So changes in R0 will change year-specific reference points and change the expected value for annual recruitments, but changes in regime shift parameter only change the expected value for annual recruitments.

Control File Updates

2020-01-21T18:11:18Z

A new User Manual was posted on June 26, 2016 that reflects the following two revisions to the control file in SS v3.30:

1. The recruitment settlement section has added an additional input for settlement age. The new setup follows the format:

#Growth Pattern, month, area, and age (for each settlement assignment)

1 1 1 0

2. In 3.24, mean size at maxage at the initial equilibrium was approximated by allowing linear growth to Linf at 2*nages and to decay the numbers at age by exp(-0.2). In 3.30, this now uses von bert growth out to 3*nages, and the decay factor, Linf_decay, is read from the control file:

1 #_Growth_Age_for_L1
999 #_Growth_Age_for_L2 (999 to use as Linf)
0.2 #_exponential decay for growth above maxage (fixed at 0.2 in 3.24; should approx initial Z)

sstrans fixes Linf_decay at 0.20 to match 3.24

Updated User Manual

2020-01-21T18:11:30Z

A revised Stock Synthesis 3.30 User Manual was posted on 5/18/16. This version is still in draft form and will continue to be updated prior to the official release of version 3.30.

revised User Manual is awesome!

2020-01-21T18:11:41Z

Good job, Chantel!