-
Notifications
You must be signed in to change notification settings - Fork 0
Meeting Notes
Andrew Allyn edited this page Feb 8, 2021
·
6 revisions
Note sure if this is the best place for this, but seemed more appropriate than an issue and didn't want to set up another thing (google drive, Box folder) that people would need to monitor. So, here we are.
- Round-the-horn updates of any related work completed during the past week and current challenges/questions, focusing on wrapping up the bias correction conversation
(Adam, Andrew, Kathy, Matt)
Round-the-horn updates and problem solving
Kathy (and Lindsay)
- Still some questions about the DFO data and have since sent along email messages to relevant folks there to hopefully get some answers. Kathy planning to send along another email on Monday afternoon if we still haven't heard back anything.
- Provided some background slides on the bias correction saga, including the difference between bias correction/nudging and then the delta method, and discussed the different methods
Adam
- US trawl data questions passed along and hoping for some answers on them next week. Appears to be a correction factor issue causing the discrepancies.
- Finished up conversation on the bias correction process, electing to go with the "delta method" -- though both likely equivalent!
Matt
- More downloading progress and discussion on what we might be able to use for bias correcting the other variables, like bottom temperature
- SODA the likely product here. Andrew/Matt will work together here to try to get some plots together for checking appropriateness of the SODA product for bottom temperature/salinity.
- Round-the-horn updates of any related work completed during the past week and current challenges/questions, focusing on bias correcting GCM projections and DFO trawl data progress
(Adam, Andrew, Kathy, Lindsay, Matt)
Round-the-horn updates and problem solving
Matt
- Successfully grabbed all of the historical bottom temperature data for available models and interpolated to common 1 degree grid
- Some weirdness when looking at the interpolated surfaces as it temperatures interpolated over the whole globe, but, doesn't seem to be causing any issues within our area of interest (Would a land mask fix this? Potentially helpful link here
- Hoping to have SST pulled by the end of the week
Adam
- Starting in on workflow for bias correcting CMIP6 GCMs with OISST data
- A lot of discussion on the different potential processes for doing the bias correcting. Went down a bit of a rabbit hole here, but there seem to be two different processes:
- Using deltas to bias correct. For this process, the first step is to calculate the CMIP6 GCM deltas (i.e., anomalies) from the CMIP6 GCM climatology. After calculating the climate model deltas (again, calculated entirely within the GCM space), they are then applied to the observed NOAA OISST climatology for the same reference period. This process yields biased corrected projected CMIP6 GMC temperatures and is the process we used during the COCA I project and used by Mike A and others.
- Using climatology differences to nudge CMIP6 GCM projected temperatures. For this process, the first step is to calculate the climatologies for the CMIP6 GCM projected temperatures and then the observed NOAA OISST temperatures for the same reference period. The differences between the two climatologies are then calculated and used to nudge the CMIP6 GCM projected temperatures up/down. In theory, this sounds like it should produce the same results, though not confirmed. Also, not sure if this method has been used within the species distribution model projection "world."
- A lot of discussion on the different potential processes for doing the bias correcting. Went down a bit of a rabbit hole here, but there seem to be two different processes:
- During this conversation, Kathy also mentioned interest in trying to figure out how to get bias corrected CMIP6 GCM projected temperatures over shorter time chunks.
- During the COCA I project, we noticed that the observed NOAA OISST data were running "warmer" than the CMIP5 GCM ensemble mean projected temperature during the recent observation period. This could be happening because the GCMs are applying anomalies to climatology based on 1982-2011. So, some interest in seeing if we could do something different, that would involve applying anomalies to a more "recent" reference period and bring the GCM projected temperatures in line with the observations. Kathy plans to ask Mike A about this.
Lindsay
- Progress with the DFO data and some "new" issues/questions
- Some changes in the number of "missions" and variability in biomass per tow that are puzzling. Plans to chat with Kathy/DFO folks about these questions and then talk to Andrew to get a better idea of the format of the trawl data that ends up feeding into the SDMs.
- Round-the-horn updates of any related work completed during the past week and current challenges/questions, focusing on temperature components (CMIP6 pulls and OISST trends)
(Adam, Andrew, Kathy, Lindsay, Matt)
Round-the-horn updates and problem solving
Matt
- Continuing with download process after finding an easier way to do this through a google site rather than the THREDDS server
- Some discussion later on about the extent of the bounding box to use when downloading. Eventually decided to go with an area that covers from Florida to Newfoundland/Greenland (Atlantic salmon regions)
-
Worked to better understand differences along the coast for interpolating CMIP output onto a regular grid..
- After discussing this with the team, decided it is best to go with the inverse distance weighting option as this matches what Jamie/Mike are doing, and, we are likely going to "crop" any temperatures to the region of interest (mask out land) eventually anyway, so land-ward creeping not a major problem so long as the temperatures along the coast are not heavily effected.
Adam
- Planned to be working a bit more on trawl data, but then caught discrepancies in OISST time series data depending on which shapefile was being used to define the "Northeast Shelf U.S. Large Marine Ecosystem".
- The "issue" seems to be that OISST trends are dampened when a high resolution shapefile with the Bay of Fundy and the Chesapeake Bay is included in comparison to shapefile with lower resolution and without the Bay of Fundy and Chesapeake Bay. For example, the 2012 anomaly is nearly a degree different between the two shapefiles.
- A lot of discussion on potential causes and ways of isolating the error. Adam will work a bit more on this.
- As a group, we should probably make a decision on shapefiles and spatial regions of interest moving forward. To some degree, for the OISST temperature work we are going to be tied to whichever shapefile can replicate the "standard" plots of the past. But, rather than having 3-4 other "NELMEs", should likely pick one that serves our purposes if we need a different one and stick to that.
Lindsay
- Started with some of the DFO data, but immediately ran into some R/FTP server issues
- Odd behavior as the FTP server seemed to work for some folks and not others. In the end, got around it and Kathy was able to send Lindsay the folder contents. This should then open up the chance to use DFO's
Mar.datawranglingR package.
- Odd behavior as the FTP server seemed to work for some folks and not others. In the end, got around it and Kathy was able to send Lindsay the folder contents. This should then open up the chance to use DFO's
Kathy
- Working with Matt to get some data to test the potentially different ways of doing the climate data "bias correction"
- Round-the-horn updates of any related work completed during the past week and current challenges/questions
- Project organization and reviewing plans to use Slack, Git/GitHub structure
- Work plans for the next week
(Adam, Andrew, Kathy, Lindsay, Matt)
Round-the-horn updates
Matt
- Making progress on downloading the CMIP6 data, though each file takes 1/2 a day to a full day
- Plans to divide and conquer these a bit moving forward after populating a "download sheet" that will provide the information others need to locate the specific models of interest on the servers
- Looming questions surrounding the number of climate models to include in any eventual ensemble and then also what to do about interpolating to "coastal" cells
- On the ensemble front, likely the more models available the better. Only a limited number at the 1 deg scale (9 ish?), but then 40 or so if we also bring in models run over a 100km grid. It would be good to check with Jamie/Mike on this to see if they have a list they plan on using for any "portal" like the one created for the CMIP5 models/ensemble.
- On the coastal interpolation front, looking at a few models, it seems that there are not the same temperature gaps as present with the CMIP5 data. So, this might not be an issue. In the meantime, worth looking into this across more models, see if Jamie/Mike have insight. Also, Andrew can pass along code for the "interpolation" done for COCA I project and these coastal cells.
- A more general discussion about storing the CMIP6 data and issues with Box. A recurring issue with any large files for that matter.
- Not sure if we completely resolved this?
Adam
- Able to move all the OISST "stuff" over from the knowledge graph storage space to the sdm_workflow space
- Figured out the "dates" issue and how to account for preliminary observations, with plans to discuss with Alex K. how we might then set this all up as an automated process
- With OISST "stuff" in a good place, plans to transition towards looking at other components of the workflow, including current GitHub issues (bias correcting temperatures, trawl data preparation and distribution model fitting)
Andrew
- Finished up the COCA I project markdown document, which will soon be on GitHub (after addressing the error of trying to push a large file)
- Progress with GitHub sdm_workflow repo and project organization components
Roles/Up-coming work plans
Matt
- Continue with CMIP6 processing
- Distribute download sheet for other folks to cut down on file download time
- Check in with Jamie/Mike to get their input/guidance on which models to download/include in the ensemble
- Connecting with Andrew and Adam to make sure temperature things start talking to each other, including CMIP6 to OISST from Adam for bias correction, and then bias corrected projections to SDM projection code)
Adam
- Connecting with Andrew and Matt to make sure things start talking to each other, including CMIP6 to OISST from Adam for bias correction, and then bias corrected projections to SDM projection code)
- Explore bias correction for BT through literature searches (if interest/time)
Lindsay
- Connect with Adam and Andrew on trawl data preparation
- Dig into DFO data and begin assessing the impending species naming nightmare
- Explore what other folks have used for "species cut offs" when assessing leading/trailing edge dynamics as well as fitting SDMs
Andrew
- Re-visit and summarize emails from Vince concerning bottom temperature data sets/bias correction and share summary with the group
- Explore bias correction for BT through literature searches (if interest/time)
- Connecting with Adam/Matt on bias correction and then Adam/Lindsay on trawl data preparation
- VAST "intra-annual" model
Kathy
- Explore what other folks have used for "species cut offs" when assessing leading/trailing edge dynamics as well as fitting SDMs, to advise Lindsay
- Round-the-horn updates of any related work completed during the past month -- what have you been up to, are you struggling with anything? No slides/prep needed.
- Work planning
- Any remaining questions about what we are hoping to do over the next three months?
- Reviewing and discussing individual team member roles/responsibilities
(Adam, Andrew, Kathy, Matt)
Round-the-horn updates
Kathy
- DFO data agreement now in place, ready to start gathering and processing these data
Matt
- Met with Jamie S on CMIP6 and ROMS outputs
- Has a script for gathering and processing CMIP6 outputs (historical and future projected SST and BT, rescaling to common 1 deg grid
- Has ROMS outputs
- I was a bit unclear here, sounds like at least four model runs and associated forcing information?
Adam
- Working on the observed SST data processing and visualization
- Dug into Eric B (ODP) code to look at how they are doing the “automated” pull and continuous integration, while also accounting for the “preliminary” most recent two-weeks OISST products
- Discussed this quickly with Alex K, likely that ODP team could help us handle any of this continuous integration/scheduling bits
Andrew
- Working on Markdown document of COCA I project workflow (should be completed by next meeting)
Roles/Up-coming work plans
Matt
- Continue with CMIP6 processing
- Adjusting “RES Data” path and double checking things still “work”
- Connecting with Andrew and Adam to make sure things start talking to each other, including CMIP6 to OISST from Adam for bias correction, and then bias corrected projections to SDM projection code
- Exploring bias correction for BT by emailing Jamie S and literature search (if interest/time)
Adam
- Continue with OISST gathering/exploration work
- Connecting with Andrew and Matt to make sure things start talking to each other, including CMIP6 to OISST from Adam for bias correction, and then bias corrected projections to SDM projection code
- Explore bias correction for BT through literature searches (if interest/time)
Andrew
- Wrap up Markdown document of COCA I process
- Re-visit and summarize emails from Vince concerning bottom temperature data sets/bias correction and share summary with the group
- Explore bias correction for BT through literature searches (if interest/time)
- Investigate FreedCamp/Slack integration
- Get team set up with FreedCamp project
Lindsay
- Connect with Kathy (and Andrew?) about DFO data and work plans