FrontPage

Details Print

Draft

Comments welcome at the following VLab forum thread:

https://vlab.noaa.gov/web/vlab-improvement-team/forum-discussions/-/message_boards/message/5929255

Overview

The VLab Improvement Team (VIT) recognizes the need for hosting video content that could be viewed from within the VLab.  Due to concerns about bandwidth usage, the VLab itself is not considered an appropriate storage venue for this type of content. Rather, the VIT believes these videos should be stored and viewed via cloud services.  With no such services currently available to NWS/NOAA staff, the VIT will attempt to initiate and support efforts to establish enterprise-wide cloud services that VLab users can easily tap into to support their video hosting needs.

Background

At its November 13, 2018 meeting, the VIT endorsed a user request (VIT Issue 38295) to host video content within the VLab.  However, during discussions on December 3, 2018, a group consisting of Brian Miretzky, Ken Sperow, and Todd Neison reviewed the practicality of the VLab serving as a repository for video content.  The group agreed that the VLab was not a suitable place to store and serve video files.  The following options were considered during the meeting.

  1. Host video content on the VLab (Suggested approach)
    • Pro: Easy for VLab users to manage their content
    • Con: VLab network resources do not support bandwidth requirements and file storage could become an issue
  2. Host video content on a YouTube Channel dedicated to VLab content (Current approach)
    • Pro: YouTube designed to host video content in terms of bandwidth and storage
    • Con: Not accessible from AWIPS, policy concerns (below), and long term availability of access to YouTube (currently issues with access and Google Plus)
  3. Host video content on a YouTube Channel dedicated to NWS training content
    • Pro: Makes intended use and ownership clearer plus see item 2 above
    • Con: Excludes non-training content plus see item 2 above
  4. Host video content on an NWS cloud services site (Best strategic option) (more below)
    • Pro: Ideally suited for video hosting and cheaper when on premise costs are fully considered (storage engineers salary, equipment replacement and service, etc)
    • Pro: Cloud storage can be encrypted, and access to it can be easily controlled via private networks, shared access signatures/keys, etc.
    • Pro: Cloud providers also enable to use Content Delivery Networks for greatly improved performance
    • Pro: There is no infrastructure to manage, to physical equipment to purchase or replace, backups can be automatic
    • Pro: Cloud storage is inherently redundant in terms of the underlying architecture, with typically a greater than 99.99% availability
    • Con: Every byte of cloud data, as well as bandwidth and possibly operation charges results in a direct cost that has to be paid by somebody
    • Con: Must also be deemed secure enough to use on AWIPS, but this is the best chance to do so.
  5. Host video content on Google Drive and Google Sites
    • Pro: Easy access to users and supports storage and bandwidth requirements
    • Con: See item 2 above 

Of the option reviewed, the group felt that using cloud services provide the greatest potential to meet user requirements for hosting video content.

YouTube Usage Policy

NWS and/or NOAA policy on YouTube usage- https://sites.google.com/a/noaa.gov/nws-social-media/platforms/youtube. Other pages on that site detail NWS/NOAA policies including the need to have office accounts and not accounts by a person.

Cloud Service Cost

Amazon and Microsoft cloud storage costs are determined by several factors in addition to the actual size ranging from the region(s) the data is kept, the activity/requests, egress bandwidth, redundancy etc.  A rough estimate of approximately $40 / TB per month is a good back of the napkin number to use to be safe.  Storage that is infrequently accessed can be much cheaper as low as a few dollars per TB per month.  Bandwidth, where you typically pay for egress bandwidth, can be difficult to predict though that is usually a small percentage of the storage costs.

Another consideration for cloud service costs is you typically pay in advance and it works like a debit card and your daily/monthly costs based on consumption are deducted as you go.  This is the opposite of what most government managers are accustomed to and requires an adjustment in thought process.

The key for managing this is proper governance.  Storage should be organized via separate accounts or 'containers' per funding account to ensure cost traceability.  The cloud service providers have incredibly detailed cost data to assist with this.  

Way Forward

  1. Draft a proposal endorsing cloud services for the purposes of hosting video content for NWS/NOAA staff. 
  2. Work with other potential users such as AWIPS and NCEP to establish an enterprise-wide cloud capability that is responsive to user needs including those needing to host video content
909 Views
Average (0 Votes)