Modernize your video streaming with personalized user Experience

By Nazir Kabani．Dec 8, 2022

Hope you had a great experience at NEXT22, during NEXT we announced support for Live streaming on Media CDN and Integration with Google Ad Manager 360 with Media CDN.

In this article I am covering integration between Media CDN with Google Ad Manager 360 and also securing your customer facing urls with dual-token authentication. Before we start technical details, let’s learn a bit about Media CDN and Google Ad Manager.

About Media CDN

Media CDN is Google Cloud’s media delivery CDN platform designed for streaming video and large object downloads. Media CDN complements Cloud CDN, our web acceleration CDN platform.

What is unique about Media CDN is that it leverages the infrastructure that we are using to deliver content to our customers- based on Google’s privately owned fiber network.

For the first time, non-Google customers will have access to this platform providing massive coverage and presence globally in over 206+ countries/territories and 1300+ cities.

Article-Modernize your video streaming with personalized user Experience-1

This map is a graphical representation of the breath of our network, not a precise indicator of the location of specific caches.

About Google Ad Manager (GAM)

Google Ad Manager is an ad management platform for large publishers. Ad Manager provides granular controls and supports multiple ad exchanges and networks, including AdSense, Ad Exchange, third-party networks, and third-party exchanges.

Ad Manager is for you if you need:

A central place to monetize all of your inventory types (websites, mobile apps, videos, or games)
To use third-party networks to compete for ad inventory
More complex reports to gain granular insights

Integrate Media CDN with GAM

Audiences today are increasingly watching TV & video across a multitude of devices. Delivering relevant ads seamlessly across these devices, while simultaneously delivering a great experience, has become incredibly difficult to achieve.

With integration between media CDN and GAM we are bringing the best of both worlds together by delivering modernize streaming experience with Media CDN and personalized user experience with server-side ad insertion (SSAI).

With integration between Media CDN and GAM, we take the complexity of client-side ad integration from the player to the cloud, and stitch the ad to the content at the stream level. What you get is

Seamless watch experience between ad and content with no rebuffering
Across any number of devices
Content and Ad delivery for live, linear and VOD content
Targeted & relevant Ads for each individual user

Before we start this integration, I would like to show an end state architecture for you to have a good understanding of how our overall solution will look once it will be deployed.

Article-Modernize your video streaming with personalized user Experience-2

Prerequisites:

Media-CDN Prerequisites:

Two Media CDN key-sets (one for short-token and one for long token)
Two Media CDN origins (One pointing to dai.google.com and one pointing to your actual origin)
Two Media CDN configurations with SSL certificates enabled (This is a must for DAI).

GAM Prerequisites:

Access to GAM 360 with Relative playlist URLs field enabled in your network, this is a pre-release feature not available on your UI by default (You will find this in content stream settings of live-stream-configuration.

See below screenshot and check if your network supports relative playlist urls. If it doesn’t show this option then please reachout to your GAM team to enable it for you.

Article-Modernize your video streaming with personalized user Experience-3

Media CDN configurations

Short-Token Key-set creation for Dual-token authentication.

DAI only supports HMAC, so you will have to create HMAC key-set for short duration tokens.

Follow the steps mentioned in https://cloud.google.com/media-cdn/docs/use-dual-token-authentication#set-up-short-duration-tokens

for HMAC key-set creation.

To generate secret you can use following command

python3.9 -c "import secrets;open('py-shared.secret','wb').write(secrets.token_bytes(32))"

Once you upload shared.secret file to secret manager it will give you 64 character key (After removing spaces it will look similar to d7258f90d2f4049e14b1c5ecfbc698fbcb925111acb808dd17f22153c1fcc71d, please note it down as this will be required in DAI configuration)

Long-Token Key-set creation for Dual-token authentication.

Follow the steps mentioned in https://cloud.google.com/media-cdn/docs/use-dual-token-authentication#create-long-duration-keyset for Media CDN managed long-token, this is CDN managed long-token because Media CDN uses ED25519 signatures for generating and validating long-tokens. ED25519 is an asymmetrical singing algorithm and requires a private-key for token generation and a public-key for token validation.

Since CDN will have to do both generations and validation, a long key-set will require to be CDN managed.

Creation of two origins

DAI-origin

This origin will point to dai.google.com for master and child manifest requests. We will configure routing rules in Media-CDN service.

Sample YAML for reference.

Before you use YAML, please change <variables> between <> below, for e.g., your project-ID, project number etc.. , variables with curly braces — {variables} are media CDN variables please don’t change them if not required.

name: projects/<your-project-Id>/locations/global/edgeCacheOrigins/dai-origin
updateTime: '2022-09-30T10:25:44.195099638Z'
originAddress: dai.google.com
protocol: HTTP2
timeout:
 connectTimeout: 5s
 maxAttemptsTimeout: 15s
 responseTimeout: 30s
 readTimeout: 15s
createTime: '2022-09-27T05:20:23.011190947Z'

Actual content origin

This origin will point to the GCS bucket where your actual content is residing. We will use this origin in both end-user facing service and dai facing service (asynchronous fetch from DAI to Media CDN for master and child manifest).

Sample YAML for reference.

name: projects/<your-project-Id>/locations/global/edgeCacheOrigins/content-origin
updateTime: '2022-09-26T06:36:53.721017008Z'
originAddress: gs://<your-GCS-bucket-name>
protocol: HTTP2
port: 443
timeout:
connectTimeout: 5s
maxAttemptsTimeout: 15s
responseTimeout: 30s
readTimeout: 15s
createTime: '2022-09-26T06:35:44.718831043Z'

Creation of two services

Create two services to get an IP address and configure SSL certificates. We will update the service once we have GAM streams created.

Create the service yaml — dai-facing.yaml and upload to cloud shell.

name: dai-facing
routing:
  hostRules:
  - hosts:
    - "*"
    pathMatcher: routes
  pathMatchers:
  - name: routes
    routeRules:
    - priority: 1
      matchRules:
      - prefixMatch: "/"
      origin: dai-origin

Import the service using the yaml.

gcloud edge-cache services import dai-facing \
  --source={path-to-the-yaml-file}

Retrieve and take note of the returned IP in the response by checking the ipv4Addresses field.
Similarly create the service yaml — customer-facing.yaml and upload to cloud shell.

name: customer-facing
routing:
  hostRules:
  - hosts:
    - "*"
    pathMatcher: routes
  pathMatchers:
  - name: routes
    routeRules:
    - priority: 1
      matchRules:
      - prefixMatch: "/"
      origin: content-origin

Import the service using the yaml.

gcloud edge-cache services import customer-facing \
  --source=<path-to-the-yaml-file>

Retrieve and take note of the returned IP in the response by checking the ipv4Addresses field.

Configure domains to point to CDN service IPs

Follow the steps in https://cloud.google.com/dns/docs/set-up-dns-records-domain-name#create_a_record_to_point_the_domain_to_an_external_ip_address to create A record to point the domain at the CDN service IP of dai-facing service
Similarly, follow the steps in https://cloud.google.com/dns/docs/set-up-dns-records-domain-name#create_a_record_to_point_the_domain_to_an_external_ip_address to create A record to point the domain at the CDN service IP of customer-facing service
Create SSL certificates for both dai-facing service and customer-facing service by following the steps in https://cloud.google.com/media-cdn/docs/configure-ssl-certificates

Now we will do CDN configuration and Linear configuration in GAM

GAM configuration

Linear CDN configuration in GAM

Click New configuration

General settings

Name: <give-your-preferred-CDN-configuration-name>
Type: Live stream content
Ingest settings
Ingest URL prefix: <FQDN-of-your-dai-facing-service>/<prefix1>/

#prefix1 is prefix of your choice, for subsequent configurations we will refer this as a prefix1

CDN authentication: Akamai
Token authentication: Enabled
Token authentication key: <64 character long HMAC key which we created earlier>

Default delivery settings

Delivery URL prefix: <FQDN-of-your-customer-facing-service>
Security Settings: Check “Use ingest security settings”
Master playlist origin forwarding type: Conventional
Master playlist origin path prefix: /<prefix1>
Click SAVE

Linear Stream configuration in GAM

Click New live stream

General settings

Name: <give-your-preferred-linear-stream-configuration-name>
End time: Unlimited

Content stream settings

Content stream: https://<fqdn-of-dai-facing-service>/<prefix1>/<path-to-master-manifest-file>/index.m3u8 (check your master manifest file name is correct or not)

# master manifest url with {prefix1} after FQDN/

Live stream content configurations: <select-cdn-configuration-name-we created-earlier>
Relative playlist URLs: Enabled

Ad break settings

Master ad tag: <select-or-add-ad-tag-url-as-per-your-business-requirement>

Click SAVE and Find the live stream that was created and note the Asset key

Asset key: ABCDEFGHIJKLMNOPQRS

Update Media-CDN services:

By this time, SSL certificates should be active. Check SSL certificate status using below command

gcloud certificate-manager certificates describe <certificate-name>

Update dai-facing service:

Update the dai-facing.yaml file as per below modifications using cloud shell and import.

Note: we are only creating routing rules for m3u8 files as DAI only expects to get m3u8 files from your origin and it doesn’t require actual segment download from your actual origin. If someone will get access to this FQDN and manifest file path then also media CDN will not serve segments to requesters as there is no route rule for .ts files.

name: projects/<your-project-Id>/locations/global/edgeCacheServices/dai-facing
updateTime: '2022-09-29T07:28:09.450208839Z'
requireTls: true
edgeSslCertificates:
- projects/<your-project-number>/locations/global/certificates/<certificate-name-of-your-dai-facing-service>
routing:
 hostRules:
 - hosts:
   - <FQDN-of-your-dai-facing-service>
   pathMatcher: path-matcher-0
 pathMatchers:
 - name: path-matcher-0
   routeRules:
   - priority: '1'
     matchRules:
     - prefixMatch: /<prefix1>/<path-to-master-manifest-file>/<master-manifest-file-name>.m3u8 #this will point to your master manifest file
     origin: projects/<your-project-number>/locations/global/edgeCacheOrigins/content-origin
     headerAction: {}
     routeAction:
       cdnPolicy:
         cacheMode: FORCE_CACHE_ALL
         defaultTtl: 3600s
         cacheKeyPolicy:
           excludeHost: true
           excludeQueryString: true
         signedRequestMode: DISABLED
       urlRewrite:
         pathPrefixRewrite: /<path-to-master-manifest-file>/<master-manifest-file-name>.m3u8 #Please note - this is without prefix1
   - priority: '2'
     matchRules:
     - pathTemplateMatch: /<prefix1>/<path-to-master-manifest-file>/{variant}/{file}.m3u8 #{variant} and {file} are media cdn supported variables here, if you don’t have folders underneath your master manifest file, please remove {variant}/ from pathTemplateMatch, do not remove curly braces from variables.
     origin: projects/<your-project-number>/locations/global/edgeCacheOrigins/content-origin
     headerAction: {}
     routeAction:
       cdnPolicy:
         cacheMode: FORCE_CACHE_ALL
         defaultTtl: 2s
         cacheKeyPolicy:
           excludeHost: true
           excludeQueryString: true
         signedRequestMode: DISABLED
       urlRewrite:
         pathTemplateRewrite: /<path-to-master-manifest-file>/{variant}/{file}.m3u8 #{variant} and {file} are media cdn supported variables here, if you don’t have folders underneath your master manifest file, please remove {variant}/ from pathTemplateMatch, do not remove curly braces from variables.
logConfig:
 enable: true
 sampleRate: 1.0
createTime: '2022-09-28T12:49:55.392769608Z'

Import the service using the yaml.

gcloud edge-cache services import dai-facing \
--source={path-to-the-yaml-file}

Update user facing service:

Update the customer-facing.yaml file as per below modifications using cloud shell and import.

name: projects/<your-project-Id>/locations/global/edgeCacheServices/customer-facing
updateTime: '2022-09-30T12:31:44.574157874Z'
requireTls: true
edgeSslCertificates:
- projects/<your-project-number>/locations/global/certificates/<certificate-name-of-your-customer-facing-service>
routing:
 hostRules:
 - hosts:
   - <FQDN-of-your-customer-facing-service>
   pathMatcher: path-matcher-0
 pathMatchers:
 - name: path-matcher-0
   routeRules:
   - priority: '1'
     matchRules:
     - pathTemplateMatch: /linear/*/*/*/*/*/*/master.m3u8
     origin: projects/<your-project-number>/locations/global/edgeCacheOrigins/dai-origin
     headerAction:
       responseHeadersToAdd:
       - headerName: x-cdn-status
         headerValue: '{cdn_cache_status}'
     routeAction:
       cdnPolicy:
         cacheMode: CACHE_ALL_STATIC
         defaultTtl: 3600s
         maxTtl: 86400s
         cacheKeyPolicy:
           excludeHost: true
           excludeQueryString: true
         signedRequestMode: REQUIRE_TOKENS
         signedRequestKeyset: <your-short-token-keyset-name> #This is to validate urls and tokens generated by DAI
         signedTokenOptions:
           tokenQueryParameter: hdnea
           allowedSignatureAlgorithms:
           - HMAC_SHA_256
           - ED25519
           - HMAC_SHA1
         addSignatures:
           actions:
           - GENERATE_TOKEN_HLS_COOKIELESS
           keyset: <your-long-token-keyset-name> #This is to add long tokens in query parameters by media CDN
           tokenTtl: 86400s
           tokenQueryParameter: hdntl
           copiedParameters:
           - PathGlobs
           - URLPrefix
           - acl
       urlRewrite:
         hostRewrite: dai.google.com
   - priority: '3'
     matchRules:
     - pathTemplateMatch: /**.ts
     origin: projects/<your-project-number>/locations/global/edgeCacheOrigins/content-origin
     headerAction:
       responseHeadersToAdd:
       - headerName: x-cache-status
         headerValue: '{cdn_cache_status}'
     routeAction:
       cdnPolicy:
         cacheMode: FORCE_CACHE_ALL
         defaultTtl: 31536000s
         cacheKeyPolicy:
           excludeHost: true
           excludeQueryString: true
         negativeCaching: true
         negativeCachingPolicy:
           '403': 120s
           '404': 120s
           '502': 120s
         signedRequestMode: REQUIRE_TOKENS
         signedRequestKeyset: <your-long-token-keyset-name> #This is to validate urls and tokens generated media CDN in response to child manifest
         signedTokenOptions:
           tokenQueryParameter: hdntl
   - priority: '2'
     matchRules:
     - pathTemplateMatch: /**.m3u8
     origin: projects/<your-project-number>/locations/global/edgeCacheOrigins/dai-origin
     headerAction:
       responseHeadersToAdd:
       - headerName: x-cache-status
         headerValue: '{cdn_cache_status}'
     routeAction:
       cdnPolicy:
         cacheMode: CACHE_ALL_STATIC
         defaultTtl: 3600s
         maxTtl: 86400s
         cacheKeyPolicy:
           excludeHost: true
           excludeQueryString: true
         signedRequestMode: REQUIRE_TOKENS
         signedRequestKeyset: <your-long-token-keyset-name> #This is to validate urls and tokens generated by media CDN in response to master manifest file
         signedTokenOptions:
           tokenQueryParameter: hdntl
           allowedSignatureAlgorithms:
           - HMAC_SHA_256
           - ED25519
           - HMAC_SHA1
         addSignatures:
           actions:
           - PROPAGATE_TOKEN_HLS_COOKIELESS #Media CDN can also generate token option for segments, you can also use the GENERATE_TOKEN_HLS_COOKIELESS option here.
           keyset: <your-long-token-keyset-name> #This is to add long tokens in query parameters by media CDN 
           tokenTtl: 86400s
           tokenQueryParameter: hdntl
           copiedParameters:
           - PathGlobs
           - URLPrefix
           - acl
       urlRewrite:
         hostRewrite: dai.google.com
logConfig:
 enable: true
 sampleRate: 1.0
createTime: '2022-09-26T06:58:15.925641821Z'

Import the service using the yaml.

gcloud edge-cache services import customer-facing \
 - source=<path-to-the-yaml-file>

It’s done, try playing GAM url and insert SCTE markers to test your integration.

Time to Celebrate 🥳

Hope you were with me till the end of this journey, give yourself a treat as you deserve it by building modernize streaming approach using Media CDN and personalized user experience using Google Ad Manager at a Google scale.

Do share your feedback with me and suggest what I should cover as a part of my next blog covering GCP’s media offerings.

Disclaimer: This is to inform readers that the views, thoughts, and opinions expressed in the text belong solely to the author, and not necessarily to the author’s employer, organization, committee or other group or individual.

The original article published on Medium.

Tags: CDN, Google ad Manager (GAM), Media CDN

Our DevOps Solution

What's DevOps?

COMMUNITY & ARTICLES