Video

HLS Streams

Video is made available primarily via HLS. You can get the available streams and their HLS manifest URLs via GraphQL:

query VideoStreamExample {
  node(id: "CA~MjAyMi9QT1NULzQvNTkxNzE") {
    ... on NFLGame {
      avStreamsConnection(first: 100) {
        edges {
          node {
            id
            name
            hlsManifestURL
          }
        }
      }
    }
  }
}

Open in GraphQL Explorer

The HLS playlists include precise timestamps via #EXT-X-PROGRAM-DATE-TIME. This can be used to synchronize data with video.

Identifying Streams

The name field is intended to be used only for humans and is subject to change. To identify a particular stream, such as the program stream, it's recommended that you use the contentDescriptor field, which is more stable and intended for this use-case. For the main program stream, this will always be "Program".

Most streams are multivariant streams. That means each stream will have multiple renditions available with varying bitrates and codecs. For example, program streams typically have 4-5 video resolutions available and an audio-only variant. Players will automatically choose an appropriate variant based on the player size and network conditions, but there are cases where it's useful to know which variants are available ahead of time. You can access this information via the variantsConnection field.

For example, this query will return the content descriptors and variants available for all streams:

query StreamIdentificationExample {
  node(id: "CA~MjAyMi9QT1NULzQvNTkxNzE") {
    ... on NFLGame {
      avStreamsConnection(first: 100) {
        edges {
          node {
            id
            contentDescriptor
            variantsConnection(first: 100) {
              edges {
                node {
                  bitrate
                  tracks {
                    rfc6381Codec
                    ... on AVStreamVariantVideoTrack {
                      height
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
  }
}

Open in GraphQL Explorer

Clips

Times and durations can be given to the hlsManifestURL field to get a URL for a clip:

query VideoClipExample {
  node(id: "CA~MjAyMS9SRUcvMS81ODUwMw") {
    ... on NFLGame {
      avStreamsConnection(first: 100) {
        edges {
          node {
            id
            name
            hlsManifestURL(time: "2021-09-10T00:24:10Z", durationSeconds: 30)
          }
        }
      }
    }
  }
}

Open in GraphQL Explorer

Downloads

Download URLs for video can be obtained in the same way as HLS URLs:

query VideoDownloadExample {
  node(id: "CA~MjAyMS9SRUcvMS81ODUwMw") {
    ... on NFLGame {
      avStreamsConnection(first: 100) {
        edges {
          node {
            id
            name
            downloadURL(time: "2021-09-10T00:24:10Z", durationSeconds: 30)
          }
        }
      }
    }
  }
}

Open in GraphQL Explorer

Augmented Reality

Some streams have camera calibration data available. This data can be used to determine precisely where the camera is looking at any given time. A common use-case for this information is rendering AR overlays on top of the video.

Streams have a projectionViewMatrix field that provides a 4x4 matrix that can be used to transform 3D world-space points into 2D screen-space points. For fixed cameras, this field is available directly on the AVStream. For moving cameras, this field is available on the nodes of the stream's cameraCalibrationsConnection:

query CameraCalibrations($streamId: ID!) {
  node(id: $streamId) {
    ... on AVStream {
      # For fixed cameras, this field is populated.
      projectionViewMatrix
      # For moving cameras, this connection must be used.
      cameraCalibrationsConnection(first: 100) {
        edges {
          node {
            time
            projectionViewMatrix
          }
        }
      }
    }
  }
}

Open in GraphQL Explorer

To use this matrix, world-space coordinates must be converted into meters from the center of the field. By multiplying those coordinates with the projectionViewMatrix, the resulting coordinates will be in normalized camera coordinates. These coordinates can then be converted into pixel coordinates:

im = Image.open('frame.png')
projection_matrix = np.array(node['projectionViewMatrix'])


def world_to_pixel_coord(x, y, z):
    world_coord = np.array([x, y, z, 1])
    camera_coord = projection_matrix.dot(world_coord)
    normalized_camera_coord = camera_coord[:2] / camera_coord[3]
    pixel_coord = (normalized_camera_coord * np.array([0.5, -0.5]) + 0.5) * np.array([im.width, im.height])
    return pixel_coord


for i in range(104):
    pos1 = world_to_pixel_coord(i - 51.5, -34, 0)
    pos2 = world_to_pixel_coord(i - 51.5, 34, 0)
    draw.line([tuple(pos1), tuple(pos2)], fill='yellow', width=2)


im.save('frame-out.png')

The output of the above Python snippet looks like this:

This site is open source!

See something that could be improved? Open a pull request on GitHub.

Contribute to This Page

FusionFeed

Video

HLS Streams

Identifying Streams

Clips

Downloads

Augmented Reality

This site is open source!

On This Page