KeystoneDepth

A collection of 37,239 rectified historical stereo image pairs from the Keystone-Mast Collection.

Xuan Luo

University of Washington

Yanmeng Kong

University of Washington

Jason Lawrence

Google Research

Ricardo Martin-Brualla

Google Research

Steven M. Seitz

University of Washington, Google Research

Term Definition

  • Stereograph: the digital scan including left and right images.
  • Full: axis-aligned bounding boxes marking the left and right image regions (green).
  • Artifact-free: the largest axis-aligned bounding box inside each image region that is free enough of artifacts for reliable disparity estimation (red).
  • Disparity Information: disparity information maps have two auxiliary channels in addition to the (horizontal) disparity maps: [channel 0] (horizontal) disparity, [channel 1] vertical disparity, [channel 2] forward-backward flow consistency check distance. When visualizing the extra channels in addition to the disparity maps, the green channel is set to 255 if forward-backward consistency distance is beyond 1px, and the blue channel is set to 255 if vertical parallax is beyond 2 px.
  • Principal point: we use the original centers of full left and right image regions to approximate the principal points for the antique cameras. We compute where these original centers are in their rectified full and artifact-free versions in pixel position.

Download Links

All data are hosted on Google Drive as below. For rectified image pairs, we show gifs made from the pairs for better visualization. The separate image pair files that are stored in the same format as in the zip files can be accessed through the [pairs] links.

Path Size Files Format Description
keystondepth-dataset/
├ data/ 573 GB
    ├ stereographs/ 6 GB 37,239 JPG
    ├ stereo_full_images/ 49 GB 74,478 PNG Left and right image regions from each stereograph (green).
    ├ rectified_full_stereo_pair_gifs/[pairs] 16 GB 74,478 PNG Rectified version of the left and right image regions (green).
    ├ rectified_artifact-free_stereo_pair_gifs/[pairs] 13 GB 74,478 PNG Rectified version of the artifact-free crops (red).
    ├ disparity_map/ 97 GB 74,478 EXR Left and right disparity maps computed over the rectified artifact-free crops.
    ├ disparity_map_
visualization_gifs/[pairs] 		9 GB 74,478 PNG Visualization of the disparity maps via colormap.
    ├ disparity_information/ 294 GB 74,478 EXR Left and right disparity information maps.
    ├ disparity_information_
visualization_gifs/[pairs] 		7 GB 74,478 PNG Visualization of disparity information.
├ metadata_and_camera_calibration/ 60 MB
    ├ metadata.json 30 MB 1 JSON Metadata including description, date, subjects, etc.
    ├ rectification_matrices_for_
artifact-free_stereo_pairs.pkl 		11 MB pickle Rectification matrices for the artifact-free image crops.
    ├ rectification_matrices_for_
full_stereo_pairs.pkl 		11 MB pickle Rectification matrices for the left and right full image regions.
    ├ principal_point_for_full_stereo_images.json 3 MB 1 JSON Approximate left and right principal points in the rectified full image regions.
    ├ principal_point_for_artifact-free_stereo_images.json 7 MB 1 JSON Approximate left and right principal points in the rectified artifact-free crops.
└ zips/ 557 GB ZIP Contents of each folder as a ZIP archive.

[Tips] We recommend using rclone if you want to download files from Google Drive via command line.

Download Command

Disparity Information


# 1) Download all disparity_information files, i.e. disparity_information.z01...
# 2) run the following command in the same directory
zip -F disparity_information.zip --out disparity_information_all.zip
unzip disparity_information_all.zip

Data Format

Metadata

metadata.json saves metadata as follows. 

{
    ...
    ,
    "30120": {                                                                  # Image index
                                                                                # Info about the original stereograph:
        "Webpage": "http://ucr.emuseum.com/view/objects/asitem/3631/1130",       # - url to the original webpage of the stereograph
        "Creator": "George Lewis",                                               # - creator
        "Publisher": "Keystone View Company",                                    # - publisher
        "Title": "The Reconstructed Cathedral, Ypres, Belgium. April 1933 service. (Inscription)", # - title
        "Date": "October 30, 1931",                                              # - date of capture
        "Medium": "Gelatin silver contact print",                                # - photographic medium
        "Credit Line": "Keystone-Mast Collection, UCR/California Museum of Photography, University of California, Riverside", # - credit line
        "Accession Number": "1996.0009.25589",                                   # - accession number
        "Inscriptions": "The Reconstructed Cathedral, Ypres, Belgium. April 1933 service.", # - inscription
        "Place Depicted": "Europe, Belgium, Flanders, Ypres",                    # - place depicted
        "Subjects": ["Cathedrals"]                                               # - subjects in the stereograph
    },
    ...
}

Stereo Camera Rectification

The stereo rectification matrices for artifact-free (rectification_matrices_for_ artifact-free_stereo_pairs.pkl) and full (rectification_matrices_for_ full_stereo_pairs.pkl) stereo pairs are stored in the following format as pickle files. 

{
    ...,
    '04733': {                                  # Image index
    'degenerate': False,                            # Whether the rectification is degenerate (see our paper for more details)
    'Hs': HomographyPair(
    Hl=array([[ 9.99999971e-01,  2.42102523e-04, -3.53605244e+01],      # Left homography
                [-2.42102523e-04,  9.99999971e-01,  8.56087440e-03],
                [ 0.00000000e+00,  0.00000000e+00,  1.00000000e+00]]), 
    Hr=array([[ 0.9952777 ,  0.00452081, -2.79633309],          # Right homography
                [-0.00452081,  0.9952777 , -2.30686815],
                [ 0.        ,  0.        ,  1.        ]]))
    },
    ...
}

Let ul and ur be the matching pixel coordinates in the original cropped artifact-free / full images. Then you can transform them into rectified coordinates as follows. Here we take the rectification for artifact-free stereo pairs as an example. 

import pickle
with open('rectification_matrices_for_artifact-free_stereo_pairs.pkl', 'rb') f:
    rectifications = pickle.load(f)
# HomographyPair namedtuple
Hs = rectifications['04733']['Hs']

# ul, ur are homogeneous pixel coordinates (x, y, 1).
ul_rectified = Hs.Hl.dot(ul)
ur_rectified = Hs.Hr.dot(ur)

Principal Point

The approximate principal points for the rectified full (principal_point_for_full_stereo_images.json) and artifact-free (principal_point_for_artifact-free_stereo_images.json) crops are stored as follows. 

{
    ...
    "04733": [
        [251.21584313764322, 315.4391800108721],    # left principal point [cx, cy]
        [268.8252675791312, 305.49944827771037],    # right principal point [cx, cy]
    ],
    ...
}