Broker HTTP API

The BGPStream Broker is a web application that provides a unified query interface to retrieve meta-data about data available from different data providers. See the data providers page for up-to-date information about the data held by the broker (this page is dynamically generated using the broker metadata API described below).

CAIDA operates a publicly accessible instance of the Broker that libBGPStream is configured to query by default:

Public Broker URL https://bgpstream.caida.org/broker

The Broker provides two query APIs: data and metadata. Metadata queries retrieve information about the Data Providers and Collectors that the broker knows about. Data queries on the other hand, retrieve information about data files known to the broker.

See the BGPStream paper for more details about how libBGPStream queries the broker.


Metadata API /meta

Data Provider Metadata /meta/projects/{project}

URL Parameters:

Parameter Description Type Values Default
{project} Optionally filter results to a single project String routeviews, ris all projects

GET Parameters:

Parameter Description Type Values Default
human Pretty-printed JSON output Boolean none false

Example Result https://bgpstream.caida.org/broker/meta/projects/ris?human

{
  "time": 1445363855,
  "type": "meta",
  "error": null,
  "queryParameters": {
    "human": true,
    "project": "ris"
  },
  "data": {
    "projects": {
      "ris": {
        "collectors": {
          "rrc00": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "939513960",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "936355260",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc01": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "964687020",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "964623600",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc02": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "985398000",
                "latestDumpTime": "1222905540"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "985376400",
                "latestDumpTime": "1222920300"
              }
            }
          },
          "rrc03": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "979693560",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "979664760",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc04": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "988137720",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "988108920",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc05": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "992450100",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "992421300",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc06": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "999184080",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "999161580",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc07": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1017953460",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1017935460",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc08": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1020793980",
                "latestDumpTime": "1094112000"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1020765180",
                "latestDumpTime": "1094137200"
              }
            }
          },
          "rrc09": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1052601300",
                "latestDumpTime": "1075910400"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1052570640",
                "latestDumpTime": "1075928340"
              }
            }
          },
          "rrc10": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1067644800",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1067644800",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc11": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1076688000",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1076860800",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc12": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1089072000",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1089023700",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc13": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1119628800",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1117807800",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc14": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1104537600",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1104537600",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc15": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1134576000",
                "latestDumpTime": "1445356800"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1136284200",
                "latestDumpTime": "1445363400"
              }
            }
          },
          "rrc16": {
            "project": "ris",
            "dataTypes": {
              "ribs": {
                "dumpPeriod": 28800,
                "dumpDuration": 120,
                "oldestDumpTime": "1201852740",
                "latestDumpTime": "1331798340"
              },
              "updates": {
                "dumpPeriod": 300,
                "dumpDuration": 300,
                "oldestDumpTime": "1201824000",
                "latestDumpTime": "1331811900"
              }
            }
          }
        }
      }
    }
  }
}

Collector Metadata /meta/collectors/{collector}

URL Parameters:

Parameter Description Type Values Default
{collector} Optionally filter results to a single collector String See data providers all collectors

GET Parameters:

Parameter Description Type Values Default
human Pretty-printed JSON output Boolean none false

Example Result https://bgpstream.caida.org/broker/meta/collectors/route-views2?human

{
  "time": 1445365080,
  "type": "meta",
  "error": null,
  "queryParameters": {
    "human": true,
    "collector": "route-views2"
  },
  "data": {
    "collectors": {
      "route-views2": {
        "project": "routeviews",
        "dataTypes": {
          "ribs": {
            "dumpPeriod": 7200,
            "dumpDuration": 120,
            "oldestDumpTime": "1004114880",
            "latestDumpTime": "1445364000"
          },
          "updates": {
            "dumpPeriod": 900,
            "dumpDuration": 900,
            "oldestDumpTime": "1004102520",
            "latestDumpTime": "1445364000"
          }
        }
      }
    }
  }
}

Data API /data

GET Parameters:

Parameter Description Type Values Default
collector Collector to retrieve data for String See data providers all collectors
collectors List of collectors to retrieve data for Array See data providers all collectors
project Project to retrieve data for String See data providers all projects
projects List of projects to retrieve data for Array See data providers all projects
intervals List of time intervals Array {epoch-start},{epoch-end} required parameter
type Data type to retrieve String ribs, updates all types
types Data types to retrieve Array ribs, updates all types
minInitialTime Start time of the current window Epoch seconds Should be set to maximum of (initialTime + duration) from previously returned results earliest interval start time
dataAddedSince Last request time Epoch seconds Should be set to the value of the time attribute in the previous result none
human Pretty-printed JSON output Boolean none false

Example Result
https://bgpstream.caida.org/broker/data?human&intervals[]=1438819200,1438819200&collectors[]=route-views2&collectors[]=rrc03&types[]=updates

{
  "time": 1445366456,
  "type": "data",
  "error": null,
  "queryParameters": {
    "human": true,
    "projects": [],
    "project": null,
    "collectors": [
      "route-views2",
      "rrc03"
    ],
    "collector": null,
    "intervals": [
      "1438819200,1438819200"
    ],
    "types": [
      "updates"
    ],
    "type": null,
    "minInitialTime": null,
    "dataAddedSince": null
  },
  "data": {
    "dumpFiles": [
      {
        "urlType": "simple",
        "url": "http:\/\/archive.routeviews.org\/bgpdata\/2015.08\/UPDATES\/updates.20150805.2345.bz2",
        "project": "routeviews",
        "collector": "route-views2",
        "type": "updates",
        "initialTime": 1438818300,
        "duration": 900
      },
      {
        "urlType": "simple",
        "url": "http:\/\/data.ris.ripe.net\/rrc03\/2015.08\/updates.20150805.2355.gz",
        "project": "ris",
        "collector": "rrc03",
        "type": "updates",
        "initialTime": 1438818900,
        "duration": 300
      }
    ]
  }
}