The data and meta-data access layer provides information about available BGP data. It also provides data annotations, e.g., the collection project and router that obtained the data.
We provide four data interfaces to identify BGP data available for processing:
The BGPStream Broker is a web service that provides a unified query interface to retrieve streams of data from different public Data Providers (i.e., Route Views, RIPE RIS). The broker interface enables several key features of BGPStream:
CAIDA operates a publicly accessible instance of the Broker that libBGPStream is configured to query by default. See the query API documentation for information about using the broker from third-party applications.
Single file allows the user to access a single local or remote (via HTTP) RIB dump and/or Update dump (similar to how the bgpdump tool operates). See BGPReader for more information about processing a single dump file.
For users with a small set of local/private MRT dump files, the CSV file interface may be appropriate. Simply create a CSV file in the following format (with one line per file):
The following line provides the meta-data for Route Views Updates dump which is stored in a local directory. The dump was generated by the Jinx collector and contains updates in the interval 1427846400 to 1427846400 + 900 (i.e. 15 minutes worth of updates).
The SQLite interface is similar to the CSV file interface, but can be used for larger data sets, as well as providing better support for live mode (that is, updating the database with new files as they are available).
In order to facilitate the insertion of data into an SQLite database with an approprate schema, we provide a script (tools/bgpstream_sqlite_mgmt.py) that adds new dump meta-data to the database.
bgpstream_sqlite_mgmt.py inserts new dump's meta-data each time it is invoked, creating the database file if it does not exist.
usage: bgpstream_sqlite_mgmt.py [-h] [-l] [-M ADD_MRT_FILE] [-p PROJ] [-c COLL] [-t BGP_TYPE] [-T FILE_TIME] [-u UPDATES_TIME_SPAN] sqlite_db positional arguments: sqlite_db file containing the sqlite database optional arguments: -h, --help show this help message and exit -l, --list_files list the mrt files in the database -M ADD_MRT_FILE, --add_mrt_file ADD_MRT_FILE path to the mrt file to add to the database -p PROJ, --proj PROJ bgp project -c COLL, --coll COLL bgp collector -t BGP_TYPE, --bgp_type BGP_TYPE bgp type -T FILE_TIME, --file_time FILE_TIME time associated with the mrt file -u UPDATES_TIME_SPAN, --updates_time_span UPDATES_TIME_SPAN updates time span