The files here are the data collected in my NTP survey of 1999. For more information, please see the paper.
All files are in bzip2 format for maximum compression and recoverability. Please save these files without viewing them first in your browser, by shift-clicking or selecting "save to disk". Otherwise your browser may corrupt them. The best too to download things like this is the command line utility wget.
The data here comes from an exhaustive "spider" search of the NTP network. For each of 175,527 NTP hosts, system information, peer list, and monitor list data was collected.
For each host, the following command was run: /usr/sbin/xntpdc -n -c sysinfo -c peers -c monlist <IPADDRESS>
The output of this program was parsed and all IP addesses in the peers and monlist report were extracted. These were fed back into the NTP spider process, a multithreaded Java program, which added new hosts to the job list. The spider code itself should be available from the same place as this data archive.
Note that this dataset is only a subset of the entire NTP network. All NTP servers do not necessarily respond to xntpdc queries, it's not the standard protocol. Other hosts might have been down at the time of the survey, or behind a firewall, or in a network entirely disconnected from this starting set. A complete survey is likely impossible.
The survey host was pinotnoir.media.mit.edu, 18.85.16.104. It was connected to a relatively quiet 10Mbps ethernet segment, and from there to a fast Internet connection. I do not believe network outages greatly affected the survey, although it is possible.
The survey was run on a Redhat 6.0 system running Linux 2.2.10. xntpdc was from the Redhat package xntp3-5.93-12.src.rpm, itself compiled from xntpdc 3-5.93e. xntpdc was dynamically linked to GNU libc-2.1.1.
The survey was started November 21, 1999 and ran intermittently through November 28, 1999. The actual running time was roughly 100 hours: downtime was because of a memory management problem in the spider that took a few days to fix.
I believe strongly in doing Internet research and sharing data and information in a free and open way. I am providing this data to the research community at large to make use of however they see fit.
All data collected here is publically available. However, please understand that many people do not realize this information is available. Treat them politely.
I only request that if you write a paper based on this data, please do me the courtesy of acknowledging where the data came from. Please cite my paper as well, if appropriate. And tell me about your work, I'd love to hear!
This data is distributed with no warranty, without even the implied warranty of merchantability or fitness for a particular purpose.
| Nelson Minar | Created: December 9, 1999 |
| <nelson@media.mit.edu> | Updated: February 16, 2000 |