NMAP Database Output : XML TO SQL

SQL support has been a much requested feature of NMAP in the Redspin office. While a number of tools exist to support NMAP SQL output, their database format has left much to be desired. Using SQLite, Perl’s DB and the NMAP Parser module, our tool extracts all supported fields in an NMAP XML file and creates the following database format:

TABLE nmap (
    sid INTEGER PRIMARY KEY AUTOINCREMENT,
    version TEXT,
    xmlversion TEXT,
    args TEXT,
    types TEXT,
    starttime INTEGER,
    startstr TEXT,
    endtime INTEGER,
    endstr TEXT,
    numservices INTEGER) 

TABLE hosts (
    sid INTEGER,
    hid INTEGER PRIMARY KEY AUTOINCREMENT,
    ip4 TEXT,
    ip4num INTEGER,
    hostname TEXT,
    status TEXT,
    tcpcount INTEGER,
    udpcount INTEGER,
    mac TEXT,
    vendor TEXT,
    ip6 TEXT,
    distance INTEGER,
    uptime TEXT,
    upstr TEXT) 

TABLE sequencing (
    hid INTEGER,
    tcpclass TEXT,
    tcpindex TEXT,
    tcpvalues TEXT,
    ipclass TEXT,
    ipvalues TEXT,
    tcptclass TEXT,
    tcptvalues TEXT) 

TABLE ports (
    hid INTEGER,
    port INTEGER,
    type TEXT,
    state TEXT,
    name TEXT,
    tunnel TEXT,
    product TEXT,
    version TEXT,
    extra TEXT,
    confidence INTEGER,
    method TEXT,
    proto TEXT,
    owner TEXT,
    rpcnum TEXT,
    fingerprint TEXT) 

TABLE os (
    hid INTEGER,
    name TEXT,
    family TEXT,
    generation TEXT,
    type TEXT,
    vendor TEXT,
    accuracy INTEGER)

The resulting database can then be queried directly using SQLite in order to extract the relevant information. The tool also prints output in the following format sorted by IP and PORT as show below:

$ nmap -A -T4 scanme.nmap.org -oX scanme >/dev/null
$ nmap_xml2sql.pl scanme | grep -v "^#"
64.13.134.52 (scanme.nmap.org)	53/tcp	domain
64.13.134.52 (scanme.nmap.org)	80/tcp	http

Compare this database structure with the other similar XML2SQL approaches:
NMAP-SQL: http://sourceforge.net/projects/nmapsql/ outdated – only supports NMAP 3.75 and MySQL:

table portstat, table runlist, table targets

NMAP-Parser (nmap2db.pl): http://search.cpan.org/dist/Nmap-Parser/ MySQL and SQLite support:

table hosts(
  ip              VARCHAR(15) PRIMARY KEY NOT NULL,
  mac             VARCHAR(17),
  status          VARCHAR(7) DEFAULT 'down',
  hostname        TEXT,
  open_ports      TEXT,
  filtered_ports  TEXT,
  osname	      TEXT,
  osfamily        TEXT,
  osgen           TEXT,
  last_scanned    TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  UNIQUE (ip))

PBNJ -x option: http://pbnj.sourceforge.net/ MySQL and SQLite support:

table machines(
  mid             PRIMARY KEY AUTOINCREMENT,
  ip             	TEXT,
  host          	TEXT,
  localh        	INTEGER,
  os      		TEXT,
  machine_created TEXT,
  created_on      TEXT)

table services(
  mid             INTEGER,
  service         TEXT,
  state          	TEXT,
  port        	INTEGER,
  protocol      	TEXT,
  version 		TEXT,
  banner      	TEXT,
  machine_updated TEXT,
  updated_on      TEXT)

The tool has been released under a NMAP equivalent license as well as a Fyodor may-use-as-he-pleases license, and can be download here: nmap_xml2sql.pl

Requirements:

Posted on by The Shell Shakespear

About The Shell Shakespear

The Shell Shakespear will eat your 100 line perl script and return to you a beautiful haiku of bash.

4 Responses to NMAP Database Output : XML TO SQL

  1. BG mail

    I have recently started a blog, the information you provide on this site has helped me tremendously. Thank you for all of your time & work.

  2. Pingback: Vulnerability Assessment Testing Automation | SANS | danielmiessler.com

  3. Pingback: Ruby, Nmap XML, and Databases « c0llateral Blog

  4. Pingback: Brinhosa on Security » Resultados do Nmap para o Banco de Dados

Add a Comment