NMAP Database Output : XML TO SQL

SQL support has been a much requested feature of NMAP in the Redspin office. While a number of tools exist to support NMAP SQL output, their database format has left much to be desired. Using SQLite, Perl’s DB and the NMAP Parser module, our tool extracts all supported fields in an NMAP XML file and creates the following database format:

TABLE nmap (
    sid INTEGER PRIMARY KEY AUTOINCREMENT,
    version TEXT,
    xmlversion TEXT,
    args TEXT,
    types TEXT,
    starttime INTEGER,
    startstr TEXT,
    endtime INTEGER,
    endstr TEXT,
    numservices INTEGER) 

TABLE hosts (
    sid INTEGER,
    hid INTEGER PRIMARY KEY AUTOINCREMENT,
    ip4 TEXT,
    ip4num INTEGER,
    hostname TEXT,
    status TEXT,
    tcpcount INTEGER,
    udpcount INTEGER,
    mac TEXT,
    vendor TEXT,
    ip6 TEXT,
    distance INTEGER,
    uptime TEXT,
    upstr TEXT) 

TABLE sequencing (
    hid INTEGER,
    tcpclass TEXT,
    tcpindex TEXT,
    tcpvalues TEXT,
    ipclass TEXT,
    ipvalues TEXT,
    tcptclass TEXT,
    tcptvalues TEXT) 

TABLE ports (
    hid INTEGER,
    port INTEGER,
    type TEXT,
    state TEXT,
    name TEXT,
    tunnel TEXT,
    product TEXT,
    version TEXT,
    extra TEXT,
    confidence INTEGER,
    method TEXT,
    proto TEXT,
    owner TEXT,
    rpcnum TEXT,
    fingerprint TEXT) 

TABLE os (
    hid INTEGER,
    name TEXT,
    family TEXT,
    generation TEXT,
    type TEXT,
    vendor TEXT,
    accuracy INTEGER)

The resulting database can then be queried directly using SQLite in order to extract the relevant information. The tool also prints output in the following format sorted by IP and PORT as show below:

$ nmap -A -T4 scanme.nmap.org -oX scanme >/dev/null
$ nmap_xml2sql.pl scanme | grep -v "^#"
64.13.134.52 (scanme.nmap.org)	53/tcp	domain
64.13.134.52 (scanme.nmap.org)	80/tcp	http

Compare this database structure with the other similar XML2SQL approaches:
NMAP-SQL: http://sourceforge.net/projects/nmapsql/ outdated – only supports NMAP 3.75 and MySQL:

table portstat, table runlist, table targets

NMAP-Parser (nmap2db.pl): http://search.cpan.org/dist/Nmap-Parser/ MySQL and SQLite support:

table hosts(
  ip              VARCHAR(15) PRIMARY KEY NOT NULL,
  mac             VARCHAR(17),
  status          VARCHAR(7) DEFAULT 'down',
  hostname        TEXT,
  open_ports      TEXT,
  filtered_ports  TEXT,
  osname	      TEXT,
  osfamily        TEXT,
  osgen           TEXT,
  last_scanned    TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  UNIQUE (ip))

PBNJ -x option: http://pbnj.sourceforge.net/ MySQL and SQLite support:

table machines(
  mid             PRIMARY KEY AUTOINCREMENT,
  ip             	TEXT,
  host          	TEXT,
  localh        	INTEGER,
  os      		TEXT,
  machine_created TEXT,
  created_on      TEXT)

table services(
  mid             INTEGER,
  service         TEXT,
  state          	TEXT,
  port        	INTEGER,
  protocol      	TEXT,
  version 		TEXT,
  banner      	TEXT,
  machine_updated TEXT,
  updated_on      TEXT)

The tool has been released under a NMAP equivalent license as well as a Fyodor may-use-as-he-pleases license, and can be download here: nmap_xml2sql.pl

Requirements:

Posted on by The Shell Shakespear Posted in Redspin Labs | Leave a comment

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>