Immediate Insight API
The APIs: Getting Data Out .......................................................................................................................... 2
HTTPS Query API ....................................................................................................................................... 2
REST API Examples ................................................................................................................................ 3
API Credentials ...................................................................................................................................... 4
Command Line Client ................................................................................................................................ 6
Installation ............................................................................................................................................ 6
Command Line Options......................................................................................................................... 7
Command Line Examples ...................................................................................................................... 7
Custom Feeds ............................................................................................................................................ 9
Sept 2015
Immediate Insight API Guide (page 1)
© 2015 FireMon LLC
The APIs: Getting Data Out
Besides interactive search and the export button, there are a variety of programmatic ways to get data
out of the system. These include:



HTTPS Query
Linux Command Line client, for use in scripting
Custom Feeds in the data router
HTTPS Query API
The Immediate Insight REST API lets you query the search engine from scripts or other products by
sending URLs.
A basic query looks something like this. You can type it into your browser to try it and see the results:
https://ip-address:3201/search?k=api-key&q=query
See the API Examples section below for more.
Queries are sent over HTTPS to port 3201 with a REST endpoint of /search . The API Key functions as a
password. You can define multiple API Keys.
API Parameters
q : search term (default = all records)
n : number of search results to return (default = 10)
endtime : end time as either a JavaScript time integer or ii human readable form (default = "now")
starttime : end time as either a JavaScript time integer or ii human readable form (default = "1 hour
ago")
time : length of time as either a JavaScript time integer or ii human readable form (default = "1 hour")
overrides starttime
k : API Key (default = none , note: use “api-key” to match the pre-configured API Key in API Credentials)
f : format text|json|html|csv (default = text)
d : type of data info|events|summary|full|timeline (default = events)
type : the field to return summary or event info about (default = entities, fulltext)
Sept 2015
Immediate Insight API Guide (page 2)
© 2015 FireMon LLC
REST API Examples
Get the 10 most recent records:
https://ip-address:3201/search?k=api-key
Get the 10 most recent records with the word root:
https://ip-address:3201/search?k=api-key&q=root
Get the 10 most recent records from the logfile "firewall.log":
https://ip-address:3201/search?k=api-key&q=sourceFile:firewall.log
Get the 50 most recent records from the logfile "firewall.log":
https://ip-address:3201/search?k=api-key&q=sourceFile:firewall.log&n=50
Display them in a browser-readable format:
https://ip-address:3201/search?k=api-key&q=firewall.log&n=50&f=html
Return them in a spreadsheet-readable format (csv):
https://ip-address:3201/search?k=api-key&q=firewall.log&n=50&f=csv
Return them with additional metadata in a software-friendly format (json):
https://ip-address:3201/search?k=api-key&q=firewall.log&n=50&f=json
Display all records matching the search "error" for the last 2 minutes (up to 250):
https://ip-address:3201/search?k=api-key&q=error&n=250&starttime=now-2m&endtime=now
Display the last 10 records matching the search "error" over the last hour:
https://ip-address:3201/search?k=api-key&q=error&starttime=now-1h&endtime=now&d=events
Display just the count of records matching the search "error" over the last hour:
https://ip-address:3201/search?k=api-key&q=error&starttime=now-1h&endtime=now&d=info
Return the count of records matching the search "error" each minute over the last hour (suitable for a
graph):
https://ip-address:3201/search?k=api-key&q=error&starttime=now-1h&endtime=now&d=timeline
Return both the event text and the metadata for the last 10 records matching the search "error" over
the last hour:
https://ip-address:3201/search?k=api-key&q=error&starttime=now-1h&endtime=now&n=10&d=full
Return a count for each of the top 100 entities in that search:
https://ip-address:3201/search?k=api-key&q=error&starttime=now1h&endtime=now&n=100&d=summary
Sept 2015
Immediate Insight API Guide (page 3)
© 2015 FireMon LLC
API Credentials
The REST endpoint for API Queries is /search. The default port number is 3201 and protocol is HTTPS.
Note: HTTPS access must be enable in Immediate Insight CLI before API calls can be made;
Type set-ssl command to enable encryption on browser sessions.
Type reload server to make changes take effect.
Quit browser and re-login using https instead of http (https://ip-address-of-server:3201)
The API Key functions as a password. You can have multiple API Keys active at once. The default API
Key is "api-key".
API Keys are defined in app/config/marshal-settings.conf You can edit this file to add or change them.
Options are:
Default:
API.key = "api-key"
No Key Required:
API.key = none
Specify an API Key (quotes are optional):
API.key = my-secret-key
Set several API Keys
API.key = ["key1","key2","key3"]
Disable the REST API entirely
API.enable = false (default value is true)
Restart the marshal for changes to take effect by using the restart command:
restart
Sept 2015
Immediate Insight API Guide (page 4)
© 2015 FireMon LLC
Since REST API calls are over HTTPS, it is convenient (& suggested) to enable certificate keys (see
example for Chrome below)
Certificates
We recommend the best practice use of matching CA certs installed in user’s browsers to reduce the possibility of
man-in-the-middle attacks and provide a smoother user experience.
During installation, a self-signed rootCA pair is generated automatically in app/config/certs.
(Note You can replace this pair with your own CA by overwriting the rootCA.key and rootCA.pem files,
however this is an advanced task - most can use the self-signed certs provided)
Type "set-certs" followed by "reload server" to activate the certificate
Next, copy app/config/certs/rootCA.pem file from the Immediate Insight server to your computer (using an SFTP
client)
Load the Certificate into your Browser (see instructions for Chrome below)
In Chrome Brower go to
Settings -> Show Advanced Settings -> HTTPS/SSL -> Manage Certificates
Trusted Root Certification Authorities -> Import (specify rootCA.pem file)
Restart Chrome browser - next time you log into Immediate Insight you should not see cert warning
Note: While the system has a reasonable set of security measures in place, the present release is designed to run in
a secure and trusted environment. If you have a need to expose it directly to the Internet, please call us to discuss
additional hardening procedures.
Sept 2015
Immediate Insight API Guide (page 5)
© 2015 FireMon LLC
Command Line Client
The command line client lets you issue search queries from a Linux shell script or terminal session. You
can run the script from the Immediate Insight console and also download it to remote systems.
The command line client uses the REST API. See the API Credentials section for details on how to create
API keys. The default API Key is "api-key".
Installation
To put the command line client script on another system (or expose the script on your own Immediate
Insight system), you first download it from the appliance:
For example:
1. Download it from a linux shell:
wget --no-check-certificate https://ip-address:3201/tools/ii
or
curl -o ii -k https://ip-address:3201/tools/ii
It is intentionally designed as a simple shell script so that you can customize or modify it for your
needs. You will need to have the curl http client utility installed on your system. Most full linux
distributions include this by default and also in their repositories:
sudo yum install curl or sudo apt-get install curl
2. Mark the script as executable:
chmod +x ii
3. Edit the first lines of the ii script (e.g. with vi or nano) to set your server address and api-key (if
you wish).
For convenience, the default server address and default API Key are set in the first few
lines of the file. You can edit it and modify these for your own environment to eliminate
the need to specify them on every query.
KEY="api-key" (substitute other api key if you are using a different one)
SERVER="localhost" (substitute ip address of II server if accessing remotely)
Sept 2015
Immediate Insight API Guide (page 6)
© 2015 FireMon LLC
Command Line Options
-h : Print a list of options
-q : The search query [default=*]
-n : Number of results to return [default=10]
-t : Time scope [default="now"]
-a : Start time [default="now"]
-b : End time [default="now"]
Time examples: "now", "now-1h", "now-1d-4h-5m" "1361683874554"
-f : Format [text|json|html|csv]
[default=”text”]
-d : Data type [info|events|summary|full|timeline]
[default=”events”]
events returns just the original text of the event
full returns the event text and also the metadata such as source, arrival time, and correlations
info returns just a count of matching records
timeline returns count-per-minute data over the time period
summary returns a list of entities and their counts
-s : Server address [default="localhost", or whatever you substituted when editing ii script]
-k : API Key [default="api-key", or whatever you substituted when editing ii script]
Command Line Examples
Get the first 10 results matching the query from server 192.168.0.1 with credential “api-key”
ii -s 192.168.0.1 –k api-key -q "search terms" -n 10
If you already set the default address and api key for your server and placed the ii file in your executable
path, this would shorten to:
ii -q "search terms" -n 10
You can pipe or direct the output in typical unix shell fashion. For example, this would get the most
recent 100 lines from the source “fw.log”, split fields based on colon, select the value from the third
field, and provide a count of each unique value:
ii -q "fw.log" -n 100 | cut -d ":" -f 3 | sort | uniq –c
Get just a count of the number of hits:
ii -q "search terms" -d info
Get just a count of the total number of hits for the last 5 minutes:
ii -q "search terms" -d info -a now-5m -b now
Sept 2015
Immediate Insight API Guide (page 7)
© 2015 FireMon LLC
Get just a count of the number of hits for the last 5 minutes, on a per-minute basis
ii -q "search terms" -d timeline -a now-5m -b now
Get a list of the entities matching the search, and the number of hits for each:
ii -q "search terms" -d summary
Get the full data (message + entities + correlations + source, etc.) for the most recent 20 records
matching the search:
ii -q "search terms" –n 20 -d full
Sept 2015
Immediate Insight API Guide (page 8)
© 2015 FireMon LLC
Custom Feeds
The Data Router lets you re-stream selected records to disk or to the network to create your own
custom feed. Data routes act on new records immediately as they arrive.
Go to DataFlow > Data Routing and create a new route by clicking on the plus icon in the upper right.
Use the feed action to write a stream to disk of all records matching your criteria.
In this example, all records that have been tagged with #SMTP will be written to a file called
myCustomStream.log on the network share that was mounted at /media/logs. You could use
autoTagging to tag the different types of records you want to go into the feed. Instead of matching on
the field “tag”, another option would be to match just on the text of the record to directly select all
records containing the word SMTP (leave “field” blank). You can have more than one route sending data
to the same feed.
The feed action takes one property – the name of the file to write to. Use the path where you mounted
the network share you wish to write to
feed(“/media/logs/myFilename.log”)
Sept 2015
Immediate Insight API Guide (page 9)
© 2015 FireMon LLC