McAfee Active Response (MAR) Search Sample

This sample queries McAfee Active Response for the IP addresses of hosts that have an Active Response client installed.

Prerequisites

• The samples configuration step has been completed (Samples Configuration)
• A McAfee Active Response (MAR) Service is available on the DXL fabric
• The python client has been authorized to perform MAR searches (see Authorize Client to Perform MAR Search)

To run this sample execute the sample\mar\search_example.py script as follows:

c:\dxlclient-python-sdk-5.6.0.4>python sample\mar\search_example.py

To perform an Active Response query via DXL the following steps must be performed:

• Create a MAR search by sending a search query (via a dxlclient.message.Request message) to the MAR service
• Extract the search identifier from the dxlclient.message.Response message received in response to the query creation request
• Start the search by sending an appropriate dxlclient.message.Request message to the MAR service
• Poll the MAR service for the status of the search by sending dxlclient.message.Request messages to obtain the search status (via dxlclient.message.Response messages)
• Once the search has completed, query the MAR service for the results of the search (via a dxlclient.message.Request message)
• Process the results of the search query in the dxlclient.message.Response message received corresponding to the results request

All of the MAR query-related dxlclient.message.Request messages described in the steps above are sent to the same DXL topic (/mcafee/mar/service/api/search).

Each of these steps are performed in the sample script and will be described in detail throughout the rest of this document. All of the steps utilize a local convenience method within the script (execute_mar_search_api()). This method is responsible for hiding the details of converting Python dictionary objects to DXL request objects and extracting dictionaries from DXL response objects. The method also hides the details of sending synchronous requests to the MAR service and handling any errors that may occur. The execute_mar_search_api() method is described in detail at the bottom of this document.

The first step is to create a MAR query as shown below. In this particular case the query will be for the IP addresses (ip_address) of any hosts (HostInfo) that have an Active Response client installed.

# Create the search
response_dict = execute_mar_search_api(
    client,
    {
        "target": "/v1/simple",
        "method": "POST",
        "parameters": {},
        "body": {
            "aggregated": True,
            "projections": [
                {
                    "name": "HostInfo",
                    "outputs": ["ip_address"]
                }
            ]
        }
    }
)

As shown below, the code block above results in a dxlclient.message.Request message being sent to the MAR service containing the query that is to be created. The dxlclient.message.Response message from the MAR service includes meta-information about the search that was created, including a status code (code) with a value of 201 (created) indicating that the creation was successful along with the identifier (id) of the search (this identifier will be used in subsequent steps).

Request:

{
    "body": {
        "aggregated": true,
        "projections": [
            {
                "name": "HostInfo",
                "outputs": [
                    "ip_address"
                ]
            }
        ]
    },
    "method": "POST",
    "parameters": {},
    "target": "/v1/simple"
}

Response:

{
    "body": {
        "aggregated": true,
        "catalogVersion": 0,
        "createdAt": 1474308184842,
        "dbVersion": 0,
        "expectedHostResponses": 0,
        "id": "57e02858e4b0217da8f65e80",
        "invalid": false,
        "projections": [ ... ],
        "running": false,
        "status": "CREATED",
        "temporal": true,
        "ttl": 60000
    },
    "code": 201
}

The next step (as shown below) extracts the identifier (id) of the newly created search from the response dictionary. This identifier is included in the next request that is sent to the MAR service requesting that the search be started.

# Get the search identifier
search_id = response_dict["body"]["id"]

# Start the search
execute_mar_search_api(
    client,
    {
        "target": "/v1/" + search_id + "/start",
        "method": "PUT",
        "parameters": {},
        "body": {}
    }
)

As shown below, the code block above results in a dxlclient.message.Request message being sent to the MAR service requesting that the search be started. The dxlclient.message.Response message from the MAR service includes a status code (code) with a value of 200 (OK) indicating that the search has been started.

Request:

{
    "body": {},
    "method": "PUT",
    "parameters": {},
    "target": "/v1/57e02858e4b0217da8f65e80/start"
}

Response:

{
    "body": {
        "aggregated": true,
        "catalogVersion": 1,
        "createdAt": 1474308184842,
        "dbVersion": 2,
        "executedAt": 1474308184964,
        "expectedHostResponses": 1,
        "id": "57e02858e4b0217da8f65e80",
        "invalid": false,
        "projections": [ ... ],
        "running": false,
        "startTime": 1474308184964,
        "status": "STARTED",
        "temporal": true,
        "ttl": 60000
    },
    "code": 200
}

The next step (as shown below) will poll the MAR service for the status of the executed search until it has reached a status of FINISHED.

# Wait until the search finishes
finished = False
while not finished:
    response_dict = execute_mar_search_api(
        client,
        {
            "target": "/v1/" + search_id + "/status",
            "method": "GET",
            "parameters": {},
            "body": {}
        }
    )
    finished = response_dict["body"]["status"] == "FINISHED"
    if not finished:
        time.sleep(5)

As shown below, the code block above results in one or more dxlclient.message.Request messages being sent to the MAR service requesting the status of the search. Each dxlclient.message.Response message from the MAR service includes the current status (status) of the search (STARTED, FINISHED, etc.).

Request:

{
    "body": {},
    "method": "GET",
    "parameters": {},
    "target": "/v1/57e02858e4b0217da8f65e80/status"
}

Response:

{
    "body": {
        "errors": 0,
        "hosts": 0,
        "results": 0,
        "status": "STARTED",
        "subscribedHosts": 0
    },
    "code": 200
}

Request:

{
    "body": {},
    "method": "GET",
    "parameters": {},
    "target": "/v1/57e02858e4b0217da8f65e80/status"
}

Response:

{
    "body": {
        "errors": 0,
        "hosts": 1,
        "results": 1,
        "status": "FINISHED",
        "subscribedHosts": 1
    },
    "code": 200
}

Once the search has completed, the next step is to obtain the results of the search from the MAR service (as shown in the code block below). In this particular case, the search results are being limited ($limit) to 10 results.

# Get the search results
# Results limited to 10, the API does support paging
response_dict = execute_mar_search_api(
    client,
    {
        "target": "/v1/" + search_id + "/results",
        "method": "GET",
        "parameters": {
            "$offset": 0,
            "$limit": 10,
            "filter": "",
            "sortBy": "count",
            "sortDirection": "desc"
        },
        "body": {}
    }
)

As shown below, the code block above sends a dxlclient.message.Request message to the MAR service indicating how results are to be received (filtered, sorted, limited, etc.). The corresponding dxlclient.message.Response message includes the search results (items) along with meta-information about the results (counts, paging-related information, etc.).

Request:

{
    "body": {},
    "method": "GET",
    "parameters": {
        "$limit": 10,
        "$offset": 0,
        "filter": "",
        "sortBy": "count",
        "sortDirection": "desc"
    },
    "target": "/v1/57e02858e4b0217da8f65e80/results"
}

Response:

{
    "body": {
        "currentItemCount": 1,
        "items": [
            {
                "count": 1,
                "created_at": "2016-09-19T18:03:07.722Z",
                "id": "{1=[10.84.200.99]}",
                "output": {
                    "HostInfo|ip_address": "10.84.200.99"
                }
            }
        ],
        "itemsPerPage": 10,
        "startIndex": 0,
        "totalItems": 1
    },
    "code": 200
}

The final code block in the script extracts the IP addresses from the search results (as shown below).

# Loop and display the results
print("Results:")
for result in response_dict['body']['items']:
    print("    " + result['output']['HostInfo|ip_address'])

The output should appear similar to the following:

Results:
    10.84.200.99

The major functionality provided by this sample resides in the execute_mar_search_api() method as shown below:

def execute_mar_search_api(client, payload_dict):
    """
    Executes a query against the MAR search api

    :param client: The DXL client
    :param payload_dict: The payload
    :return: A dictionary containing the results of the query
    """
    # Create the request message
    req = Request(CREATE_SEARCH_TOPIC)
    # Set the payload
    req.payload = json.dumps(payload_dict).encode(encoding="UTF-8")

    # Display the request that is going to be sent
    print("Request:\n" + json.dumps(payload_dict, sort_keys=True, indent=4, separators=(',', ': ')))

    # Send the request and wait for a response (synchronous)
    res = client.sync_request(req, timeout=30)

    # Return a dictionary corresponding to the response payload
    if res.message_type != Message.MESSAGE_TYPE_ERROR:
        resp_dict = json.loads(res.payload.decode(encoding="UTF-8"))
        # Display the response
        print("Response:\n" + json.dumps(resp_dict, sort_keys=True,
                                         indent=4, separators=(',', ': ')))
        if "code" in resp_dict:
            code = resp_dict['code']
            if code < 200 or code >= 300:
                if "body" in resp_dict and "applicationErrorList" in resp_dict["body"]:
                    error = resp_dict["body"]["applicationErrorList"][0]
                    raise Exception(error["message"] + ": " + str(error["code"]))
                elif "body" in resp_dict:
                    raise Exception(resp_dict["body"] + ": " + str(code))
                else:
                    raise Exception("Error: Received failure response code: " + str(code))
        else:
            raise Exception("Error: unable to find response code")
        return resp_dict
    else:
        raise Exception("Error: " + res.error_message + " (" + str(res.error_code) + ")")

This method creates a dxlclient.message.Request message that will be delivered to the search topic (/mcafee/mar/service/api/search) of a MAR service on the fabric. Prior to delivering the request, the dictionary specified as a method parameter (payload_dict) is converted to a JSON string and placed in the payload of the request message.

The request message is delivered to the fabric via the dxlclient.client.DxlClient.sync_request() method on the DXL client.

The payload of the dxlclient.message.Response message received is converted to a Python dictionary object. The status code (code) within the dictionary is examined to ensure that the request was successful. If the request was successful, the dictionary extracted from the response is returned to the caller of the method. The method will raise exceptions for any errors that occur during the request itself or during validation.