Splunk HTTP Event Collector Python 3 Example

With Splunk’s latest release of version 6.3 a new feature called HTTP Event Collector has been added. It allows for sending JSON formatted data to Splunk via an HTTP call. I won’t go into all the details of this feature in this post, but for the curious more information can be found here.

This feature is great for anyone who wants to easily get data into Splunk using their own scripts. With this being a new feature there is not yet many examples of how to use this on the scripting side. In this post I want to provide an example in Python that others can use to build upon in their own code.

Below is a short and documented example using the urllib library to craft an HTTP request that Splunk’s HTTP Event Collector will accept.

import urllib.request
import json

def send_event(splunk_host, auth_token, log_data):
   """Sends an event to the HTTP Event collector of a Splunk Instance"""
   
   try:
      # Integer value representing epoch time format
      event_time = 0
      
      # String representing the host name or IP
      host_id = "localhost"
      
      # String representing the Splunk sourcetype, see:
      # docs.splunk.com/Documentation/Splunk/6.3.2/Data/Listofpretrainedsourcetypes
      source_type = "access_combined"
      
      # Create request URL
      request_url = "http://%s:8088/services/collector" % splunk_host
      
      post_data = {
         "time": event_time, 
         "host": host_id,
         "sourcetype": source_type,
         "event": log_data
      }
      
      # Encode data in JSON utf-8 format
      data = json.dumps(post_data).encode('utf8')
      
      # Create auth header
      auth_header = "Splunk %s" % auth_token
      headers = {'Authorization' : auth_header}
      
      # Create request
      req = urllib.request.Request(request_url, data, headers)
      response = urllib.request.urlopen(req)
      
      # read response, should be in JSON format
      read_response = response.read()
      
      try:
         response_json = json.loads(str(read_response)[2:-1])
         
         if "text" in response_json:
            if response_json["text"] == "Success":
               post_success = True
            else:
               post_success = False
      except:
         post_success = False
      
      if post_success == True:
         # Event was recieved successfully
         print ("Event was recieved successfully")
      else:
         # Event returned an error
         print ("Error sending request.")
      
   except Exception as err:
      # Network or connection error
      post_success = False
      print ("Error sending request")
      print (str(err))

   return post_success

def main():
   splunk_auth_token = "00000000-0000-0000-0000-000000000000"
   splunk_host = "10.11.12.13"
   
   log_data = {
      "data_point_1": 50,
      "data_point_2": 20,
   }
   
   result = send_event(splunk_host, splunk_auth_token, log_data)
   print (result)

main()

A few things to note: this example is not using SSL, so the Enable SSL check box in the HTTP Event Collector global settings must be unchecked. Also Splunk is picky about the top level JSON keys, only a few specific keys can be used. Those keys are: time, host, source, sourcetype, index and event. All custom data should be under the event key. Finally this code should work in all versions of Python after 3.0.