среда, 15 октября 2014 г.

Splunk TRANSACTION Command


Splunk definition of the command:

Given events as input, this command finds transactions based on events that meet various constraints. Transactions are made up of the raw text (the _raw field) of each member, the time and date fields of the earliest member, as well as the union of all other fields of each member.
Additionally, the transaction command produces two fields to the raw events, duration and eventcount. The duration value is the difference between the timestamps for the first and last events in the transaction. The eventcount value is the number of events in the transaction.


• A transaction is any group of related events that span time
• Events can come from multiple applications or hosts
- Events related to a single purchase from an online store can span across an
application server, database, and e-commerce engine
- One email message can create multiple events as it travels through various queues
– Each event in the network traffic logs represents a single user generating a single http request
– Visiting a single website normally generates multiple http requests - HTML, JavaScript, CSS files
- Flash, Images, etc. 

• <field-list>
–  One field or a list of field names
–  The events are grouped into transactions based on the values of this field list
–  If a quoted list of fields is specified, events are grouped together if they have the same value for each of the fields

• Common constraints:
– <maxspan> | <maxpause> | <maxevents> | <startswith> | <endswith> | <keepevicted> 

• Here you can see a number of events that share the same JSESSIONID
• However, it is difficult to display or gain insight to what is happening






Example 1
Display customer transactions in the online store during the last 15 minutes.

• Use the transaction command to create a single event from a group of events that share the same value in a given field
• Transactions can cross multiple tiers (i.e., web server, application server) using a common field(s), in this example, JSESSIONID



Type in search:
sourcetype=access_combined | transaction JSESSIONID

Example 2
Display a table of what customers are doing in the online store during the last 60 minutes.
With the transaction command, you can also use tables to easily view the information that you want
Scenario ?
Type in search:
sourcetype=access_combined | transaction JSESSIONID | table JSESSIONID, action, product_name

Specific Fields of transaction:
• The transaction command produces some additional fields, such as:
–  duration – the difference between the timestamps for the first and last event in the transaction
–  eventcount – the number of events in the transaction

Example 3
Use of maxspan/maxpause
Display customer actions on the website during the last 4 hours.
• You can also define a max overall time span and max gap between events
Scenario ?
- maxspan=10m
         ▸ Maximum total time between the earliest and latest events
         ▸ If not specified, default is -1 (or no limit)
 - maxpause=1m
         ▸ Maximum total time between events 
         ▸ If not specified, default is -1 (or no limit)
Note
Assumptions: Transactions spanning more than 10 minutes with the same client IP are considered unrelated, nor can there be more than one 1 minute between any two related events.

Type in search:
sourcetype=access_combined | transaction clientip maxspan=10m maxpause=1m | eval duration = round(duration/60,2) | sort -duration | table clientip duration action | rename duration as "Time Spent", clientip as "Client IP",   action as "Client Actions"

Type in search:
sourcetype=access_combined | transaction clientip maxspan=10m maxpause=1m | eval duration = round(duration/60,2) | sort -duration | table clientip duration action | rename duration as "Time Spent", clientip as "Client IP",   action as "Client Actions"

Example 4
Use of startswith/endswith
Determine the length of time spent by customers in the online store to purchase.
• To form transactions based on terms, field values, or evaluations, use startswith and endswith options
• In this example, the first event in the transaction includes addtocart and the last event includes purchase

Type in search:
sourcetype=access_combined | transaction clientip startswith=action="addtocart" endswith=action="purchase" | table clientip,  JSESSIONID, product_name, action, duration, eventcount, price

Example 5
Investigating with Transactions
Find emails that were rejected during the last 24 hours.

• Transactions can be useful when a single event does not provide enough information
• This example searches email logs for the term “REJECT”
• Events that include the term don’t provide much information about the rejection Scenario ?

Type in search:
sourcetype=cisco_esa REJECT


Find emails that were rejected.
• By creating a transaction, we can then search and see additional events related to the rejection, such as:
- IP address of sender
- Reverse DNS lookup results
- Action taken by the mail system following the rejection
• mid – Message ID
• dcid – Delivery Connection ID 
• icid – Incoming Connection ID

Type in search:
sourcetype=cisco_esa | transaction mid dcid icid | search REJECT




Reporting on Transactions
• You can use statistics and reporting commands with transactions
• This example takes advantage of the duration field
–  It shows a trend of the mail queue slowing, then correcting, then slowing again
- Adding events to the transaction from additional hosts or sources can uncover the cause of the slowdown

Type in search:
sourcetype=cisco_esa | transaction mid dcid ion) | timechart avg(duration)


Transaction vs. Stats
• Use transaction when you
     –  Need to see events correlated together
     –  Must define event grouping based on start/end values
• Use stats when you:
     –  Want to see the results of a calculation
     –  Can group events based on a field value (e.g. "by src")
• When you have a choice, always use stats
     –  stats is faster and more efficient, especially in large Splunk environments

Lets consider example:

sourcetype=sales_entries                                                             sourcetype=sales_entries
| transaction CustomerID                                                             | stats count as eventcount
| table CustomerID eventcount                                                    by CustomerID        
Returned 101 by scanning 31,767 events                                    Returned 101 results by scanning                                
 in 0.802 seconds                                                                          31,797 events   in 0.499 seconds