[ELK] logstash

Feb 03, 2016/May 18, 2018

###개요###

###기본 실행###

logstash -f <conf file>

###Conf 파일 내 구조###

간단하게 Apache acesslog 를 가져와서 JSON으로 변환하는 작업을 해보자.

first-pipeline.conf


input {
  file {
    path => "/home/system/logs/test/access-2016-02-03.log"
    start_position => beginning
  }
}
filter {
    grok {
        match => { "message" => "%{COMMONAPACHELOG}"}
    }
    geoip{
       source => "clientip"
   }
}
output {
   stdout{ codec => json }
   elasticsearch{}
}

grok filter plugin

geoip filter plugin

###Check conf file###

./logstash -f ./first-pipeline.conf --configtest

###결과###

{
    "message": "127.0.0.1 - - [03/Feb/2016:14:43:58 +0900] \"POST /test/web HTTP/1.1\" 200 22410",
    "@version": "1",
    "@timestamp": "2016-02-03T05:43:59.585Z",
    "path": "/home/system/logs/test/access-2016-02-03.log",
    "host": "0.0.0.0",
    "clientip": "127.0.0.1",
    "ident": "-",
    "auth": "-",
    "timestamp": "03/Feb/2016:14:43:58 +0900",
    "verb": "POST",
    "request": "/test/web",
    "httpversion": "1.1",
    "response": "200",
    "bytes": "22410",
    "geoip": {
        "ip": "127.0.0.1",
        "country_code2": "KR",
        "country_code3": "KOR",
        "country_name": "Korea, Republic of",
        "continent_code": "AS",
        "region_name": "13",
        "city_name": "Seongnam",
        "latitude": 37.43860000000001,
        "longitude": 127.13780000000003,
        "timezone": "Asia/Seoul",
        "real_region_name": "Kyonggi-do",
        "location": [127.13780000000003, 37.43860000000001]
    }
}

References


#dev  #logstash  #elk