Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add auth #358

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
74 changes: 0 additions & 74 deletions .eslintrc

This file was deleted.

2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
build/
node_modules/
target/
*.iml
.idea/
5 changes: 5 additions & 0 deletions .i18nrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"paths": {
"logtrail": "./"
}
}
2 changes: 1 addition & 1 deletion .kibana-plugin-helpers.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@
"{lib,public,server,webpackShims}/**/*",
"logtrail.json"
]
}
}
99 changes: 38 additions & 61 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,68 +1,45 @@
# LogTrail - Log Viewer plugin for Kibana

[![Github All Releases](https://img.shields.io/github/downloads/sivasamyk/logtrail/total.svg)](https://github.com/sivasamyk/logtrail/releases) [![Kibana 6.4.2](https://img.shields.io/badge/Kibana-v6.4.2-blue.svg)](https://www.elastic.co/downloads/past-releases/kibana-6.4.2)
[![License](https://img.shields.io/github/license/sivasamyk/logtrail.svg)](https://github.com/sivasamyk/logtrail) [![Donate](https://img.shields.io/badge/Donate-PayPal-green.svg)](https://paypal.me/sivasamyk)

LogTrail is a plugin for Kibana to view, analyze, search and tail log events from multiple hosts in realtime with devops friendly interface inspired by [Papertrail](https://papertrailapp.com/).
LogTrail에 자세한 내용은 [여기](https://github.com/sivasamyk/logtrail/)를 참고하세요

![Events](screenshot.png)

Features
--------
- View, analyze and search log events from a centralized interface
- Clean & simple devops friendly interface
- Live tail
- Filter aggregated logs by hosts and program
- Quickly seek to logs based on time
- Supports highlighting of search matches
- Supports multiple Elasticsearch index patterns each with different schemas
- Can be extended by adding additional fields to log event
- Color coding of messages based on field values
## 설명
기존 logtrail에 유저별 index 목록을 다르게 표기할 수 있는 기능을 추가하였습니다.\
elastic 6.8 버전 이상 부터는 키바나 로그인 기능을 기본으로 지원하여 플러그인 내 로그인 기능을 삭제 하였고, 키비나 로그인 유저 계정 정보를 사용합니다.

Installation
------------
- Prerequisites
- Download and install Elasticsearch , Logstash and Kibana
- Logtrail is supported and tested with Kibana 6.x and 5.x
- Install logtrail plugin (requires restart of Kibana after install)
- Kibana 6.4.2 : `./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.30/logtrail-6.4.2-0.1.30.zip`
- Kibana 5.6.5 : `./bin/kibana-plugin install https://github.com/sivasamyk/logtrail/releases/download/v0.1.23/logtrail-5.6.5-0.1.23.zip`
- Other versions : [https://github.com/sivasamyk/logtrail/releases](https://github.com/sivasamyk/logtrail/releases)
- Kibana requires exact match of plugin version to the Kibana version. If you can't find logtrail plugin release for a Kibana release, follow the instrcutions [here](docs/how_to.md#2-update-kibanaversion-in-logtrail-plugin-archive) to update Kibana version in your logtrail plugin archive.
- Refer [Logtrail Config Examples Repo](https://github.com/sivasamyk/logtrail-config-examples) for sample configurations for syslog, Java app, Kubernetes logs.
## 환경 구성 (Kibana 7.2 기준)
[여기](https://github.com/parkjungwoong/elastic-stack/blob/master/kibana/%ED%94%8C%EB%9F%AC%EA%B7%B8%EC%9D%B8%20%EA%B0%9C%EB%B0%9C%20%ED%99%98%EA%B2%BD%20%EC%84%A4%EC%A0%95.md) 참고하여 구성

Configuration
-------------
- Logtrail can be configured by editing following fields present in `logtrail.json` file located inside`./plugins/logtrail` directory.
- `default_index` - Elasticsearch index where the syslog events are stored (default: logstash-*)
- `default_time_range_in_days` - Default time range in days to search when time is not specified using Seek button.
Example: Value of 30 means logtrail will search only in logs from last 30 days, unless time is specified using Seek button.
Value of 0 means logtrail will search in all available logs by default.
- `display_timezone` - Timezone to display the timestamp in Event Viewer. e.g. `America/Los_Angeles`. Default value of `local` will use the timezone of browser. The time specified in `Seek To` popup will always use browser timezone.
- `display_timestamp_format` - Format to display the timestamp in Event Viewer. For list of valid value refer [here](http://momentjs.com/docs/#/displaying/)
- `default_search` - if specified, this will applied as default search text while launching logtrail. The value can be any search text. e.g. `ssh` - shows all logs with `ssh` in message field. or `log_level:SEVERE` - shows all logs where `log_level` field is `SEVERE`. The field name should be a valid field in elasticsearch document. The default search field is the field mapped to `message`.
- `fields` - Edit this parameter to map the event fields in ES to logtrail fields
- `timestamp` - maps to @timestamp field inserted by logstash. This will be used for querying internally. Logtrail recommends @timestamp to be stored in UTC in ES.
- `hostname` - hostname from where the events were received. Also used by hostname filter. Hostname field should be of type keyword. For more info checkout [Hostname field need to be of type keyword](docs/how_to.md#1-hostname-field-need-to-be-of-type-keyword)
- `program` - program that generated this event.
- `message` - actual event message. This field will be used by search.
- Example: If the event fields names are @timestamp, host, process, message the mapping should be
```json
"mapping" : {
"timestamp" : "@timestamp",
"hostname" : "host",
"program": "process",
"message": "message"
}
```
- By default each line displayed in the events view is of format:
`display_timestamp hostname program:message`
- `message_format` - Used to add additional fields to be shown for log event. For more details refer [Adding additional fields](docs/add_fields.md)
- `keyword_suffix` - Specifies the keyword suffix to be appended for hostname & program fields. Set it to `""` to not append any suffix. If not specified (`undefined`) logtrail will append `keyword`.
- `color_mapping` - Color code messages based on field values. For more details refer [Color coding messages](docs/color_mapping.md)
- Any changes in `logtrail.json` requires restart of Kibana
- Logtrail can read `logtrail.json` configuration from Elasticsearch instead of filesystem. This will be useful when sharing same configuration across multiple installations. For more info refer [Load Logtrail configuration from Elasticsearch](https://github.com/sivasamyk/logtrail/blob/master/docs/how_to.md#3-load-logtrail-configuration-from-elasticsearch)
- Refer [logtrail-config-examples](https://github.com/sivasamyk/logtrail-config-examples) repo for sample configurations
- Logs & Events from Windows, Java, Python, PHP, Perl, Ruby, Android, Docker, .Net can be shipped using syslog protocol.
- For more configuration options refer to [Papertrail Configuration Help](http://help.papertrailapp.com/).
- Beats/Fluentd can also be used to ship events to ES and fields can be mapped using `fields` parameter in `logtrail.json`
## 설정 파일
- 기본 설정 파일 형식은 [여기](https://github.com/sivasamyk/logtrail#configuration) 참고
- 계정별 인덱스 리스트 예시 :
```
{
"list": [
{
"id": "super",
"indexList": "*" //모든 인덱스를 조회할 계정은 '*' 로 설정
},
{
"id": "apiUser",
"indexList": ["api-*"] //나머지는 배열 형식으로 작성
},
{
"id": "admUser",
"indexList": ["adm-*", "api-*"]
}
]
}
```
- 로컬 파일 설정은 지원하지 않고 elasticsearch에 설정 정보를 입력
```
#설정 파일 작성 후 저장
vi logtrail.json
#설정 파일 내용을 elasticsearch에 저장
curl -XPUT 'localhost:9200/.logtrail/config/1?pretty' -H 'Content-Type: application/json' -d@./logtrail.json -u el계정:el비밀번호

#계정별 인덱스 정보 작성
vi user.json
curl -XPUT 'localhost:9200/.logtrail/config/2?pretty' -H 'Content-Type: application/json' -d@./user.json -u el계정:el비밀번호
```
3 changes: 0 additions & 3 deletions TODO.md

This file was deleted.

16 changes: 5 additions & 11 deletions index.js
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
var serverRoute = require('./server/routes/server');

export default function (kibana) {
return new kibana.Plugin({
name: 'logtrail',
Expand All @@ -7,17 +8,10 @@ export default function (kibana) {
app: {
title: 'LogTrail',
description: 'Plugin to view, search & tail logs in Kibana',
main: 'plugins/logtrail/app',
url: '/app/logtrail',
// injectVars: function (server, options) {
// var config = server.config();
// return {
// kbnIndex: config.get('kibana.index'),
// esShardTimeout: config.get('elasticsearch.shardTimeout'),
// esApiVersion: config.get('elasticsearch.apiVersion')
// };
// }
}
main: 'plugins/logtrail/app'
},
hacks: [
]
},
init: function (server, options) {
// Add server routes and initalize the plugin here
Expand Down
57 changes: 41 additions & 16 deletions logtrail.json
Original file line number Diff line number Diff line change
@@ -1,31 +1,56 @@
{
"version" : 2,
"index_patterns" : [
{
"es": {
"default_index": "filebeat-*"
},
{
"es": { "default_index": "api*" },
"tail_interval_in_seconds": 10,
"es_index_time_offset_in_seconds": 0,
"display_timezone": "local",
"display_timestamp_format": "MMM DD HH:mm:ss",
"display_timestamp_format": "YYYY MM DD HH:mm:ss",
"max_buckets": 500,
"default_time_range_in_days" : 0,
"default_time_range_in_days" : 1,
"max_hosts": 100,
"max_events_to_keep_in_viewer": 5000,
"default_search": "",
"fields" : {
"mapping" : {
"timestamp" : "@timestamp",
"hostname" : "beat.hostname",
"program": "source",
"message": "message"
"mapping" : { "timestamp" : "@timestamp",
"hostname" : "beat.hostname",
"program": "source",
"message": "message",
"log_level": "log_level",
"host_thread": "host_thread",
"offset": "offset"
},
"message_format": "{{{message}}}",
"keyword_suffix" : "keyword"
},
"color_mapping" : {
}
"message_format": "{{{host_thread}}} {{{log_level}}} {{{message}}}",
"secondary_sort_field": "offset",
"keyword_suffix" : "keyword"},
"color_mapping" : { "field":"log_level","mapping": {"ERROR": "#FF0000","WARN": "#FFEF96","DEBUG": "#B5E7A0","TRACE": "#CFE0E8"} }
}
,{
"es": { "default_index": "admin*" },
"tail_interval_in_seconds": 10,
"es_index_time_offset_in_seconds": 0,
"display_timezone": "local",
"display_timestamp_format": "YYYY MM DD HH:mm:ss",
"max_buckets": 500,
"default_time_range_in_days" : 1,
"max_hosts": 100,
"max_events_to_keep_in_viewer": 5000,
"default_search": "",
"fields" : {
"mapping" : { "timestamp" : "@timestamp",
"hostname" : "beat.hostname",
"program": "source",
"message": "message",
"log_level": "log_level",
"host_thread": "host_thread",
"offset": "offset"
},
"message_format": "{{{host_thread}}} {{{log_level}}} {{{message}}}",
"secondary_sort_field": "offset",
"keyword_suffix" : "keyword"},
"color_mapping" : { "field":"log_level","mapping": {"ERROR": "#FF0000","WARN": "#FFEF96","DEBUG": "#B5E7A0","TRACE": "#CFE0E8"} }
}
]
}

36 changes: 19 additions & 17 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,41 +1,43 @@
{
"name": "logtrail",
"version": "0.1.30",
"description": "Plugin to view, search & tail logs in Kibana",
"version": "1.0.1",
"description": "logtrail with login",
"main": "index.js",
"kibana": {
"version": "6.4.2"
"version": "7.2.0"
},
"repository": {
"type": "git",
"url": "https://github.com/sivasamyk/logtrail.git"
"url": "https://github.com/parkjungwoong/logtrail.git"
},
"license": "MIT",
"bugs": {
"url": "https://github.com/sivasamyk/logtrail/issues"
"url": "https://github.com/parkjungwoong/logtrail.git/issues"
},
"scripts": {
"preinstall": "node ../../kibana/preinstall_check",
"kbn": "node ../../kibana/scripts/kbn",
"lint": "eslint **/*.js",
"preinstall": "node ../../preinstall_check",
"kbn": "node ../../scripts/kbn",
"es": "node ../../scripts/es",
"lint": "eslint .",
"start": "plugin-helpers start",
"test:server": "plugin-helpers test:server",
"test:browser": "plugin-helpers test:browser",
"build": "plugin-helpers build"
"build": "plugin-helpers build",
"clean": "rm -rf node_modules; rm -rf yarn.lock"
},
"dependencies": {
"ansi-to-html": "^0.6.4",
"handlebars": "4.0.6",
"lodash.escape": "4.0.1",
"lodash.get": "4.4.2",
"lodash.set": "4.3.2",
"lodash": "4.17.11",
"moment": "2.22.2",
"sugar-date": "1.5.1",
"moment": "2.22.2"
"yarn": "^1.13.0"
},
"devDependencies": {
"@elastic/eslint-config-kibana": "link:../../kibana/packages/eslint-config-kibana",
"@elastic/eslint-import-resolver-kibana": "^0.9.0",
"@kbn/plugin-helpers": "link:../../kibana/packages/kbn-plugin-helpers",
"@elastic/eslint-config-kibana": "link:../../packages/eslint-config-kibana",
"@elastic/eslint-import-resolver-kibana": "link:../../packages/kbn-eslint-import-resolver-kibana",
"@kbn/expect": "link:../../packages/kbn-expect",
"@kbn/plugin-helpers": "link:../../packages/kbn-plugin-helpers",
"babel-eslint": "^8.0.2",
"eslint": "^4.11.0",
"eslint-plugin-babel": "^4.1.1",
Expand All @@ -46,4 +48,4 @@
"eslint-plugin-react": "^7.0.1",
"expect.js": "^0.3.1"
}
}
}
Loading