Laravel Log Management Using Elasticsearch and Kibana

Laravel Log Management Using Elasticsearch and Kibana

Laravel is one of the most popular PHP frameworks that provides a robust application development experience. However, as your Laravel application grows, it becomes increasingly difficult to manage and monitor your logs. That’s where Elasticsearch and Kibana come into play. Elasticsearch is a distributed search and analytics engine that allows you to store and search through large amounts of data quickly and efficiently. Kibana is a data visualization tool that allows you to create dashboards and visualizations based on data stored in Elasticsearch.

By integrating Elasticsearch and Kibana with your Laravel application, you can easily manage and monitor your logs. Laravel provides built-in support for logging into various channels, including files, databases, and the syslog. By configuring your Laravel application to log to Elasticsearch, you can centralize your logs and easily search and analyze them using Kibana.

To get started, you will need to install Elasticsearch and Kibana on your server or use a cloud-based Elasticsearch service such as Elastic Cloud. The monolog provides an Elasticsearch log driver for Laravel. Then, you can configure Laravel to log into Elasticsearch by updating the “logging.php” configuration file.

With Laravel logging to Elasticsearch, you can use Kibana to create visualizations and dashboards based on your logs. Kibana provides a user-friendly interface for creating visualizations such as line charts, bar charts, and pie charts. You can also create dashboards to display multiple visualizations on a single page, making it easy to monitor your application’s health and performance.

Laravel application setup:

First, install a Laravel application

composer create-project laravel/laravel log-management-via-es
cd log-management-via-es

Now, open the logging.php config file and add a new channel named es-log shown bellow

     <?php
return [
   //
    'channels' => [
       //
        'es-log' => [
            'driver' => 'monolog',
            'level' => 'debug',
            'handler' => \Monolog\Handler\ElasticsearchHandler::class,
            'formatter' => \Monolog\Formatter\ElasticsearchFormatter::class,
            'formatter_with' => [
                'index' => env('ELASTIC_LOGS_INDEX'),
                'type' => '_doc',
            ],
            'handler_with' => [
                'client' => \Elasticsearch\ClientBuilder::create()->setHosts([env('ELASTIC_HOST')])->build(),
            ]
        ]
    ]
];

Add this variable to the .env file

ELASTIC_HOST=http://127.0.0.1:9200 
ELASTIC_LOGS_INDEX=laravel_logs
LOG_CHANNEL=es-log 

ElasticeSearch and Kibana setup via docker-compose file

Copy and save it in the docker-compose.yml file

version: '3.9'
services:
  elasticsearch:
    container_name: elasticsearch
    image: docker.elastic.co/elasticsearch/elasticsearch:7.0.1
    ports:
     - "9200:9200"
    environment:
     - http.host=0.0.0.0
     - transport.host=127.0.0.1
     - xpack.security.enabled=false
     - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - "./elk-data:/usr/share/elasticsearch/data"
    networks:
      - elk

  kibana:
    container_name: kibana
    image: docker.elastic.co/kibana/kibana:7.0.1
    environment:
      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
      ELASTICSEARCH_URL: http://elasticsearch:9200
    depends_on:
      - elasticsearch
    ports:
      - 5601:5601
    networks:
      - elk

networks:
    elk:
      driver: bridge

Run using the command

docker compose -f "docker-compose-ek.yml" up -d --build

Browse the URL: http://localhost:5601/

Create an index pattern for log monitoring in kibana by browsing the following link localhost:5601/app/kibana#/management/kiban..

Press the Create index pattern Button and create a pattern from the below index list

Then publish your system log anywhere your application

  \Illuminate\Support\Facades\Log::info("Laravel log-management-via-es");

Monitoring log in Kibana by browsing the URL localhost:5601/app/kibana#/discover

Hope you enjoy the article. Do you have any questions? Please add this in the comment section.