NLog with Splunk in Docker

by

Logging is a crucial part of any application. In this blog, I will walk through using NLog and Splunk for application logging.

Traditionally we used to use file-based logging. But with container-based application saving logs to a file is not a good strategy.

In enterprise, there is the use of log collectors, which is provided by major hosted logging solutions. These log collectors collect log from file and send them to a centralized viewing/storage system.

In the world of containers, installing a log collector is not a good idea either. Since containers can boot up and shut down at any time.

Today in thing blog, I am going to discuss how to use NLog in a containerized .NET Core application. And use Splunk as the log store and search tool.

Installing Splunk

Firstly, I will install the docker image of Splunk.

docker pull splunk/splunk:latest

Secondly, I will start the docker image, during which I will expose two ports; 8000 and 8088. The post 8000 is for the Web UI and port 8088 is for the API access.

docker run -d -p 8000:8000 -p 8088:8088 -e "SPLUNK_START_ARGS=--accept-license" -e "SPLUNK_PASSWORD=Password_Here" --name splunk splunk/splunk:latest

Configuring Splunk

For Splunk to be able to log application logs from NLog, I will configure the HTTP Event Collector.

  • Firstly, I will click Settings -> Data Inputs
  • Secondly, I will click "+ Add New" in the HTTP Event Collector row
  • Thirdly, provide the name of the event collector as "nlog-logger" and click Next
  • Fourthly, I will select "Search & Reporting (search)" in the "App Context" section and add all available items in the "Index" section and click Review.
  • Finally I will click Submit.
splunk settings
http listener for splunk
index configuration
settings review

Enabling the Http Event Collector

To enable the Http Event Collector, I will click on Settings -> Data Inputs -> HTTP Event Collector.

Once I am in the HTTP Event Collector page, I will click "Global Settings" and enable the event collector clicking the "Enable" Button. I will also set the "Default Index" as main.

global settings

.NET Core Console Application

Since the Splunk configuration is complete; hence now I will create a .NET Core Console application to log into Splunk using NLog.

Once the console project is created, I will install the following Nuget packages:

  • Microsoft.Extensions.DependencyInjection – For dependency injection
  • Microsoft.Extensions.Logging – Logging extension
  • NLog – Logging framework
  • NLog.Extensions.Logging – To get NLog into .NET Core IoC container logging integration
  • NLog.Targets.Splunk – Needed for NLog to log ti Splunk

Adding NLog configuration

Once the Nuget packages are installed, I will create nlog.xml file for the NLog configuration. I will copy the initial content of the file from the NLog.Targets.Splunk‘s GitHub page : https://github.com/AlanBarber/NLog.Targets.Splunk

I will update the following values in the nlog.xml file:

  • serverUrl – With my http://localhost:8088 url.
  • token – With the token value from the Http Event Collector “nlog-logger” I created.

IoC Container configuration

Once the Nuget packages are installed, I will create ContainerConfiguration to register the logger to the IoC container.

In this class, firstly, I will load the NLog configuration. Secondly, add NLog as the log provider. And finally, build the service collection and return the ServiceProvider instance.

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using NLog;
using NLog.Extensions.Logging;
using System;


namespace Splunk.Demo
{
    internal static class ContainerConfiguration
    {
        public static IServiceProvider Configure()
        {
            var serviceCollection = new ServiceCollection();

            LogManager.LoadConfiguration("nlog.xml");

            serviceCollection.AddLogging(l => 
            {
                l.AddNLog();
            }).Configure<LoggerFilterOptions>(c => c.MinLevel = Microsoft.Extensions.Logging.LogLevel.Trace);

            return serviceCollection.BuildServiceProvider();
        }
    }
}

Logging in Main

Finally, I will update the Main method in the Program class to get an instance of the ILogger from the IoC container; and log to Splunk.

using Microsoft.Extensions.Logging;
using Microsoft.Extensions.DependencyInjection;
using System;

namespace Splunk.Demo
{
    class Program
    {
        static void Main(string[] args)
        {
            var serviceProvider = ContainerConfiguration.Configure();

            var logger = serviceProvider.GetService<ILogger<Program>>();
            logger.LogInformation($"Test splunk log {DateTime.Now}");
        }
    }
}

Checking log in Splunk

Next, I will go to the Splunk “Search & Reporting” page. Enter the search text source=”Splunk.Demo.Program” in the search box hit enter. And this should give me the result of my log.

Conclusion

In conclusion, when running applications inside of the container, sending log to a remote location is key for crucial insights. Splunk helps us with that with few easy steps.

GitHub code location: https://github.com/choudhurynirjhar/splunk-demo

Most importantly, here is the link to the YouTube video: https://youtu.be/j4L9QDg6zas