Ingesting logs into Loki

Here is the standard Loki log processing flow that I use for my logs. The pipeline is comprised of the following stages: adding job label (so that I can query all logs ingested from files) add directory label (by obtaining the directory name from the filename label) packing the filename label into the log entry using the stage.pack stage (reducing cardinality of the labels, querying can be done by the directory label) adding hostname and agent_hostname labels to the logs (agent_hostname refers to the machine running the agent, hostname is obtained from the logs. Continue reading

Ingesting SCCM logs into Loki

System Center Configuration Manager (SCCM) is still in use in my lab, mostly as a means of deployment of applications and updates. Whilst I’m working on moving some of the functionality into Intune, SCCM will remain for forseeable future the update orchestrator for my server environment. SCCM logs seem to be standardized around two formats. Here are the examples of them: Service is up and running.~~ $$<SMS_REST_PROVIDER><01-26-2025 17:35:20.122+00><thread=13372 (0x343C)> <![LOG[Worker M365ADeploymentPlanWorker was triggered by timer. Continue reading

My LG OLED TV is crashing

I love my LG OLED TV, I believe the picture quality is unbeatable and I could call myself an LG TV advocate. However, this is not to say they’re flawless. I like WebOS, but’s it becoming clunky and slow - there’s too many advertisements all over the place and the UI is not as responsive as it used to be - this is said based on my experiences of owning two C1 models of 55 and 65 inch size. Continue reading

DNS service discovery for Prometheus

Background story Back when I ran SCOM, in addition of Windows machine monitoring and Event Log aggregation, it was performing a duty of ping testing all the servers in my environment. This was a very useful feature as it allowed me to quickly identify servers that were down or unreachable. The main quirk was that it was only aware of the servers that I manually added and not able to discover things automatically as I didn’t have the right setup to leverage the SNMP based discovery. Continue reading

Veeam Exporter for Prometheus

Over the past couple of months I’ve put a considerable amount of time into deployment of a monitoring infrastructure in my home-lab that would replace Splunk and SCOM. In a way, this setup introduced a new level of monitoring which I did not have before, I’ve deeply fallen for metrics and the power of Prometheus and pretty much sunk into the Grafana’s LGTM ecosystem, quickly implementing Tempo and Loki for a full experience. Continue reading

Hello and welcome

Welcome to the first post on my new blog. I will be writing about all things related to IT, mainly focusing on VMware and Microsoft technologies. I will also be writing about my personal projects and other things that I find interesting.