itb-au logo
Story image

Interview: Why log management is crucial to modern business

08 Mar 2019

Recently I had the chance to sit down for an interview with Balázs Scheidler, an industry veteran in IT security and software engineering.

In 2000 he co-founded Balabit because he was tired of using the old syslog protocol and not getting valuable data fast enough from log messages for security analytics and IT operations.

In just a few years his syslog-ng log management solution became the de facto industry standard for log collection in Linux-based installations, used by millions of corporations around the world. Having established a name for itself, the company then delved into the Privileged Access Management (PAM) solution space.

Last year Balabit was acquired by One Identity, with Scheidler’s role now focused on shaping the future strategy for its PAM and log management solutions.

Scheidler says syslog was originally created in the early 1980s to debug software as part of the Sendmail project.

"Basically, it was designed to be a fire and forget approach to log management and logging in general. It was very simple as they didn't care too much about how reliable it was or even security in general,” says Scheidler.

Their point was to make it possible to get activity data from the mail servers off those hosts and to their separate repository where they could run their own statistics and reporting.

"Vendors of network gear, applications and OSes started to adapt syslog for their own use, which expanded the scope of data considerably. It was very difficult to extract value from log messages because of the huge amount of unstructured data, making it a bit like finding a needle in a haystack," says Scheidler.

Despite this, businesses still saw the huge value in log messages as if there was an incident or some kind of outage, the only data source was this data. Even if an incident was recognised years after it happened, you could still go back and find the details and root causes.

However, Scheidler says there are still organisations in this age relying on syslog over UDP, which doesn’t guarantee delivery and doesn’t use encryption in the logging, which means they’re being deprived of the many benefits of the latest technology to support their log management

“As I mentioned, the adoption of syslog-ng was significant early on, but today still more and more devices are becoming capable and the demand to integrate them into logging infrastructure is always increasing. Furthermore, log data is needed by more and more downstream systems. A SIEM solution is a natural consumer of log messages, but these days things like UBA or business intelligence systems also tap into logs. Using something that can connect log-producing devices with downstream processing systems is needed and syslog-ng is one of the biggest implementations of this idea,” says Scheidler.

“Fixing the reliability and security issues while adopting the growing set of devices that do logging and be able to feed more downstream systems - those are the main reasons for improving log management practices today.”

In terms of how syslog-ng differs from other log management solutions, Scheidler says the original reason he started the project was to improve reliability.

“Originally it was using UDP only, and then I introduced the TCP based-logging system and we have constantly improved that ever since. So now in a distributed enterprise-wide and global log management system we are capable of never losing messages within the log management infrastructure. We are also very good at security - when it comes to data at rest and data in transit we solve both of these issues,” Scheidler says.

“And we do this at a high scale. So I mentioned the relationship between log-producing devices and SIEMS, and feeding multiple teams which naturally means that we have terabytes of data to transport daily, and scale is a key feature of a system like that. To put it into perspective, syslog-ng can process hundreds of thousands of messages a second, and it does that efficiently. What other systems would do on 10 computers, it can do on one.”

Scheidler stressed that while there is an overlap, syslog-ng isn’t a SIEM solution. Often they can meet all the requirements for SMEs that are unable to afford a complete SIEM system or deployment (because they can tend to be quite expensive), but he generally positions syslog-ng as the ‘glue’ between log-producing and log-consuming devices.

“Effectively it can run on end systems and ensure that the collection, centralising, storage, archival, indexing, and searching functions are all performed on our side,” says Scheidler.

“You don't have to send everything into the SIEM, only those things that you really need to, which enhances the performance of your existing SIEM.”

Looking to the future, Scheidler says they’re moving with the demands of their customers.

“Our priorities now lie in the cloud space. Centralised system functions or log management functions are usually an on-premise system because you have to collect everything that you already have on-premise. Customers now expect us to fold logs coming from Cloud systems into the same repository. For example, they want to see Office 365 logs in their Splunk installation, or they want to see AWS access logs in their SIEM, to talk about it broadly,” says Scheidler.

“And that means we have to integrate with those systems as well and provide solutions for customers to solve this hybrid issue. ‘I have a lot of on-premise systems that produce logs; I have a lot of cloud-based systems that produce logs; I want to see them in the same place be that either in the cloud or on-premise’ - that is the opportunity we’re looking to capitalise on.”

In terms of implementation of the log management solution, Scheidler says it is a very simple process.

The syslog-ng Store Box (SSB) for example can be deployed in just a few days, offering a high-performance, high-reliability log management appliance that enables you to search logs, secure sensitive information with granular access policies, generate reports to demonstrate compliance and forward log data to third-party analysis tools.

Click here to learn how to adopt log management best practices.

Story image
IDC names ESET a Major Player second year running
“ESET is strong in the areas of threat research, especially around Android malware identification and behavior detection.”More
Story image
D-Link launches new router and cloud IoT management solution
The DWM-315 is a 4G LTE Cat 6 Dual SIM M2M VPN Router with EWAN and GPS and the D-ECS edge cloud management solution expands the D-Link IoT range.More
Story image
Snowflake announces updates geared towards data mobilisation
"The new features announced today are another example of Snowflake's commitment to delivering the technology customers need to fully mobilise their data and achieve meaningful business value.”More
Link image
On 10 December, find out how data centers will fare in the face of climate change
From pandemic disruption to the urgency to address climate change, data center development in Asia Pacific needs to remain resilient and sustainable. On 10 December, join this webinar to learn about viable solutions data center operators can use to overcome environmental challenges. Register now.More
Story image
The cybersecurity risks that come with re-onshoring Australian manufacturing
As technology such as IoT, robotic process automation (RPA) and artificial intelligence (AI) reshapes the manufacturing landscape, organisations are simultaneously put at an increased risk of a cyberattack.More
Story image
Trintech expands Australian Data Centre and makes additions to local team
"This investment enables mid-market companies in the APAC region to leverage Trintech's industry-leading financial close solution, Adra, in complete compliance with onshore data requirements."More