IT managers that want to secure their Linux environments and keep things running smoothly have a very powerful tool at their disposal: Security Enhanced Linux, or SELinux, an implementation of mandatory access controls originally developed by the National Security Agency (NSA) and integrated in to most mainstream Linux distributions.
"[SELinux] stops theft; it stops spam relays, it stops worms from attacking your site," said Dan Walsh, principal software engineer at Red Hat and a regular contributor to the SELinux project. As such, IT managers should leave it on at all times in every facet of their data centers.
The problem is, these days many users simply turn SELinux off (it's built into Red Hat Enterprise Linux).
While the open source security technology is widely accepted as incredibly secure, it is also seen as wildly complex. A slew of new tools and policy management features in RHEL 5 could help change that perception, but is it too late?
SELinux: Reality versus mindshare
"The biggest problem for SELinux is mindshare," said Jim Klein, the director of information services and technology at California-based Saugus Union School District. "It developed a stigma early on due to the lack of tools for configuration and troubleshooting, which led people to simply turn it off.
Sadly -- for SELinux advocates anyway -- the problem got to the point where an administrator's first question when troubleshooting a system would be, "Is SELinux turned on?" Klein said. Klein said SELinux is turned off in his data center, and he won't consider reactivating it until his district's planned migration to RHEL 5 is complete.
But Red Hat's Walsh said the SELinux "complexity problem" could be waning.
Walsh recently dissected SELinux, the application security technology that now comes turned on by default in Red Hat Enterprise Linux 5, during a session at the annual Red Hat Summit in San Diego. SELinux was included in RHEL 4, but only now can Walsh and other SELinux experts safely say : "Leave SELinux on everywhere."
"RHEL 4 was like a demonstration of the technology," Walsh said. "We had confined it to a certain amount of domains, or 15 targeted programs [within RHEL] that applications had access to."
With RHEL 5, however, the number of targeted systems was ratcheted up to 200. "The goal [with RHEL 5] is too leave SELinux on everywhere.
SELinux: complex, but Troubleshooter could help
One expert who knows SELinux better than most is author and SELinux expert Frank Meyer.
"I won't accuse anyone specifically of putting that [complexity] idea out there, but the perception is there because SELinux has the ability to protect everything the Linux kernel provides," he said. "The Linux kernel itself is complex and you have to address everything the Linux kernel provides."
To Meyer, when a user says SELinux is too complex to be deployed effectively, it's like them saying they can't use the Linux kernel because they don't know how to write a device driver. "Logically, it just doesn't make sense," he said.
To address this complexity, Red Hat has introduced SELinux Troubleshooter in RHEL 5.
SELinux Troubleshooter, also known as settroubleshoot, is a tool that watches the audit log files for access vector cache (AVC) messages.
According to the Fedora Project website, where much of SELinux's testing with future Red Hat Linux builds takes place, users, system administrators, and developers often run afoul of AVC Denials. When the SELinux policy is fully debugged and properly configured the only AVC Denial which should occur are those triggered by actual security violations. However, because SELinux is still an emerging technology, the policy is still under development, and because users are still learning how to configure SELinux the vast majority of AVC Denials are unintentional as opposed to an actual security violation.
Today, when an AVC denial arrives the tool runs through the SELinux plug-ins database looking for a match and then sends a message to the user with a description and a suggested fix. Industry watchers like Meyer and Walsh say the tool goes a long way in helping users distinguish between legitimate concerns and the false alarms that hampered SELinux use among IT managers.
Klein said the tool was a welcome addition, but it may have arrived too late to help SELinux's mindshare difficulties. Troubleshooter and its ilk are a "great starting point for any administrator seriously considering SELinux again," he said, but "the tough thing will be to convince those that have already dismissed the technology as the 'source of all problems' to not simply turn it off as a first step when something doesn't work."
Up until now, Klein said Saugus has left SELinux off for most servers, defaulting to traditional mechanisms for application security, while they waited for SELinux tools and policies to mature.
Those mechanisms include traditional application space controls for Linux/Unix environments, which amounts to creating a dedicated user for a service, and only granting that user rights to the parts of the system they should have access to, Klein said.
For example, if Klein were to launch Apache as if the user named "apache" started it, it only gives it access to the file system in places the apache user has access. "This does not allow much control over port usage and the like, which SELinux provides, and can be weak should a hacker find a way to switch users after accessing a service," he said. "As we migrate to RHEL 5, however, we are finding that we now have the tools and resources we need to use the technology effectively."
SELinux policy modules and RHEL 5 to the rescue?
SELinux's moment of clarity for IT managers, so to speak, may indeed lie with RHEL 5. Experts like Meyer laud Red Hat for tools like Troubleshooter as well as for the addition of basic security policies "right out of the box."
For example, RHEL 5 has a SELinux policy that provides enhanced security protection for several dozen pre-existing services and applications. You don't have to do anything -- it's just there, Meyer said.
"Red Hat rightly provides enhanced security for 50 or so services out of box," Mayer said. "And SELinux Troubleshooter is this out-of-box, basic, holistic system tool that allows administrators to try and attempt fix them."
There is also a lot of ongoing innovation in the SELinux tools that help IT administrators develop advanced policies. First, the way policy source is organized and structured has been completely re-invented to ease development. This is the so-called reference policy, Meyer said.
A loadable policy module infrastructure also exists to help manage the installation of new policy modules. Both of these improvements are fully integrated into current Linux distributions, including RHEL 5.
Managing these is accomplished via semanage, which can be used to configure certain elements of SELinux policy without requiring modification to or recompilation from policy sources. There are also third party tools available, like Tresys Technologies' Brickwall, a suite of commercial open source SELinux tools.
Red Hat's Walsh said SELinux has a new GUI in RHEL 5 to assist in management, as well as a set of configurable Booleans (Editor's Note: if, then statements) that allow IT managers to modify network ports, file labeling and event user mappings.
"The goal is to get IT managers comfortable enough with SELinux so that they write their own customized policies to lock down their own environments," Walsh said.
"The bottom line is your network will be breached; the walls of the castle are going to be breached," Walsh said. "We want you to run SELinux everywhere. Turn it on. If you had a problem with it, there are tools that now exist to fix it.