Steps to Creating a New Metrics Program

Metrics definitely seem to be a buzz word in information security circles these days. It seems that I can hardly give a presentation or meet with clients without the topic coming up at some point in our discussion. But to be fair, I think these discussions are healthy and I’m glad to see so many people beginning to ask the question, “how can I measure my efforts in securing my organization’s data?”

My fear though, is that there are a number of organizations who see other organizations implementing metrics and they believe that they can skip the core foundational program steps and jump right to interesting dashboards and metrics. It’s like watching someone who never exercises observing a marathon runner and saying, “I think I’d like to run a sub-four hour marathon too this year.” Without establishing a base and putting in the effort to practice, achieving great results can be difficult to impossible.

So this begs the question, in terms of information security, how would you build a base, and what steps could you take to lay the foundation for a solid, mature information security program. After working with our clients for the past 10-15 years, this would be our advice:

  1. Obtain a security management charter from senior management
  2. Create an organization wide IS Steering Committee
  3. Document your organization’s overall security goals
  4. Create & approve appropriate security policies, procedures, & standards
  5. Educate your organization on those documents

Then what? If we finish those steps, then can we start a metrics program? Absolutely! Once you have a proper foundation and you know what your organization is trying to achieve, then I think it’s a perfect time to kick off a program to develop metrics in your organization. But take it one step at a time. Don’t try to bite off too much too quickly. Take it slow, achieve results, and then progressively add to the sensors and data you collect.

  1. Here are the steps we would take to start your initial metrics program:
  2. Identify what information security sensors you have already successfully deployed
  3. Determine what meaningful metrics can be gleaned from these sensors
  4. Deploy a tool that can centrally aggregate, normalize, and report on the data collected by the sensors
  5. Create basic reports based on the metrics from strep #2
  6. Work with business owners to remediate risk

Our best advice though is this: achieve value from a small number of metrics first, and then grow your program. If you can’t achieve value from a small number of measurements, you certainly won’t achieve value with a greater volume. Eat that elephant one bite at a time.

Basic Steps for Executive Engagement

Recently a met with an organization who mentioned to us that they had identified executive engagement in information security (or lack thereof) the biggest risk to their organization. It’s not to say that the organization’s executives didn’t care. The issue was that this organization had its hands in a number of other important activities, and securing the organization’s assets simply was not one of their top concerns.

But this raises a few questions – What does it mean for executives to be engaged in information security? How much buy in is enough for management? Is there a bare minimum standard for executive engagement?

So we started thinking, executives have a lot on their plates, so what should a busy executive do to sponsor an information security program without getting sucked into a black hole of too much engagement and allocating too much of his or her attention to the program. It seems to me executives have the responsibility to ensure that all the plates are spinning in the organization to continue pursuing the overarching business strategy. It takes a number of moving parts to achieve an organization’s mission. So what steps should they take to make sure the information security plate stays in the air?

Step One:            Assign an information security champion (think senior project manager).

Step Two:            Sign a charter for an information security steering committee, or add security responsibilities to the IS steering committee’s charter.

Step Three:        Task the steering committee with documenting policies & procedures for how information security will take place in the organization.

Step Four:           Support the steering committee in their efforts with other executives and in front of the organization as a whole.

Step Five:            Monitor the committee’s progress and encourage them along the way.

Remember this is a journey, not a destination. This is a program, not a project. But like any part of an organization, the information security team needs executive support and guidance in order to help the organization to achieve their goals. Security is expensive and requires effort, but when balanced with other aspects of the business it can help the organization to be healthy and succeed in the long term.

Elements of an Information Security Charter

Part of any solid project / program management effort is a program charter that defines the program in order to ensure its success. Too many times projects begin without a clear definition of success and as a result it becomes very difficult to measure success or often even to make progress on the project in any way. Information security programs are no exception.

In order to have a successful information assurance program, organizations need to take the first step of creating a charter for the information assurance team. The benefit of these charters is that they should function in the same way as any other program charter. Because of that we have a solid body of knowledge available to us as to what elements should exist in a mature program charter.

Elements of a mature information security program charter should include:

  • Name / Title
  • Start and end date / timeline
  • Approval authorities / executive sponsorship
  • Team leadership / management
  • Key players / stakeholders
  • Business case / purpose / regulatory requirements
  • Problem statement or opportunity
  • Business benefits
  • Measurable performance outcome / metrics
  • Scope of work
  • Key milestones
  • Roles and responsibilities
  • Manpower and budget requirements
  • Barriers to success and risks
  • Communication plan

If you haven’t taken the time to define a program charter for your security team, why not start now. Not only will this process help to define the goals of your effort, but it can focus your team’s efforts and align executives with the team’s efforts.


Practical Risk Assessment Tools

In a previous blog post we cataloged a number of risk management methodologies that we’ve seen a number of organizations employ in an effort to manage the security of their information systems. A number of people have asked us though, what tools best assist people implementing those models? Are there tools available to make the process easier or do companies have to develop their own tools to make one of these methods a reality?

Unfortunately the answer is that most of the companies we’ve worked with have chosen to develop their own risk management tools. Though to be fair the majority of companies we meet choose to manage their efforts through very simple tools such as Microsoft Excel spreadsheets. Whole there’s nothing wrong with that, the questions inspired us to consider following up to the previous post with a list of some of the risk assessment toolkits we’ve seen people use.

In the open source world there are a few choices, and more and more seem to be springing up all the time as the need for visual risk assessment tools increase. Some of the more popular tools we’ve encountered are:

  • Binary Risk Assessment Tools
  • Babel Enterprise (free & commercial)
  • Cyber Security Evaluation Tool (DHS)
  • OSSIM SIEM (free & commercial)
  • Practical Threat Analysis (PTA) Professional

But this doesn’t mean that there aren’t commercial tools that are also available to purchase to jumpstart this process. Most tools in this commercial space are known as Governance, Risk, and Compliance (GRC) tools and Gartner even publishes a Magic Quadrant on the subject. Some of the more popular commercial tools are:

  • OpenPages Enterprise GRC
  • Thomson Reuters Paisley
  • Bwise GRC
  • Oracle Enterprise GRC Manager
  • MetricStream
  • Methodware ERA
  • Cura Enterprise
  • Archer Technologies SmartSuite
  • Protiviti Governance Portal
  • Mega Suite
  • Aline Operational Suite
  • CCH TeamMate, Sword, & Axentis
  • IDS Scheer ARIS

Hopefully this list gets you thinking and gives you a good place to get started as you consider which tool is the best option for you. Happy hunting!

Formal Risk Assessment Methods

Many organization’s we have been speaking with lately have begun the process of risk management in an effort to formalize information assurance programs. For many organizations this is a new step and involves uncertainty. One of the biggest questions we get when talking with companies is: what risk assessment models are there to choose from?

Certainly the catalog of potential methods is long and every practitioner we know seems to have a preference for a particular risk management methodology. Certainly we don’t want to start any wars over this, but we do want people to understand what some of their options are. So we’ve been compiling a list of some of the more popular methodologies that we’ve seen companies implementing.

In no particular order, here are some of the more popular methods we’ve seen companies using:

  1. Single Loss Expectancy (SLE) / Annualized Loss Expectancy (ALE)
  2. Operationally Critical Threat, Asset, and Vulnerability Evaluation (OCTAVE)
  3. National Institute of Standards & Technology (NIST) Special Publications 800 Series
  4. ISO 27005 Risk Management Methodology
  5. Factor Analysis of Information Risk (FAIR)
  6. Microsoft STRIDE & DREAD
  7. Guide to the Assessment of IT Risk (GAIT)
  8. Failure Modes and Effects Analysis (FMEA)
  9. Failure Modes, Effects and Criticality Analysis (FMECA)
  10. Cause Consequence Analysis (CCA)
  11. Modeling & Analysis of Safety and Risk in Complex Systems (MASR)
  12. Open Governance, Risk and Compliance Maturity Management Methodology (OGRCM3)
  13. Open Risk Model Repository (ORIMOR)

Does this mean that all the companies we deal with choose one of these methods – of course not! Most companies we meet choose a personalized method that customizes aspects from multiple models in order to create a model that works best for them. Of course there’s nothing wrong with that either. We just hope that by understanding some of your choices that you’ll be able to make a better decision which model is best for you.

The best risk management model will always be the one that helps you and your organization to achieve your goals.

Automating Audit Baselines – A Case Study

Quite a few times in this blog we’ve talked about automating audit and assessment tasks, especially as it relates to system baselines. We’ve tried quite a few times to give our readers tools for creating baselines and always hope that people will turn those baselines into automated processes that will alert them to deviations or changes to their systems.

Certainly one way to do this is via commercial products. Many companies have purchased file integrity assessment toolkits, such as those by Tripwire, in order to help automate this process. Others have turned to lower cost mechanisms such as scripting tools to do the same thing, although maybe not as elegantly.

In doing a little reading online at the Microsoft Script Center, I found a nice case study who decided to use scripting to accomplish their automation goals. Ben Wilkinson from Microsoft posted a series of blog articles where he describes the process of automating and alerting on changes to group memberships using Microsoft scripting technologies. Here’s a link to the full set of articles:

I think it’s worth highlighting and applauding efforts like this. If you aren’t already using an automated process like this to assist you with your continuous audit efforts, hopefully you can receive some inspiration from what Ben has posted here. Thanks Ben for the posts.

More PowerShell Audit One Liners

In our last couple posts we described how to gather a general baseline of system demographics on a Microsoft Windows system you’ve been tasked with auditing. Hopefully the posts gave everyone some ideas for the capabilities that PowerShell offers, even if the information we gathered isn’t all that exciting. In this post I thought we would show you additional examples you could try if you want to explore other pieces of information you might be able to gather with PowerShell and WMI during an audit.

Once you have the general syntax of these commands, even if you don’t fully understand the scripting behind it, you should be able to copy and paste these commands into an audit script. If you want a full library of the various WMI objects that Microsoft makes available or the attributes they return, check out this link over at Microsoft:

So here are a few other examples of WMI queries that might be useful during an audit:

List the available IPv4 Address(es) from a system:

get-wmiobject Win32_NetworkAdapterConfiguration | fl Name,IPAddress

List the available IPv6 Address(es) from a system:

ifconfig -a | awk '/inet6 addr:/ { print $3 }'

List the available MAC Address(es) from a system:

get-wmiobject Win32_NetworkAdapterConfiguration | fl Name,MACAddress

List the User Accounts on a system:

get-wmiobject Win32_UserAccount | ft Name,SID

List the Groups on a system:

get-wmiobject Win32_Group | ft Name,SID

I hope these help to inspire you to try out scripting in your audits and maybe even consider writing a few audit scripts of your own.

Using SystemInfo.exe to Baseline a System

After our last post on gathering system demographics using PowerShell (specifically the Get-Object cmdlet) we had a few auditors mention to us that there are other ways to do it as well. We couldn’t agree more and we’re glad they brought it up. Microsoft seems to like to give us choices for how we perform job tasks, and this is no exception.

One other very popular way to gather information from a Microsoft Windows system is through the built-in systeminfo.exe utility. This command has been available at the command line since Microsoft Windows XP, and so in the course of an audit you’re very likely to find this command native on any Windows system you happen to be auditing.

One of the other nice things about this command is the fact that it is very, very simple to run. Simply type the name of the binary into a cmd.exe or powershell.exe terminal window and the tool will query information about the underlying system you’re examining.

There aren’t many options or command line switches that you can use to customize the output, but there are a few. Microsoft documents all of the options you do have at From that same article, here are the options that they make available to you:

[framed_box]/s Computer : Specifies the name or IP address of a remote computer (do not use backslashes). The default is the local computer.
/u Domain \ User : Runs the command with the account permissions of the user specified by User or Domain\User. The default is the permissions of the current logged on user on the computer issuing the command.
/p Password : Specifies the password of the user account that is specified in the /u parameter.
/fo { TABLE | LIST | CSV } : Specifies the format to use for the output. Valid values are TABLE, LIST, and CSV. The default format for output is LIST.
/nh : Suppresses column headers in the output. Valid when the /fo parameter is set to TABLE or CSV.
/? : Displays help at the command prompt[/framed_box]

So a few of the nice features you can see from the utility already is the ability to run the command against remote computers, the ability to specify the output format of the data (including CSV format), and even the ability to suppress the headers in a CSV file to make it easier to parse later.

So if you haven’t tried this utility as a part of your baselining efforts yet, we definitely would recommend that you check it out. It’s another one of those nice auditing goodies Microsoft has built into the operating system for us.

PowerShell Audit One Liners

Over our last few posts we’ve talked a lot about using Unix BASH scripting to audit Unix systems. But we certainly don’t want our Windows friends to feel left out. The more I talk with people and listen to their security challenges, the more interest I hear about how to use PowerShell for audit or security purposes. Who knows, maybe it’s your New Year’s resolution to learn PowerShell this year and integrate it more into your audit activities. Well if it is, maybe we can help to inspire you and get you started on the right foot.

Just like last month, we thought we would post scripting one liners that you can use to query information about a system you’re auditing. These one liners also work very nice in incident response scenarios as well if you find your self in that situation.

For consistency’s sake, I’ll start by following the same script we used on Unix the last few months. As a first step, what commands might someone issue in order to gather general demographic information about a Windows system they’re auditing using PowerShell. Here’s a few to get started:

Display the name of the system:

(get-wmiobject win32_computersystem).Name

Display the domain name of the system:

(get-wmiobject win32_computersystem).Domain

Display the CPU installed in the sytem:

(get-wmiobject win32_processor).Name

Display the CPU speed of the installed CPU:

(get-wmiobject win32_processor).MaxClockSpeed

Display the installed physical memory:

(get-wmiobject Win32_ComputerSystem).TotalPhysicalMemory / 1GB

Display the available memory on the system:

(get-wmiobject Win32_OperatingSystem).FreePhysicalMemory / 1GB

In all these cases so far we’re using the Get-WMIObject cmdlet in PowerShell to gather general demographic information. The nice thing about running each of these commands in PowerShell is that you can easily place them all into one script and you aren’t dependent on any OS specific or version specific binaries being present on the system. As long as PowerShell is available on the system (which certainly most all Windows boxes should have it by now), you’re able to use these commands.
We’ll post more ideas to add to your scripts later, but hopefully scripting is on your list of things to learn this year and we can give you a little shove in the right direction.

Audit Script to Detect Unix Operating System

In the last few blog entries we have been focusing quite a bit on displaying information from a Unix system via a BASH script. One question that’s come up by quite a few people is, do these commands work on all Unix system? That’s a very valid question.

It turns out that one of my favorite sayings about Unix is that “Unix systems are always the same, they’re just different.” In other words – most Unix flavors share all sorts of similarities. The problem is that there are often very subtle differences between flavors of systems that make it difficult to write one script that will work on all systems. Basically when writing an audit script you have to decide, do you write one script for all flavors and sacrifice on which commands you run to make it consistent, write multiple scripts (one per flavor of Unix), or do you try to do OS detection within your script.

If you decide to do OS detection, I know there are multiple ways to do it. But here is some basic code that we have used in the past in a BASH script to detect which operating system or flavor of Unix is running on a system:

if [ -f /etc/debian_version ]; then
VER=$(cat /etc/debian_version)

elif [ -f /etc/redhat-release ]; then
OS="Red Hat"
VER=$(cat /etc/redhat-release)

elif [ -f /etc/SuSE-release ]; then
VER=$(cat /etc/SuSE-release)

OS=$(uname -s)
VER=$(uname -r)

echo "Operating System Name: $OS"
echo "Operating System Version: $VER"

Like I said, this likely won’t work on all flavors, you’ll need to test it out on your favorite to make sure it works. In fact if you have suggestions to improve it, please submit them to [email protected] and we’d be happy to update ours too. Happy scripting!