Critical Incident Response and CIRT Development

Critical Incident Management

In modern organizations, the combination of easily available data, poorly administered safeguards, and malicious individuals make systems vulnerable and attractive to attacks. Almost daily, we hear of businesses being robbed of critical information assets or suffering outages through virus infections or denial-of-service attacks. Computer networks are still relatively new, having their birth only 30 years ago. It sometimes seems hard to put in perspective, but the vaunted Information Highway was just getting its feet of the ground in the early 1980s. And, as information became an extremely valuable commodity, the exploitation of vulnerabilities seemed to keep pace with the growth of network systems.

Illustrating this point is one of the most famous misdeeds, the 1988 "Morris Worm" incident resulted in a significantly large percentage of the network systems with Internet connections being corrupted and removed from service. This was the catalyst that caused Internet users to have postmortem meetings where they decided that preventative, detective, and corrective steps had to be made active parts of their business practice.

For the past seven years, the Computer Crime and Security survey has been conducted jointly by the Computer Security Institute (www.gocsi.com) and the Federal Bureau of Investigation's San Francisco, California, office. The purpose of this survey is to raise levels of computer system awareness while measuring the magnitude and frequency of computer crimes. The 2002 survey results are based on 503 responses from computer security professionals practicing in U.S. business and government agencies. Responses to this survey confirm that computer systems threats continue to spiral upwardly with corresponding financial losses following.

Here are a few highlights from the most recent survey:

• 90 percent of the survey respondents detected computer security breaches in the last twelve months.

• 80 percent acknowledged financial losses attributable to the computer security breaches.

• Of the 503 respondents, 44 percent estimated their financial losses at more than $455 million.

• The most serious financial losses occurred through the theft of proprietary information with 26 respondents reporting more than $170 million and 25 respondents reporting more than $115 mission in financial fraud.

• Of the respondents, 74 percent reported their Internet connection as the more-frequent point of attack than their internal system.

• In 1996, only 16 percent acknowledged reporting intrusions to law enforcement, but in 2002, 34 percent reported their intrusions to law enforcement authorities.

• 40 percent detected systems' penetration from outside the organization.

• 78 percent detected employee abuse of Internet access privileges or inappropriate use of e-mail.

• 38 percent suffered unauthorized access or misuse of their Web sites in the last 12 months, while 21 percent reported they did not know if there had been unauthorized access or misuse.

Patrice Rapalus, CSI Director, remarked that the Computer Crime and Security Survey has served as a reality check for industry and government:

Over its 7 year life span, the survey has told a compelling story. It has underscored some of the verities of the information security profession; for example, that technology alone cannot thwart cyber attacks and that there is a need for greater cooperation between the private sector and the government. It has also challenged some of the profession's 'conventional wisdom;' for example, that the 'threat from inside the organization is far greater than the threat from outside the organization and that most hack attacks are perpetrated by juveniles on joy rides in cyberspace. Over the 7 year life span of the survey, a sense of the facts on the ground has emerged. There is much more illegal and unauthorized activity in cyberspace than corporations will admit to their clients, stockholders, and business partners or report to law enforcement. Incidents are widespread, costly, and commonplace. Since September 11, 2001, there seems to be a greater appreciation for how much information security means not only to each individual enterprise but also to the economy itself and to society as a whole. Hopefully, this greater appreciation will translate into increased staffing levels, more investment in training, and enhanced organizational clout for those responsible for information security.

Experience Note

The most frequent system attacks originate outside the business organization, but the most successful attacks are those committed by insiders.

Critical Incident Response

The best response to critical incidents is characterized by the "ounce of prevention is worth a pound of cure" philosophy. It is much more financially prudent to implement a sound risk management program characterized by written policies, procedures, and standards, with compliance ensured by comprehensive and unannounced audits, than it is to deal with financially devastating events after they happen.

But there are times when "bad things happen to good people" and a response must be made to a critical incident occurring despite your best efforts. It is virtually impossible to predict when someone is going to attack your system and steal your critical information except to say it is not a matter of if as much as it is a matter of when.

Firefighter Response Model

Responding to a critical incident is similar to responding to a fire. Fire departments work tirelessly to educate us about the best means to prevent fires. Safety training starts with simple programs when we are young by talking about fire-related hazards at home and school. Television and radio public service announcements tell us of the safety measures we can take to safeguard our lives at home. We see fire-safety slogans telling us "only you can prevent forest fires" and similar signs as we enter campgrounds and picnic areas. Sometimes we are visited by Fire Marshals inspecting our facilities, making certain there are marked exits and equipment to extinguish fires and save lives.

When the worst happens, a company of firefighters responds to an emergency:

• Respond to emergency contact numbers

• Trained to handle wide-ranging emergency situations

• Organized in the deployment of their tactics and equipment

• Frequently cross-trained as Emergency Medical Technicians

• Confirm that an emergency exists and the nature of it

• Take all appropriate steps to control the emergency

• Take all appropriate steps to prevent the fire from destroying priority order:

  • Lives

  • Surrounding property

  • Property where the fire is presently burning

• Take every possible step to collect and preserve evidence of criminal behavior but not at the risk of life and property

• Testify at judicial proceedings about their actions and findings

• Conduct reviews and critique improving their performance

Critical Incident Response Strategy

No one would argue that responding to critical system incidents is a complex area that is not as easy as taking a pill and waking up feeling better in the morning. Critical Incident response methodology closely follows that of the firefighters:

• Precritical incident preparation. Designated and specially trained response personnel, contact methods, equipment, and tool availability and response posture.

• Detection of critical incidents.

• Initial response evaluation. This is a preliminary step in which an initial investigation is performed and an evaluation is made quickly to determine which type of response is appropriate.

• Response. This is the step where necessary resources are deployed responding to the critical incident. The response goals are very similar to those of the firefighters: contain the damage, prevent it from further spreading, dedicate efforts in a priority manner, and pursue resumption of normal operations.

• Response posture strategy. This step is where the preliminary facts are ascertained and a "best response" plan is proposed. At this time, the proposed plan is passed to senior managers for their review and approval. It is imperative that this step be accomplished within the framework of response demands and priorities. Time is of the essence, dawdling is not acceptable here. Depending on the nature of the emergency, there will be times that an immediate hammer-to-nail response is made and there will be times when the matter may be handled the next business day.

  Experience Note

  Be careful of "crying wolf" too frequently; if every case is declared an emergency, there are no emergencies.

• Law enforcement notification. Having previously established a relationship with law enforcement authorities, responders know whether they should collect the evidence first, or secure the crime scene and wait for officers to respond.

• Legal determination. Responders must include their legal counsel in the decision process surrounding response strategy. On receiving the responder's observations and recommendations, legal counsel should be prepared to render an opinion whether the responders should collect evidence for future legal proceedings, notify law officers so they can collect relevant evidence or take immediate steps to correct damage and restore operations possibly destroying evidence. It is possible that in destroying evidence that responders are violating laws or regulations by not preserving evidence and not coordinating their efforts with law enforcement authorities. For this reason, senior managers and legal counsel must be part of the decision process.

• Evidence collection. This step collects key evidence with interviews, photographs, sketches, and physical evidence.

• Forensic duplications. This step provides bit-by-bit, forensically sound, duplications of critical media.

• Recovery. Responders take appropriate steps to isolate, contain, recover from the incident, and resume business operations.

• Reporting. Take appropriate steps to draft accurate and timely reports to stakeholders and law enforcement authorities, where applicable.

• Postmortem. This is the after-action critique and report of the actions taken during the critical incident response.

Critical Incident Planning

• If you do not plan, you're planning to fail.

Writing and implementing a critical incident plan ensures that emergencies are addressed carefully, thoroughly, and in conformity with risk management programs. As part of the response plan, draft checklists where common incidents are addressed minimizing the required time for response actions. For example, having a response checklist addressing a workstation virus will be significantly different from an employee who is discovered stealing intellectual property and e-mailing it to a competitor.

Here are some recommended elements for a critical incident response plan:

• Obtain and follow the organization's risk management plans. If your organization does not have one, today is an excellent time to start one. This plan should provide details relative to the priority of critical assets, their restoration, and the steps to be taken for resuming profitable operations.

• The critical incident response plan should outline the means of detecting emergencies, collecting preliminary information, assessing the gravity of the system attack, systems affected, spread of damage, steps necessary to stop damage, and protect personnel, data, and facilities. The recommended plan structure is simple, direct, and understandable.

• The critical incident response plan should provide a means to easily contact all relevant employees and outside resources.

• The critical incident response plan should provide specific instructions about policies, procedures, and legal requirements.

• The critical incident response plan should provide templates for any documents required during the emergency. For example, the plan should include a template for logging responder's actions and significant events during the response.

Many critical incident response plans fail because they do not include a response-owner and a senior management correspondent as part of the process. A response-owner is the employee responsible in most cases for the response the emergency receives including relevant actions from start to completion. The senior manager correspondent is the employee who will deliver information to stakeholders.

Command Post Operations

This is a sensitive topic relative to the initiation, staffing, and operation of a command post. Do not think that CPs are intended only for military or government operations because all agencies, while addressing emergency situations, should consider this response strategy. Basically, a CP is a temporary business unit assembled to address one of more crises and will remain in operation until all emergencies are stable and settled. CPs work very closely with regular business operations but have the executive "horsepower" to function independently in decision making, assigning resources, taking action, and following up.

CPs are staffed with specialists assigned particular tasks with dedicated resources at their disposal. In their most common configuration, CPs are housed in segregated facilities located within the business' headquarters. If this is not possible, plans should include relocating the CP to a secondary and equipped facility. They should be equipped with dedicated facilities such as office space, electrical generation, high-speed satellite-linked Internet connections, telephones having multiple direct lines separate from other business units, satellite-linked television for news reception, and a LAN connecting CP workstations to the business LAN and the Internet.

CP reporting structure is funnel-shaped. Information flows from telephone calls, radio, news broadcasts, and e-mail to those designated for information processing. Telephone callers may be employees, specialized response teams, members of the press, stakeholders, or the general public. Carefully trained employees are tasked to interview outside callers and collect information. They are trained relative to the information they may disclose because any comments will be attributed to the organization.

Individuals collecting information for the CP should complete a simple contact report form synopsizing the information from their call, news broadcast, or e-mail. This form may be paper-based or electronic with one copy being passed to the function-point (the single point where all collected information flows), another copy is passed to the data input unit, and the last copy is retained and archived as "work papers." If it is significant, she immediately briefs the function-point and follows the briefing with the written contact report form.

The function-point unit is the person or unit that screens incoming information and makes a determination of where the information should be routed, its priority and processing action. The function-point is a critical position requiring decisions to be based on sound business sense. The data input unit is responsible for routing the information to the unit or employee assigned to the task by the function-point. Another unit must be responsible for collecting the work papers and organizing them for future review and retrieval.

Within the CP are several critical business unit representatives. Depending on the nature of the emergency, these are suggested units that should have representatives in the CP:

• Legal

• Human Resources

• Public/Media Relations

• Senior Management

• Operations Staff

• Maintenance Staff

• Supply/Logistics Staff

• Communications Staff

• Data Input Staff

• Function-Point staff

At least in the initial stages, it will probably be required that the CP is open and staffed for 24 hours.

Experience Note

CP staff will have stages of burnout. Replace all staff members at the end of their 8-hour shifts. At the end of shifts, there should be a briefing by the outgoing shift of the events so the oncoming shift knows what has happened during the past eight hours.

There is a good reason to maintain an events log — so the oncoming employees can review it for reference purposes. Activity logs and other work papers could be made part of legal actions, so care in this area is advised. Employees should be trained that documenting facts is acceptable, while documenting opinions or editorializing are not.

Experience Note

While working in a CP, an employee made a note that was later maintained as a work paper about an event that was only hypothetical and not actual. However, when legal action was sought, the plaintiff introduced the note was as if the event actually happened. Despite the defendant's protestations and objections, the note was accepted as evidence causing significant damage to the defendant's case.

Once the emergency begins to abate, staff, duty-hours, and activities can be reduced. It is a common practice having CP unit leaders meet every half-hour during the first few hours of CP operations. At this time, they should bring important events to briefings along with any concerns. Meetings should last not more than a few minutes and are driven by the nature and treatment of the emergency.

CP employees should understand that press inquiries can have grave consequences for the organization. They should be trained to handle press calls in an appropriate manner. For example, in the face of a disaster, the CP receives a telephone inquiry from a noted news organization; the employee handling the call accepts the information and documents the inquiry by completing the contact report form. Once completed, the form is passed to the function-point where it is screened again and passed to the public relations unit at the CP for handling. One copy of the intake report is passed to the data-input unit that is creating a chronology database of events, and while making an assignment to the public relations unit with a request, they respond when the assignment is completed. In this fashion, assignments can be tracked whether they have been completed or not. Frequently, the input unit will list all uncompleted assignments and pass them to the function-point that will screen them again deciding if they need to be completed in light of the most recent events. Once the public relations unit receives the assignment from the function-point, they contact the news organization and provide appropriate information.