Theme, The role and environment of IT Security people
Reference, Chapters 1 - 3 of text, Notes, Chapter 1 - The Security Mind, security best practices, know ourselves and our attackers, applies principles and rules in dynamic environment, cf. time management, focus on doing right thing rather than being concerned and busy, to thwart new security reality of technology, constant risk, problems with technology are not all technical, security if a difficult problem to grasp, Chapter 2 - Take a new Look, Security is an Art, young and dynamic, sub-discipline, Environment, us, them, batteground, The Fear Factor, overly large consideration in, security decision making, combat with defined security management, follow, virtues, and, rules, Chapter 3 - The 4 Virtues, Daily Consideration, Community, Higher Focus, Education
Theme, Universal principles to support decision making
Reference, Chapters 4 & 5 of text, Notes, Chapter 4, The 8 Security Rules, universal principles supporting decision making, explains, comments and applies, act as filters in complex situations, encoded into policies and procedures, Chapter 5, Higher Security Mind, core practices, architectures, zoning, checkpoints, layering, defense in depth, many architectures here
Theme, The process of crafting Security Decisions
Reference, Chapters 6 & 7 of text, Notes, Chapter 6 - Security Decision Making, Role of policy, Decision-making process, Identify components, scope, context, assets, Identify Risks and Threats, Compare to rules and policies, Make Decision, Example, Threat & Risks to WAN connection, reasonable level of detail, detailed but unstructured, Consider, General rules & principles, Architecture, Zones, Layering, General context, Chapter 7 - Know the Hacker, List of Hacker categories, considers them as threat agents, Vulnerabilities, what threat agents exploit, systematic approach in NSW threat and control list, ISF tools, says DoS is popular, most popular, Domains, Network, Physical, Software, OS, Applications, OWASP, Dependencies, nice diagram, demonstrated non-linear nature of vulnerabilities, Targets, DNS, Email Servers, Web Servers, Dial-up modems, Exploits, specific attacks methodologies, Two case studies, full explanation of attack exploit and why succeeded, Profile development, for a company, which hackers, what assets, reducing attack surface (essentially)
Theme, Eight rules of security by showing how they can be applied to many real-life scenarios
Reference, Chapters 10 -12 of text, Notes, Chapter 10 - Modern Defences, Firewalls, IDS, Vulnerability Scanners, Open Source, Wireless, Encryption, VPN, Chapter 11 - The Rule in Practice, Perimeter Defences, Internal Defences, Zoning, Physical Security, Hardening, Logging, Outbound connections, authentication, Chapter 12 - Going Forward, Summing up the book
Theme, Take a more formal look at the science of risk management, which includes both quantitative and qualitative techniques
Reference, Additional readings to the textbook, Henry, K. (2004). Risk management and analysis., Notes, Is RM merely a hot topic or something of benefit to business?, Consider the relation Risk = Threat * Vulnerability * Asset Value, bully in playground example, not overly helpful, but does explain Threat, Vulnerability, Asset Value, Quantitative RM, use for some stakeholders, accounting and auditing, difficult to quantify some aspects, such as damages on assets, in general, the impact on people, social costs of an incident to, shareholders, customers, regulatory bodies, Qualitative RM, scenario-based, likelihood and impact rated by "experts", ratings are qualitative, high, medium, low, relative rankings, represent ranges not values, Still suggests to follow the CIA triad, generic impacts, generate assessments from control frameworks based on CIA, general approach, generate scenarios, quantify costs of impact for each scenario, The Three Keys, Knowledge, sound knowledge of threats and vulnerabilities in environment, what scenarios are involved?, Observation, collect data to further identify and callibrate scenarios, Business Acumen, which scenarios are the biggest risk to business, how to convince business to act on mitigating these scenarios, decision support, Risk Responses, Reduce, Add one or several compensating controls, from a possible list or pool, must be maintained over time, Outsourcing, the RM function, choose not to have in-house experts, Assign, mitigate risk through third party, insurance, service level agreement, someone else runs your network to 99.999% availability, Kaplan, R. (2004). Risk management 101., Notes, Start with a diagram, Balancing Risk and Cost, Is this a graph or a graphic?, Need definitions to proceed, Vulnerability, weakness, Threat, something that exploits weaknesses, Risk, essentially same as the equation in first reading but they do not say that explicitly, Countermeasure, the traditional name for what we now call a control, The Basic Issue, How to balance, the expense of incurring cost from risk materializing, against, the cost of mitigation - reducing the likelihood of the risk materializing, How to apply to IT infrastructure?, direct simplistic application is daunting, vulnerabilities multiply quickly, impossible for all but smallest companies, suggests there is outmoded thinking here, use the Don Parker framework, How to analyse and quantify vulnerabilities?, only way forward, but many proposed methods have failed, hard to have common risk rating everywhere, yet cannot combine risks without common scales, quantification is sought in general, management and bean counters seek it, technical people seek it, judging between what is Necessary vs. Sufficient, don't spend more than something is worth to protect it, there are few hard numbers around security, hard to justify security budget, in particular, increasing the budget, Further discussion of Risk-Cost curve, adds more detail with new diagram, more qualitative terms, more scales, more mental models, gives Do-It-Yourself advice, essentially a method to gauge risk appetite, opines that this remains a creative process for experienced risk practitioners, The Bottom Lines, gives a bottom-up approach to translating infrastructure risks into business risks, I would say this is really a top down process written in reverse, Gaining Trust, provides a diagram showing the overview of trust process from the US Government Accounting Office (GAO), probably now superseded by risk management frameworks
Theme, Security requires trade-offs, and that these trade-offs are subjective.
Reference, Additional readings to the textbook, Schneier, B. (2003). Security trade-offs are subjective. In Beyond fear: Thinking sensibly about security in an uncertain world, Notes, perfect security cannot be obtained, how to deal with inherent trade-offs in decision making, Extreme trade-offs, grounding all planes tops 9/11, done temporarily, No credit card, prevents data theft, what are we getting and what are we giving up?, how to make sensible trade-offs?, mentions threat and risk, risk being product of likelihood and impact (seriousness), RM is about playing the odds, suggesting of quantification, but more, but RM is about placing bets, on what is worth worrying about, against what is not, how likely are the threats we are facing?, goal not to eliminate risk but bring to a manageable level, not so well-defined what this means, Security risks is just one type of risk a company faces, think of the risks with a global supply chain, for oil or clothing or computer components, insurance, converts a variable cost risk into a fixed expense, risk is situational, companies, want to maximise profit, therefore, they want, adequate security at reasonable cost, RM is really about cost-benefit analysis, but analysis is subjective, some people do not want nuclear power no matter how remote a risk of a meltdown, the downside is too down, so down that cannot even consider, the presence of well-defined countermeasures make no difference, think of Fukishima, similar opinions in, genetic engineering, intuition may not be a good guide to risk decision making, people exaggerate spectacular rare risks, as do media, estimating risk is bound to your personal context, perception, personified vs. anonymous risks, are hackers anonymous?, control, when we choose risk is lower than we we are forced, media influences our notion of risk, what is risky, plane crash seems more risky than car crash but not really, Often confused perceived risk against actual risk, represents a fundamental disconnect, more people get killed by pigs than sharks each year, psychological basis, Technology progress outstrips our ability to understand it, Average user, no idea about the relative risks of, giving out a credit card number to a Web site, sending un-encrypted email, sharing folders, and so on, emphasis on what is visual, what gets amplified, plane crash vs. diabetes, media has trouble working with non-visuals, table of top killers, not what we would think, Blakely, B., McDermott, E., & Geer, D. (2001). Information security is information risk management. Proceedings of the 2001 workshop on new security paradigms., Notes, Risk, possibility of an event that reduces business value, ALE, Annualized Loss Expectancy, cost * likelihood happens in a year, Managing Risk, Liability transfer, Indemification, pooling, hedging, Mitigation, Retention, Information Security, who should do what with sensitive information, expressed in policy, how to implement and enforce?, collections of "measures", controls, The Problem, current measures not working, process is biased, towards technical measures, against physical measures, reduce likelihood rather than impact, Effectiveness, FBI survey data, incidents rising despite security technology deployment, Quantification, qualitative methods well developed, high/medium/low, Quantitative methods weak though used elsewhere, finance, healthcare, safety, lack of good data is a problem, agreed to by insurers, but policies are being offered, however they have been caught out by technology advances, largely absent in Basel II, operational risk framework, need data on, vulnerabilities, incidents, losses, control effectiveness, Current situation, equivalent of 19th century medical practice, poor understanding of health in general, much different today, dramatic increased professionalism, education, study and collection of public health data, studies on effectiveness of treaments, do the same for security, Managing Risks, IT RM people, trained not educated, not true professionals in any sense, want to be like a doctor or lawyer, become quantitative, measure, collect, record and report, overhaul of how security technology is evaluated, Reading 6: Gerber, M., & von Solms, R. (2005). Management of risk in the information age. Computers and Security, Notes, Start with remarks on history of risk, discussion of natural and social sciences, risk for society and individual, modern definition, ‘the probability that a particular adverse event occurs during a stated period of time, or results from a particular challenge’’, 1995 publication of National Institute of Standards and Technology (NIST) handbook’s definition of risk which describes risk as ‘‘the possibility of something adverse happening’’, Risk Analysis process, identification, estimation, for IT environments this is often simply guessing, evaluation, long discussion on theoretical basis of technology risk management, as compared to other disciplines
Theme, Assigning a probability to the chances of an attack as well as determining how much damage a successful attack is likely to cause. This topic explores ALE (Average Loss Expectancy) and ROSI (Return on Security Investment).
Reference, Additional readings to the textbook, Ozier, W. (2004). Risk analysis and assessment. In Information security management handbook, Notes, Core questions for RM, Analysis, What could happen?, threat, If it happened, how bad?, impact, How often could if happen?, frequency, How certain are you of these answers?, uncertainty, estimation, Mitigation, What can be done?, mitigations, How much will it cost?, costings, Is is cost effective?, cost-benefit analysis, Terms and definitions, ALE, frequency of occurence, annualised, ARO, Annualised Rate of occurance, think 30-year flood, happening this year, EF, Exposure Factor, magnitude of loss, Information Asset, something to be protected by IT, Risk, as above in Analysis, plus, Analysis, Treatment, Management, Safeguard, countermeasure, control, effectiveness, how well it works, Central task in IRM (Information RM), establish policy, create IRM team, establish tools and methodology, identify and measure risk, establish risk acceptance criteria, mitigate risk, monitor risk, Process benefits, understand risk position, remove ignorance, arrogance, fear, Qualitative vs. Quantitative, problematic process for quant assessment in US government, no one kept common data or metrics, defaulted to qualitative, needed to collect risk metrics, 6 primitive elements, Asset Value, Threat Frequency, Threat Exposure Factor, Safeguard Cost, Uncertainty, Pros, Qual, simple calculations if any, assets need not be valued, mitigations need not be costed, Quant, process based on objective metrics, produced meaningful statistical data, uses monetary values to express assets and actions, credible basis for costings, process can be tracked over time, help stakeholders make decisions, Cons, Qual, subjective, cannot allocate resources according to a budget, cannot evaluate objective improvements, Quant, complex, black box calculations, are the numbers credible?, must be supported by tooling, rules out light weight assessments, large preparation, not standardised, Skeptical Stakeholders, provide examples of quant assessment being more convincing to management, management cannot make decision based on high/medium/low, Tasks of Risk Assessment, project sizing, how much effort will the assessment take?, threat analysis, perhaps with tooling, use standard lists here, asset identification and valuation, tangible assets, intangible, invokes CIA as well, threat-vulnerability-asset mapping, risk metrics, 6 primitive elements, Asset Value, Threat Frequency, Threat Exposure Factor, Safeguard Cost, Uncertainty, risk mitigation, costing, gives list of automated tools for this process, about 20, must be new ones by now, Geer, D., Soo Hoo, K., & Jaquith, A. (2003). Information security: Why the future belongs to the quants. IEEE Security and Privacy, Notes, Fundamental disconnect between security rhetoric and security spending, Security people cannot answer simple business questions, Is security better now than last year?, What value am I getting for my money?, How do I compare with my peers?, Risk Management, the only way forward, FUD is silly and non sustainable, what is reducing FUD, companies run on information, how is it managed?, demonstrating security, what are improvements?, cost justification, why is it this much?, accountability, regulations defining security, Data Sharing Roadblocks, everything from lack of common metrics, Verizon report, professional embarassment, couldeven be illegal, anti-trust laws, What should be gathered?, whatever supports business decision making, at a minimum the output of security tools, Data Models to steal, from other areas, including, Quality Assurance, Public Health, Portfolio managament, Accelerated test failure, Insurance, Hoover project, example of pooled data, for application security, based on OWASP, analysis using, Business adjusted risk model, simple, mix of qual and quant, The Future is here, people want numbers, we have numbers, no will to use them, trend data, what is happening in the short term, you can do this, quants will own the industry, Endorf, C. (2004). Measuring ROI on security. In Information security management handbook, Notes, Discuss from basic methods for finding ROSI, basic issue, hard to get security money before an attack, but easy afterwards, many managers use FUD, without money, hard vs. soft return, 7 steps value, like above, Asset identification and valuation, EF, SLE, ARO, ALE, Control survey, Calculate ROSI, Small example given for protecting confidential data, Berinato, S. (2003). Everything’s coming up ROSI. In CIO: Australia’s magazine for information executives, Notes, article from trade magazine, start with an example of patching, how to justify buying automated patching, some views that ROSI not worthwhile trying, endless path, lots of legwork, counterview, there are ROSI tools, just do not use them for security, management want hard numbers, not available, or cannot calculate, what is the ROSI on fire extinguishers?, $3 for every $1 invested, According to one study the American Society of Safety Engineers (ASSE), Step 1, Do a rethink, Precision is not the goal, One of the reasons that ROSI might feel like an endless path comes from the fact that there has been a natural tendency in the tech sector toward approaching problems with the precision a software engineer would expect., The dogmatic IT mind-set must be eliminated., obvious why IT tends to approach problems with binary thinking., Step 2, The Legwork, talk to your teams, make use of the useful data out there, play detective for what else you need, build a threat profile of your company, calculate conservatively, know your stakeholders, Step 3, The Math, In the end, the maths is simple. You subtract cost from benefits, seriously?, suggests ALE and a few other metrics, Jacobson, R. V. (2002). Risk assessment and risk management. In S. Bosworth & M. E. Kabay (Eds.),
Theme, Qualitative risk assessment relies more on observational, subjective data rather than hard facts. There are many advantages to this approach. ‘Hard numbers’ are often difficult to come by when assessing security threats.
Reference, Additional readings to the textbook, Peltier, T. (2005). Quantitative versus qualitative risk assessment. In Information security risk analysis (2nd ed., pp. 77-114)., Notes, Introduction, qualitative, good for prioritizing risks but not measuring costs vs. benefits, quantitative, good for prioritzing based on cost but measurements may end up having a qualitative interpretation, GAO report finding, RM works best when have a simple process that can be applied to many aspects of an organisation, involving business and technical people, Qualitative, Outlines a 10-step process for Qualitative Risk Assessment, 1. Scope Statement, 2. Assemble a Team, broad representation, see Table 4.4 for example team, 3. Identify Threats, Long table given, 4. Prioritize Threats, 5. Threat Impacts, 6. Risk Factor Determination, threat * probability * impact, 7. Identify Safeguards and controls, 8. Cost-Benefit Analysis, 9. Rank Safeguards, 10. Risk Assessment Report, Augment 10-step process with tables, give numerical ranges which can be applied to all costings, value of assets, cost of disruption, reputation cost, Threat Evaluation Matrix, tabulate probability against impact, identify critical High/Medium/Low regions, The 30 minute approach, ISRA, a facilitated process, uses CIA as the basis, based on a simple 2 x 3 matrix, Munteanu, A. (2006). Information Security Risk Assessment: The Qualitative Versus Quantitative Dilemma. Paper presented at the Proceedings of the 6th International Business Information Management Association (IBIMA) Conference., Notes, Introduction, Issues in Information Security RM, inconsistent or too general definitions, lack of rigour, subjectivity, lack of up-to-date data, different standards that are essentially the same, discovered the same principles, reviews, NIST, OCTAVE, OECD, ISO, Qualitative Limits, lack of good data, estimating likelihoods of threats is hard, subjective, Quantitative Limits, better to have some quantitative measurements than none at all, review of, ALE, ARO, SLE, EF, "It is not easy to apply these formulas", mention Canadian Risk Assessment Guide, get from Scribd or Beat
Theme, Taking out an insurance policy is the most common approach for transferring risk. Cyber insurance, however, is still in its infancy and faces many challenges before being fully accepted as a market solution in the same way as traditional insurance.
Reference, Gordon, L. A., Loeb, M. P., & Sohail, T. (2003). A framework for using insurance for cyber-risk management. Communications of the ACM, 46(3), 81-85., Written 2003, which is 10 years ago, need an insurance approach, since cyber losses cost real business money, not so different from fire, there is liability with connecting to the Internet, connectivity risk, Describe a cyber-risk framework, some characteristics are unique, the attacker can be far removed, what is the extent of damage from a virus?, breaches are hard to detect, need new insurance products, which are being offered, Product considerations, Pricing, no "actuarial tables" available, current pricing strategies probably involve capping payouts, Adverse Selection, how do you identify "good risk" customers?, normally a security audit of a potential customer is required, pricing or selection may vary based on customer sector or technology choices, Moral Hazard, will taking out an insurance policy reduce commitment to best practices and keeping up to date?, use big deductibles, reduce premiums after mediation taken, insurance companies teaming with tech companies, Coverage, First Party coverage, to the company itself, Third Party, to someone else, capped payout limits, but still in the millions, expect to rise over time with more data, When to use?, perform risk assessment, mitigate, identify residual risk, insure if you find an appropriate policy, Majuca, R. P., Yurcik, W., & Kesan, J. P. (2005). The evolution of cyberinsurance., 2005, a bit later, The Issue, Insurer wishes to reap profit from premiums that exceed payouts, Insured wants to mitigate cyber risk and move on to other business activities, business case, Pricing, finding the optimal value is difficult, valuing intangibles, threat can change regularly and quickly, Spreading Risk, insurance works best when customers are all independent and losses are not related, several worms and malware have struck many companies at once, Cyber risks are excluded from existing policies, tested in court, no physical damage may invalidate a given policy, Initial policies joint offering between tech company and insurance company, small capped exposure, $250,000, table given, mostly from the late 90's, Drivers for new products, new demand after 9/11, global worms such as Code Red, DDoS, gathering of statistic to show how widespread problems are, increased legal requirements for the retention of information, post Enron, business people involved as they could go to jail, see Table 3, specialised, narrower coverage, options for exclusion, options for differentiation to customers, Problems to be addressed, Adverse Selection, unable to determine if applicant is high-risk or low-risk, but applicant normally knows this, pricing diagram hard to follow, require thorough risk assessment for applicants, questionaires, physical assessments, can be costly and discourage applicants, Moral Hazard, insurance provides dis-incentives to continue with good security posture, addressed by exclusions, see Table 5, capped liability
Reference, Slovic, P. (1987). Perception of risk. Science, 236(4799), Notes, Risk Perception, Judgements people make when characterize and evaluate hazardous activities and technologies, the survival instinct writ large, What we have, chemical and nuclear technologies, catastrophic consequences, rare and often delayed, difficult to assess by statistics, not well suited to management by trial and error, underlying mechanisms, complex technologies that are, unfamiliar, incomprehensible, analysts can use RM, but non-specialist use, risk perception, which are subjective judgements, often informed by media, what do people mean when they say something is "risky", important to health and safety policy makers for example, Research, Sociology, hazards mediated by social influences, friends, family, co-workers, Psychology, rooted in assessing, probability, utility, and decision making, identified mental models, that give us, heuristics and, biasses, Dobelli book, initial views strongly resistant to change, even given evidence, search for confirmation not contradiction, Psychometric Paradigm, discover expressed preferences for risk, Table 1, ranks the risk of 30 activities or technologies by 4 groups, no IT Risk!, Factor Analysis, risk perceptions are correlated, via factors, can explain key characteristics of perception, Figure 1, landscape view of activities or technologies on two scales, Uncertainty, unknown, unobservable, new, delayed manifestation of harm, Dread, perceived lack of control, catastrophic potential, fatal consequenes, uneven distribution of risks and benefits, these two factors define a "factor space", quadrant approach, see Fig 2, cognitive map, horizontal for dread, most important, want this regulated, vertical for unknown, cf. WEF measures, Accidents as Signals, the spread of the impact from an event, traditional measures, loss of life, property damage, Three Mile Island Accident, 1979, no deaths but large societal cost, created huge costs for, company, nuclear industry, regulation, society, not isolated, Bhopal India, Dow, Challenger Space Shuttle, Chernobyl, larger ripple events on people in general, alters their perception of the event in the factor space, train accidents with chemicals sending stronger signals than train accidents with people, more ripple, effort and expense beyond cost-benefit analysis may be required, potential loss of reputation, lawsuits, Public Exceptance, for example, will people accept nuclear power?, generally seen as risks greatly outweighing benefits, some experts see this as irrational, correct but ineffective, nuclear power leads to nuclear weapons, is is possible to educate people to another view, Perspective, express hazards in a unidimentional index, such as death, risk per hour, attempt to explain relative magnitudes of risks, Asgharpour, F., Liu, D., & Camp, L. J. (2007). Mental models of security risks. Lecture Notes in Computer Science, 4886, 367-377, Notes, mental model, internal conception of how something works in the real world, risk communication, messages formulated by experts to warn a community of non-experts against threats, not conveying "truth" but prompting to appropriate action, security awareness, increase efficacy by taking these models into account, differing levels of knowledge between expert and non-expert, make risk communications with this in mind, environmental risks, medical risks, privacy risks, how to extract mental models?, not easy, many methods, card sort, sort cards into piles to express preferences, ran experiments, 74 participants, varying knowledge, provide 66 words, allow participants to cluster, tested these models, security models to be considered, physical security, medical infections, criminal behaviour, warfare, economic behaviour, use multi-dimensional analysis to give visual representation, distance mean dissimilarity, similar to factor space, explain how this is calculated, tables and analysis in Appendix, findings, none of the 5 proposed models has a good fit with the participants, further research is needed
Theme, The relational risk assessment process was pioneered by the author of your text, Kevin Day. It is similar in style to a qualitative risk analysis, but with more emphasis on concepts such as vulnerability inheritance and chained risks.
Reference, Return to the textbook, chapter 8, Notes, Security Audits, very important, proactive tool, think in terms of risk and threats, threat, a bad thing that might happen, risk, the impact of a threat, support decisions, under or over protecting?, Traditional assessments, quantitative, simple example in Table 8.1, qualitative, repeat the example in Table 8.1, Issues, give a formal and repeatable assessment process, nice, drawbacks, quant, how to assign numbers, costs, likelihoods, time intensive, qual, based on opinions, may vary widely, large effort of many people to collect, Wisdom of Crowds, interpretation of results, both, do not scale to large environments, Who or what does scale?, hard to evaluate security relationships, next section, overly complex results, not actionable, not decision-friendly, Relational Security Model, Combination of qual & quant, www.relationalsecurity.com, seems to be doing well, video, ppt, Basic Rules, use reasonable worst case scenario, avoid tunnel vision, avoid existing measures, consistent scale and measures, downtime example, Patterson paper, Risk Levels, measures "degree of risk", like say data classification model, secret, classified, ..., Risk Factors, A risk factor is an individual detail about an object in relation to an organization, factual basis for assigning Risk Level, Controls, specific to each object being considered, Server, Router, Mobile Device, levels here as well, example, differing types of locks on a door, score or rate objects, risk level vs. control, applied level (current) vs. required (planned), Tactical process, this means "lightweight", affordable, accurate, useful, easy-to-comprehend, steps, preparation, object discovery, define scope and what is there, e.g. network IP ranges, risk assessment, understand each object, discover risk factors, discover risk level, control discovery, vulnerability scanning, hands-on audit, desktop audit, physical inspection, compile and score data, report and review, Analytical measures, process above simplifies audit process most of the time, assessments beyond objects, Apply The Eight Rules of Security Defense from Chapter 4, Architecture, Perimeter, Internal, Applications, OWASP, Additional considerations, Acceptable risk, sign-off on residual risk, staffing an audit, One vital requirement for an audit is that the entire team must have a common understanding of the organization, its risk levels, and the related risk factors.
Theme, Is our security better this year than last year? Could you respond? And if so how?
Reference, Jaquith, A. (2007).Defining security metrics. In Security metrics: Replacing fear, uncertainty and doubt (pp. 9-37). Upper Saddle River, NJ: Addison-Wesley., Notes, Conumdrum, Security experts have trouble answering, Is security better this year?, What am I getting for my security dollars?, How do I compare to my peers?, Answered by other businesses, other cost centres, Measurement, only viable alternative to the old-world security model, murky domain of oracles and sooth-sayers, get beyond FUD, drivers, fragile nature of information assets, provable security, improving security is only bounded by what you can pay, cost pressures, competition for the technology budget, accountability, mandated mechanisms for controlling security risks, Security measurement and quantification is a stumbling block for risk, Roadblocks to sharing information, lack of common terms and metrics, vocubulary fraught with imprecision and overlapping meanings, people have considerable strength to argue about this, potential legal issues, general lack of trust, Modelling metrics, modelling vs measures, modelling, mental model to compensate lack of data, think about equations and data, measurement, self-rewarding activity, think about, what actually happened, Table 2.1 shows differences, author admits he is looking for causality as does not have math background, determine from incident data which controls work best, correlation does not produce understanding, good models provide rational for measurement, Other areas to borrow from, quality control, vast topic, public health, again vast, could use concept of herd immunity, Nimda worm, virulence and immunity, create a body similar to the US CDC, http://www.cdc.gov/, electronic analogs to the CDC’s epidemiologic coverage:mandatory reporting of communicable diseases,statistical identification of excess incidence,and longitudinal trend analysis to calibrate epidemiologic models, portfolio management, balance risk and reward across several investements, more control on risk setting and calculate aggregate position, accelerated failure testing, testing by compressing time, lifetime of abuse all at once, compare to, a penetration test, insurance, estimating risk using long term data, assumes some stability in the data set, factors that impact life expectancy are changing, much fluctuation in technology world, cyber insurance covers named risks, how much cascade effect is there in an incident?, how far does the damage spread?, What makes a good metric?, goal is to answer business questions meaningfully, such as " Am I spending the right amount of money?", The primary goal of metrics is to quantify data to facilitate insight., other industry metrics, freight, cost per mile, warehousing, cost per square foot, cable TV, average revenue per user, characteristics, easy to explain and calculate, transparent, expressed in common units, permit benchmarking, compare with, defense in depth, security is a process, no security in obscurity, What makes a good metric?, characteristics given in Table 2.4, consistent, not subjective, repeatable, different from ratings, would two people get the same answer?, cheap to gather, number or percentage, cardinal, size, not ordinal, rank, "traffic lights" and RAG charts are not metrics, use a unit of measure, context specific, must invoke meaning in a context, avoid the "So What?" response, What makes a bad metric?, basically the opposite of above, False metrics, Security Framework measurements, taxonomies, fixation for ISO 17999, it is a safe horse to bet on, Conventional Wisdom, It worked as an audit standard so why not here?, but it is not about measurement in any real sense, ALE, Dan Geer comment, "The numbers are too poor to even lie with", modelling outliers, estimating probabilities, sensitivity to changes in assumptions, math is deceptively simple, practitioners of ALE suffer from a near-complete inability to reliably estimate probabilities or losses., you can make those numbers say anything you want, gives a long personal example on modelling component failures, Fig 2.3