Comments on: Still no evidence… http://www.hubbardresearch.com/2010/02/483/ The Applied Information Economics Company: Consulting, Training, and Execution Wed, 21 Dec 2011 13:49:28 +0000 hourly 1 http://wordpress.org/?v=3.3 By: spoppe2001 http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-294 spoppe2001 Sun, 25 Sep 2011 02:07:43 +0000 http://blog.hubbardresearch.com/?p=483#comment-294 Just finished FRM. I am immersed daily in innumerable IT risk assessment questionnaires, which are now routinely hundreds of questions long and getting longer all the time. The relevance of some questions to the assessment of risk is hard to fathom. As a supplier of IT services we have a finite budget for reducing risk for customers. I hope to use the ideas in FRM to build an objective basis for deciding whether to invest further in a particular area. Comments from you and your readers who have made progress in this area are most welcome. Just finished FRM. I am immersed daily in innumerable IT risk assessment questionnaires, which are now routinely hundreds of questions long and getting longer all the time. The relevance of some questions to the assessment of risk is hard to fathom. As a supplier of IT services we have a finite budget for reducing risk for customers. I hope to use the ideas in FRM to build an objective basis for deciding whether to invest further in a particular area. Comments from you and your readers who have made progress in this area are most welcome.

]]> By: jot1212 http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-261 jot1212 Wed, 22 Jun 2011 21:32:25 +0000 http://blog.hubbardresearch.com/?p=483#comment-261 I am looking forward to the new edition of the books and thanks for you work in the area. Yours book has a lot of key points which are really interesting and I would suggest everyone to read The Failure of Risk Management. I am looking forward to the new edition of the books and thanks for you work in the area. Yours book has a lot of key points which are really interesting and I would suggest everyone to read The Failure of Risk Management.

]]> By: dwhubbard http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-175 dwhubbard Fri, 09 Jul 2010 11:42:51 +0000 http://blog.hubbardresearch.com/?p=483#comment-175 Richard, But of course there is room for debate. And there will be much benefit in it. Your experience with the review process shows a key point I make in the book. It is simply not considered necessary in the PMI culture to provide citations showing measurable benefits of a method before it is made a part of canonical methods. We can imagine what the pharma industry would be like if new drugs had the same lack of standards for demonstrating both safety and effecacy. Thanks for your input, Doug Hubbard Richard,

But of course there is room for debate. And there will be much benefit in it.

Your experience with the review process shows a key point I make in the book. It is simply not considered necessary in the PMI culture to provide citations showing measurable benefits of a method before it is made a part of canonical methods. We can imagine what the pharma industry would be like if new drugs had the same lack of standards for demonstrating both safety and effecacy.

Thanks for your input,
Doug Hubbard

]]> By: Richard Watson http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-172 Richard Watson Fri, 09 Jul 2010 08:42:31 +0000 http://blog.hubbardresearch.com/?p=483#comment-172 Regarding Risk Maps & Matrices (and hence Scoring) these are so deeply embedded in our risk culture there is, unfortunately, no room for debate. I undertook a pre-publishing review for a PRM book where the author left out PI diagrams because he saw no real value in them. This was a decision I fully endorsed. However the other reviewer must have insisted they should be in (unknown to me), as I was very suprised when I recieved my free copy to find a piece of text & diagrams extolling their virtues!! One mapping I have used to illustrate risk (on a purely graphical/name format ..... with effort placed on removing ambiguity) are Impact -Manageability plots, which can generate very different reactions to the effect of PI plots. For example a 90% probability project threatening risk has to be made manageable and the cost of doing that has to be included in the project budgets or bid pricing .... no option! regards Richard Watson Regarding Risk Maps & Matrices (and hence Scoring) these are so deeply embedded in our risk culture there is, unfortunately, no room for debate. I undertook a pre-publishing review for a PRM book where the author left out PI diagrams because he saw no real value in them. This was a decision I fully endorsed. However the other reviewer must have insisted they should be in (unknown to me), as I was very suprised when I recieved my free copy to find a piece of text & diagrams extolling their virtues!!

One mapping I have used to illustrate risk (on a purely graphical/name format ….. with effort placed on removing ambiguity) are Impact -Manageability plots, which can generate very different reactions to the effect of PI plots. For example a 90% probability project threatening risk has to be made manageable and the cost of doing that has to be included in the project budgets or bid pricing …. no option!

regards
Richard Watson

]]> By: winslow7 http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-151 winslow7 Tue, 11 May 2010 14:51:06 +0000 http://blog.hubbardresearch.com/?p=483#comment-151 Mr: Hubbard: I have very much enjoyed your book and approach to measurement of risk. I have a question regarding the measurement of impact (for a variable or event) as a means of calculating EOL where the impacted entity is not a for-profit group (it could be either a government agency or a non revenue profit center in a company). I have received significant pushback on the insistence that any impact does have a value that is quantifiable here. I have taken the approach of valuing the minimum 100% impact as being the budget of the not for profit entity itself in a given year or over the period of the risks being measured. My rationale is that funds are given to the not for profit entity with the expectation that the entity should be producing benefit that is at least equal to their budget. Any thoughts on how you have handled this in the past? Thanks, Dan Mr: Hubbard:

I have very much enjoyed your book and approach to measurement of risk.

I have a question regarding the measurement of impact (for a variable or event) as a means of calculating EOL where the impacted entity is not a for-profit group (it could be either a government agency or a non revenue profit center in a company). I have received significant pushback on the insistence that any impact does have a value that is quantifiable here.

I have taken the approach of valuing the minimum 100% impact as being the budget of the not for profit entity itself in a given year or over the period of the risks being measured. My rationale is that funds are given to the not for profit entity with the expectation that the entity should be producing benefit that is at least equal to their budget.

Any thoughts on how you have handled this in the past?

Thanks,

Dan

]]> By: dwhubbard http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-146 dwhubbard Wed, 21 Apr 2010 15:42:27 +0000 http://blog.hubbardresearch.com/?p=483#comment-146 Thanks for your observations and your work in this area. As you have already seen, much more attention on this subject is needed. Thanks for your input, Doug Hubbard Thanks for your observations and your work in this area. As you have already seen, much more attention on this subject is needed.

Thanks for your input,
Doug Hubbard

]]> By: John Poppelaars http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-144 John Poppelaars Tue, 20 Apr 2010 08:14:00 +0000 http://blog.hubbardresearch.com/?p=483#comment-144 Hi Doug, There is much ground to cover in improving risk assessments and make them more fact based. Many times methods are used that are not fit for purpose or even do harm. Triggered by your book I took a look at the way public risks are being assessed in the Netherlands, for example in case of storage of dangerous gasses or substances. Dutch law prescribes specific methods and parameters to be used in that kind of analysis. These prescribed methods are carved in stone and cannot be adapted to better model the risks to be analysed. You will agree that the value of that risks analysis will be questionable. To try and improve this, I submitted several articles on the flaws of the risk assessment methods used and the consequences it may have to the press, also Dutch television made a news item about it. See my blog (http://john-poppelaars.blogspot.com/) for further details. Things are getting into motion now, but there is still a long way to go to improve on the methods used. John Poppelaars Hi Doug,
There is much ground to cover in improving risk assessments and make them more fact based. Many times methods are used that are not fit for purpose or even do harm. Triggered by your book I took a look at the way public risks are being assessed in the Netherlands, for example in case of storage of dangerous gasses or substances. Dutch law prescribes specific methods and parameters to be used in that kind of analysis. These prescribed methods are carved in stone and cannot be adapted to better model the risks to be analysed. You will agree that the value of that risks analysis will be questionable. To try and improve this, I submitted several articles on the flaws of the risk assessment methods used and the consequences it may have to the press, also Dutch television made a news item about it. See my blog (http://john-poppelaars.blogspot.com/) for further details. Things are getting into motion now, but there is still a long way to go to improve on the methods used.

John Poppelaars

]]> By: dwhubbard http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-109 dwhubbard Fri, 12 Mar 2010 04:13:21 +0000 http://blog.hubbardresearch.com/?p=483#comment-109 Phil, You make several points I should respond to separattely. Here is a three-point response: 1. Does it matter at all whether the analysis method actually *works* or is the only issue is whether it is "structured"? If structure is all that impresses them, and not a measurable improvement in decisions, then astrology should suffice. It doesn't work, but it has structure. 2. If the method doesn't really work then why does it even matter that they listen to it? 3. Why assume calculations are a "waste of time"? The evidence shows that the doing some of the math I mentiion actually does improve forecasts and decisions. The evidence also makes it clear that the risk anslysis based on ordinal scales are the real wast of time. Or are you saying that the that it would take you too long to do meaningful calculations that would provably reduce decision error? I don't really find that to be the case. If the latter, that can be solved. I just spoke to a group of CISO's today about this as well. I think they all left convinced that most of the methods they use are a placebo effect - it makes people feel better but doesn't actually improve anything. Whehter a decision analysis method works really SHOULD matter. Otheriwsie, it really is a waste of time. Thanks for your comment, Doug Hubbard Phil,
You make several points I should respond to separattely. Here is a three-point response:
1. Does it matter at all whether the analysis method actually *works* or is the only issue is whether it is “structured”? If structure is all that impresses them, and not a measurable improvement in decisions, then astrology should suffice. It doesn’t work, but it has structure.
2. If the method doesn’t really work then why does it even matter that they listen to it?
3. Why assume calculations are a “waste of time”? The evidence shows that the doing some of the math I mentiion actually does improve forecasts and decisions. The evidence also makes it clear that the risk anslysis based on ordinal scales are the real wast of time. Or are you saying that the that it would take you too long to do meaningful calculations that would provably reduce decision error? I don’t really find that to be the case. If the latter, that can be solved.

I just spoke to a group of CISO’s today about this as well. I think they all left convinced that most of the methods they use are a placebo effect – it makes people feel better but doesn’t actually improve anything. Whehter a decision analysis method works really SHOULD matter. Otheriwsie, it really is a waste of time.

Thanks for your comment,
Doug Hubbard

]]> By: PhilAgcaoili http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-108 PhilAgcaoili Fri, 12 Mar 2010 03:34:56 +0000 http://blog.hubbardresearch.com/?p=483#comment-108 Great article yet again and so very true. Although I agree with your point, “decisions may be worse than what unaided intuition would yield" http://twitter.com/Hacksec/statuses/10354088764 , I also believe that the business is more likely to listen when we use structure to support our opinion (in my case using a risk analysis). Perhaps from a practical perspective in my field (information security), business decisions are easier made using risk examples so this is quick and dirty method to avoid wasting time on the calculations and provides a good balance. Regards, Phil Agcaoili Great article yet again and so very true.

Although I agree with your point, “decisions may be worse than what unaided intuition would yield” http://twitter.com/Hacksec/statuses/10354088764 , I also believe that the business is more likely to listen when we use structure to support our opinion (in my case using a risk analysis). Perhaps from a practical perspective in my field (information security), business decisions are easier made using risk examples so this is quick and dirty method to avoid wasting time on the calculations and provides a good balance.

Regards,
Phil Agcaoili

]]> By: dwhubbard http://www.hubbardresearch.com/2010/02/483/comment-page-1/#comment-106 dwhubbard Tue, 09 Mar 2010 01:27:57 +0000 http://blog.hubbardresearch.com/?p=483#comment-106 David, Thanks again for your support. I think you will like the new material. The book expanded by more than 20 pages and quite a lot of other pages changed. Doug David,
Thanks again for your support. I think you will like the new material. The book expanded by more than 20 pages and quite a lot of other pages changed.
Doug

]]>