Expert Panel Participants’ Portal
In Sep-Oct 2020, the Hunt Lab is convening an Expert Panel to rapidly form a Collective View on analytic rigour for intelligence. This will be a unique opportunity for leading experts worldwide to gather in an online space, contribute ideas, and engage in informed debate.
The Expert Panel is a key part of our Analytic Rigour project. That project, sponsored by the Australian national intelligence community, will deliver a major report on analytic rigour, covering three topics:
- The nature of analytical rigour
- Factors contributing to (or detracting from) analytic rigour – individual, social, organisational, technological
- Opportunities for intelligence organisations to improve analytic rigour
The Expert Panel process will unfold over three weeks from 28th September to 16th October 2020. It will have three phases. In the first phase, a short survey will draw out panelists’ views on the three topics above. The Hunt Lab team will sort these views into groups which express essentially the same idea, and formulate a single statement capturing that idea. In the second phase, the synthesized statements, and other topics, will be available for discussion by the panel on an online forum. In the final phase, panelists can indicate their levels of agreement with the statements in another short survey. Views receiving clear majority support will form part of the final Collective View.
Participants will be drawn from academia, government, and industry. It will include leading scientists alongside practitioners – people with extensive experience producing and managing intelligence.
Participation is primarily by invitation, but we will also consider anyone with relevant expertise.
Questions and Answers
- When is the Expert Panel process?
The process will unfold over a three week period, from Monday 28th September to Friday 16th October 2020.
- What is the process?
In designing the process, we have had to take into account some challenging constraints: a short time frame, participants distributed across time zones, and COVID-19 conditions precluding travel and face-to-face contact. We have decided to use an online version of the well-known Delphi process. This will proceed in three stages:
Week 1: Panelists will independently generate a small set of statements under each of the three headings listed above, and submit these on a survey form. The Hunt Lab will sort these into affinity groups (sets of statements making essentially the same point). For larger affinity groups, a formulation which seeks to express the common meaning will be drafted.
Week 2: The affinity formulations will be posted as topics on an online forum. Participants will be able to discuss each formulation, or raise other topics.
Week 3: In another survey, panelists will indicate their level of agreement with the (revised) affinity formulations. Those formulations endorsed by a clear majority will be included in the Collective View.
- What would I be doing as a panelist?
You are invited to:
- Join a Zoom session with all panelists at the start of each week
- Respond to a survey in Week 1
- Participate in a discussion forum in Week 2
- Respond to another survey in Week 3
The panel activities will mostly be asynchronous, so you can participate at whatever times you choose. We will be holding a small number of one-hour Zoom sessions, and warmly encourage you to attend those.
- What will the Collective View be like?
The Collective View is an attempt to articulate the collective position of the expert community on analytic rigour. It will consist of sets of statements under each of three topics:
- The Nature of Analytic Rigour
- Factors Impacting Analytic Rigour
- Opportunities to Enhance Analytic Rigour
Statements will only be included in the Consensus View if they are endorsed by a majority of the expert panel. The level of agreement with the statements will be included in the Collective View.
- What is the time/effort commitment?
We understand that participants will vary widely in their ability to contribute. There will be two surveys (one at the start of the first week, and another in the third week) which can each be completed in approximately 20 minutes. Your participation on the online forum can be as minimal or extensive as you choose.
- Is there an honorarium or similar?
Unfortunately we are not in a position to provide payment.
- Who else will be participating?
We are inviting experts from universities, government research organisations, and other government organisations. We expect approximately 50 participants, making up a good selection of both well-known authorities and others with highly relevant knowledge and expertise.
- What is the status of the Collective View?
The Collective View will not have any official status. That said, we anticipate that it will become an important point of reference for future thinking about analytic rigour. We expect academic publications to result from this process at a later time.
- Will my contributions be recognised in the Hunt Lab's final report?
Yes, we plan to list active panel participants as contributors in the final report. However, we will not list any participant without their explicit permission.
- Can I participate without revealing my identity or affiliation?
Yes, your participation can be discreet. For example, your presence in the online forum can be under a pseudonym, and you can choose not to be listed in the final report.
- Can I withdraw my participation?
Yes, you can withdraw at any time. If you choose, we can remove any survey data you have contributed.
- Who will be able to see the report?
A version of the report will be made public, after approval by the agency sponsoring the Rigour Project. Participants in the expert panel will be notified when the report is available. We also expect that the project will generate academic publications.
- I have additional questions
Please feel free to contact us for any further queries.