ALIA home | Online home | search | papers | keynotes | program | sponsors | satellite events
|
This paper explores the interface between technology and an organisation's business, particularly focusing on the strategic decision-making process and the critical role of intelligence in this process. It outlines the intelligence process in detail and explains where the modern librarian 'fits' within this process, and how a variety of technologies and tools - particularly those that enhance collaboration and co-operation - have given the librarian an unprecedented opportunity to be part of an organisation's strategic decision-making process. The aim of the paper is to challenge librarians' perceptions of themselves - to move away from a defensive view of technology (something that either displaces or replaces them) to a more proactive view - where grasping emerging technologies can lead them to their rightful place: sitting alongside the strategic decision makers within the organisation.
This presentation will address the interface between technology, business processes, strategic decision-making and intelligence. It will focus on the role of the modern day information professional and bring together some strands in competitive intelligence, knowledge management, content management and how they apply to the intelligence process. The views presented here are those from an unashamed technologist; nevertheless it also draws on a strong belief that it is business that drives technology, not vice-versa.
The paper will focus on the role of the information professional - based on observations during the author's experience working in the Australian Intelligence community and subsequently in law enforcement intelligence. It will attempt to draw strands to some of the challenges presented given 11 September and the need to lock information professionals closer into the intelligence cycle using collaborative frameworks and help break down closed information silos.
The paper aims to move away from a defensive view of technology (i.e., as something that either replaces or displaces information professionals) to a more proactive view - presenting a challenge to information professionals to grasp these technologies, know their rightful place and sit alongside the strategic decision-makers within their organisations.
Much of the author's understanding of intelligence has been informed by the author's experience working in the national security domain as an economic analyst for the Office of National Assessments (ONA). This agency advises the Australian Prime Minister and Cabinet on geopolitical, defence-related and economic issues that have national security implications for Australia.
Unlike the CIA, ONA is composed primarily of analysts brought from the bureaucracy, from academia and also from the private sector and law enforcement area but has no collection capabilities as such. It sits on the top of a pyramid of collection efforts from organisations such as the Defence Signals Directorate (DSD), the Australian Secret Intelligence Service (ASIS), and also shares intelligence with counterparts in the UKUSA arrangements.
In 1996, the author moved out of the intelligence community and moved into the commercial domain after having attended a conference on Open Sources Intelligence (OSINT) in Canberra. The author also has spent a lot of time becoming familiar with competitive intelligence (joining the Society for Competitive Intelligence Professionals - SCIP). The author now finds himself working in the law enforcement domain where he has been managing a $7.3 million initiative that aims to bring together the Australian Customs Service (ACS), the National Crime Authority (NCA), and the Australian Federal Police (AFP) in terms of sharing intelligence related to illicit drugs.
The underlying challenges faced in each of these domains in which the author has worked remain the same. It is basically a knowledge management (KM) challenge that is familiar to most people working in the information profession - bringing the right information to the right people at the right time, and helping to break down information silos and make information more transparent to the organisation.
The author believes that this knowledge management challenge is also very much an intelligence challenge, particularly after the events of 11 September 2001. This event revealed a need for better intelligence sharing between the law enforcement and intelligence communities, a requirement to break down information silos and a knowledge-is-power mentality. It also presented a compelling case to collaborate within and across all the organisations involved (law enforcement and national security, State versus Federal domains, customs, border control, etc). The author is fascinated by what is happening in the United States at the moment with the creation of the Department of Homeland Security in an attempt to overcome these problems. He believes that other countries and other organisations will take up much of what is happening with homeland security in the US. Robert Steele's work on bringing together these organisations within an open source context goes some way to addressing this, but while considered necessary is still not sufficient.
Intelligence is driven by a very simple process, which is often called the 'intelligence cycle'. Essentially, this process is requirements driven, that is, it is highly targeted in terms of its collection processes, analysis processes and reporting processes. Intelligence isn't something that simply happens out there in the field. To be effective, it needs to be driven by strategy, longer-term requirements that separate it from simple tactical and operational intelligence activities. From the author's experience, the national security intelligence capabilities tend to be highly strategic in nature, whereas law enforcement (policing, border control, enforcement, customs, immigration, etc) tends to be tactical in nature. What is the difference between these two types of intelligence?
For intelligence to be strategic, it must be both consistent and enduring, but also sufficiently flexible to accommodate changing requirements. This is often a difficult balancing act. For example, prior to Tampa last year, intelligence on people smuggling operations underway in the Indonesian archipelago may have been less important than other issues (the war in Afghanistan for instance). The Bali bombing is an event that automatically shifts collection and analysis requirements. However, the intelligence system can swiftly adapt to changing requirements - indeed, this is an indication of its maturity, stability and effectiveness.
The intelligence cycle looks as if it flows in one direction only. In fact, to be effective, the relationship between collection, analysis and reporting needs to be highly iterative and interactive. There has to be a large degree of information exchange between all the actors across the intelligence cycle to make it work properly. As we will see shortly, there is a good degree of quality control in the whole process as information is questioned, validated, cross-referenced and checked again.
Within a national security context, the key client for the whole intelligence process - and the reason why intelligence is collected in the first place, is the decision-maker (in this case the Prime Minister and Cabinet). Within a business context, some would say that the key consumer of competitive intelligence is the CEO. However, note that intelligence never seeks to make the decision or create policy; it only ever seeks to inform the decision-maker; to make it easier for the decision-maker to have an appropriate context in which to make the decision. This is just as true for competitive intelligence as it is for national security-related intelligence.
The following diagram is taken from Leonard Fuld's website, http://www.fuld.com. Fuld is one of the pioneers in competitive intelligence. While there are some subtle differences from the proceeding diagram, the following diagram would still be instantly recognisable by anyone working within the national security intelligence community.
Just as with national security, the primary reference point for any intelligence officer is the decision maker. Also, there is a strong emphasis on quality control, accountability, auditability and validation of assumption made and conclusions reached.
At the end of the day, analysis is also about taking a gamble: an analyst will never have enough information to make a call on how events are likely to transpire but they need to be in an organisation where they are allowed to make a call and where action from that call can follow. The analyst must be given enough freedom to take a risk while at the same time having to comply with a sufficiently rigorous process of quality control. The trick is to make sure that the quality control doesn't inhibit the analyst too much in terms of making the call and allows the analyst to bring the intelligence to the decision maker's attention as quickly as possible. This is not always an easy balancing act. In the author's view, the failure to achieve this balance was one of the primary reasons why intelligence let the United States down in the lead-up to 11 September.
Therefore, there is a certain skill required by the analyst in knowing when the threshold has been reached in making a call and then being able to defend this judgment as it goes through a robust validation process. The validation process will expose the analyst's reasoning, his or her evidence and their conclusions at each step of the intelligence cycle. Getting the balance right is never easy.
As a gatekeeper or facilitator to a large array of publicly available information, whether online or available through traditional media, the information professional is a key part of the intelligence cycle.
Information professionals have access to, and a detailed knowledge of the nature, characteristics, format, size, availability and cost of these sources. This is something that an analyst can only ever have the barest of knowledge about, no matter how expert they are in their particular subject area.
An analyst is very rarely trained in information science, and generally knows little about metadata management (indexing, cataloguing, abstracting, etc). When given a choice, they will always want access to more and more information, even if they might really need less information, but information that is more accurate.
So where does the information professional fit in to the intelligence cycle? In the author's view, and on the basis of the author's experience working in intelligence, the author would say that first of all they are vital in helping the analyst with collection:
Secondly, there is no reason why the information professional should stop at just being a collector. An information professional should also endeavour to:
Lastly, information professionals should spend time predicting, anticipating and helping set some of the collection requirements. They should move away from a silo-like understanding of their roles and responsibilities and should try to collaborate right across the intelligence cycle and participate in as many of the intelligence activities in the intelligence cycle that they can.
Any analyst who has been lucky enough to work alongside a skilled information professional knows their true value. Unfortunately, in a world of diminishing resources, outsourcing, downsizing and multi-skilling, the information professional has often been the first victim of budget cuts. Managers of intelligence organisations, just like their private sector counterparts have often cut their libraries and librarians first. Even worse, sometimes the information professionals within these organisations have made it easy for this to happen - by not understanding their role in the intelligence cycle, by sticking to outmoded concepts of their roles and by not marketing their services as well as they should have.
Information professionals now have access to a wide variety of information management tools that can help them become even more indispensable to intelligence analysts. These tools will never replace or displace the information professional, but need to become an essential part of the tools they use.
This section will address, at a high level, some of the tools that are now available to information professionals as they become further engaged in the intelligence process. It will focus in particular on a set of tools that more broadly fall under the category of resource discovery tools (including search engines, data mining tools etc). The author has a particular interest in intelligent agent applications, which have driven some of the knowledge management systems and web content management applications available today. These can be subdivided into a further three subdivisions:
A good overview of CI-specific IT tools is available from Leonard Fuld's website that was mentioned previously: http://www.fuld.com. Rather than addressing the full list of products currently available, the reader is asked to consider downloading the Intelligence Software Report: A Global Evolution from Fuld's website. It presents a comprehensive overview and evaluates a wide range of products against their suitability across each component of the intelligence cycle.
The author spoke at the Society of Competitive Intelligence Professionals (SCIP) Conference in Atlanta in 2000. In the exhibition hall for that conference, there were perhaps only four to five vendors of competitive intelligence-specific IT applications. Since then, the number of specialised vendors of products designed to support competitive intelligence capabilities has grown signficantly. Fuld's list now contains over 13 separate products designed specifically for CI purposes.
So what characterises this kind of software?
Two examples of this kind of software are presented in the following digram:
Knowledge.Works by Cipher Systems (also known as G2 in its earlier Lotus version). This product was designed to leverage the power of GroupWare (either in the form of MicroSoft Exchange Server or Lotus Notes). It contains the following features:
The other product referred to in the above diagram is WinCite, which also use a Key Intelligence Topic framework to help guide the analyst through to making a judgment and furnish actionable intelligence. The people who developed Cipher's products are ex-CIA and ex-NSA whereas WinCite shows it pedigree as a marketing intelligence tool.
There are lot more players in the KM application market than there is in the specialized CI applications market. A good overview of the KM application market can be obtained from a number of sites, including BRINT (www.brint.org), KM Tools (www.kmtools.com) and KM World (www.kmworld.com). There are plenty of other sites as well on the Internet where you can get comprehensive lists and evaluations of these technologies.
KM solutions typically get sold under a variety of names: enterprise information portals, web content management systems, portals, vortals, enterprise resource planning and enterprise resource management, content management systems, and so on.
So what is common to each of these applications? Typically they will contain all or some or all of the following:
The two examples of KM solutions referred to in the following diagram come from the Computer Sciences Corporation (CSC), where the author worked in CSC's Knowledge Management National Practice, and from Verity - one of the leading product vendors in the KM solutions space.
The CSC portal uses Plumtree and includes a variety of capabilities, including over 300 online communities ranging across all aspects of CSC's business; 'gadgets', which are integrated components linking the Plumtree portal to other legacy applications (e.g., e-mail, corporate directories, Notes databases, etc). It also uses a formal control process, which is a very tight workflow, a network of community owners and approvers and guidelines to contributing to repositories, etc.
Verity instead, is less a portal than a very powerful text retrieval search engine, with a number of value-added capabilities including the following:
The next set of tools is typically being used by analysts within the law enforcement area, but is also being used by agencies where tactical/operational intelligence and case management workloads form the bulk of their work.
One such product is InterQuest from an Australian company called The Distillery. The Distillery has accounts with Australian Customs (ACS), the Australian Secret Intelligence Organisation (ASIO), the Department of Immigration Multicultural and Indigenous Affairs (DIMIA) and the Commonwealth Ombudsman. Basically, InterQuest is an Oracle database with some clever applications overlaying it. These include:
Within a structured database, the entities or metadata structure is very important. IT architects need to spend a lot of time setting up the database structure with a careful definition of all data entities. A sophisticated workflow often needs to be built into the system for quality assurance, and lockdown of specific information sets - in order to be able to create automated intelligence reports.
Despite these capabilities, it should be remembered that it takes human beings to determine the relationship between entities in a database; no software can do this alone. While rules can be built into database applications, no software can do the analysis for you. At its basic level, InterQuest remains a relational database - and remember the golden rule for database - GIGO = garbage in equals garbage out.
This part of the paper addresses a collection of technologies where the rubber really hits the road with the application of technology and the intelligence process - collaboration tools.
Contrary to some expectations, it is not really necessary to invest in expensive technology to set up an effective and fully functioning intelligence capability. Intelligence stands and falls on human capabilities, not simply on the technologies used to search, retrieve, mine or manipulate data. At the end of the day, it is the human ability to place information into context and an understanding of the repercussions of that information that drives intelligence. Context comes from collaboration.
Some collaboration applications are free or relatively cheap (US$100) to download off the Internet. One of the author's favourite tools is Groove (www.groove.com), a peer-to-peer product that has some of the following functionality:
The following screenshot shows how shared repositories can be set up using Groove.
There are many other collaboration tools on the market - a reflection of how important these products have become for knowledge management. Some of the earlier pioneers in this field included companies such as Orbital and Lotus. Other companies, such as Communispace, Intraspect, Engenia Unity, LiveLink (Open Text), Infoworkspace, Eroom Technologies, Virtualteams, and Skydesk Inc, to just name a few, are also active in the market.
Etienne Wenger also provides a good map of where these products fit in terms of their main functionality. These are listed in the following diagram:
So why does the author consider these collaboration tools as the 'killer applications' in intelligence? Simply because they can bring together, virtually, all of the players that have been mentioned before in terms of the intelligence cycle and the intelligence process into the same shared, transparent, dynamic and highly interactive space. This also includes the information professional.
Let's now move on to some examples of how these tools have been used to support competitive intelligence activities in particular, bringing together the collectors, analysts, clients and decision-makers into a tight and transparent system.
In all of these cases, we used collaboration tools to lock in our information professionals as a key part of the competitive intelligence process, to have the customer of the intelligence product engaged, and to keep all analysts and interested parties remain abreast of each others' activities.
The experiences mentioned above have shown the author that the information professional has a key and enduring role in the intelligence process:
Some words of advice to the prospective information professional as they were about to engage in an intelligence effort, would be the following:
Yet we are still in very early days. Other than the examples above, the author knows of very few other examples of where information professionals are such a fundamental stakeholder in the intelligence process and where collaboration tools are being used to underpin intelligence processes in organisations.
Within a national intelligence framework, this may now change greatly with the impact of events in the United States on 11 September.
Robert Steele, from Open Source Solutions has just written a book on the underlying causes for the failure of US intelligence to put the pieces of the Al Qaeda operations in the US together prior to the tragic attacks on New York and Washington. Basically the US agencies concerned failed to share intelligence, curtailed collaboration as a result of turf wars and operated in operationally- and strategically-defined silos. Other factors, such as questionable management decisions, lack of foreign-language analysts, and a failure to detect and infiltrate terrorist operations contributed to this. When taken together, all of the intelligence pieces already held by the US Intelligence Community, if put together in time, might have prevented 9-11. But, as always, that's very easy to say post factum.
Amongst the 26 or so measures Steele advocates to move towards a 'new craft of intelligence' is collaboration. He writes:
'The bureaucratic office with analysts physically co-located with one another must give way to virtual task forces comprised of the top individuals important (in terms of the analyst's day-to-day work) than their parent organisation's reputation. Personal 'brand names' and informal peer-to-peer networks will be subsidised and nurtured by organisations that understand that only a vibrant self-directed network with global reach will attract the bulk of the relevant information - analysts and nodes will be magnet for relevant information from private sector peers.'
The author read that as an invitation for the information professional to stake their claim for a role in this new virtual intelligence task force. Collaboration, in addition to a greater reliance on open source - your domain as information professionals - must become a an even more important plank within the new intelligence environment.
And there is no reason why the private sector should also not adopt exactly the same intelligence processes and procedures, where silos, territoriality, and poor management can produce as equally counter-productive, if not as devastating a result for the organisations concerned.
In conclusion it should be remembered that technology is an enabler, not an enemy to the information professional. Collaboration tools are relatively new on the scene when we consider that the other elements of knowledge management (search engines, directories, agents, etc) have been around for a decade or so now). It is the author's firm belief that, if you take the time to experiment with them, and build them around your existing processes, these tools will help change your role, your status, and who knows, even your budget, in the organisations for whom you work.