Volcano icon

Community-Based
Fire Preparedness
Programmes

The Australasian Journal of Disaster
and Trauma Studies
ISSN:  1174-4707
Volume : 1999-1


Community-Based Fire Preparedness Programmes:
An Empirical Evaluation


Dr Bernd Rohrmann, Dept. of Psychology, Univ. of Melbourne, Parkville, Victoria 3052, Australia. Phone: +61 3 93446349 Fax: 93476618
WebSite: http://www.psych.unimelb.edu.au/staff/rohrmann.html
Email: rohrmann@unimelb.edu.au
Keywords: fire, disaster preparedness, risk communication, evaluation

Dr Bernd Rohrmann

Dept. of Psychology, Univ. of Melbourne,
Parkville, Victoria 3052,
Australia


Abstract

In countries such as Australia, fires are a constant threat to communities, with the potential for grave disasters to occur. Wherever people are exposed to the risk of fires, hazard prevention becomes a very important task. Risk information/communication/ education about fire hazards in the workplace or in homes, and about bushfires near residential settings is an indispensable part of that. Yet are the efforts of fire authorities successful? Do they enhance community safety? And what are the preconditions of effective programs? To answer such questions, evaluation research is required.

In this article, firstly methodological considerations are elaborated, dealing with evaluation approaches, assessment criteria and study designs. Secondly, a theoretical model of the fire risk communication process is presented, focussing on the socio-psychological factors which influence the outcomes of a campaign. Thirdly, empirical data from an evaluation study conducted in Melbourne, Australia will be reported. In this project, participants in a novel program of the Victorian CFA, "Community Fireguard", were surveyed, based on a Pre/Post design. For comparisons, a control group not exposed to the campaign was included. A special sample dealt with people who had recently experienced a major fire. In addition, the perspectives and experiences of fire officers were investigated. The results available so far are very encouraging for CFA's new approach to bushfire preparedness of residents.

The findings also indicate for which issues further in-depth research is needed. Finally, some conclusions for the design of risk communication campaigns are discussed.


Community-Based Fire Preparedness Programmes:
An Empirical Evaluation


Approaches to Improving Fire Preparedness

Preparedness for the hazard of fires - that is, house fires, fires in industrial or transport facilities, and wildfires - is obviously very important, given that not only the belongings but the very lives of humans are at risk. In Australia, bushfires in particular are a bigger threat than anywhere else in the world (Pyne 1991). In fire disasters, hundreds of lives, large numbers of stock and assets worth billions of dollars have been lost. As recent events (e.g., the 1994 fires in N.S.W. and QLD, or the fires in Melbourne and Victoria in 1997 and 1998) have shown, the risk of disasters is still very present - in spite of major advances in bushfire control.

Consequently, hazard management becomes a crucial task. Public authorities are responsible for fire prevention, fire risk information/communication/education and fire emergency management. These tasks need to be addressed not only from a technical but also organizational and socio-psychological perspective. People who might be exposed to fires need to be optimally informed about the hazard characteristics, preventive measures and appropriate behaviors during the onset of a fire event, and they must understand their own responsibilities. Authorities have to compose pertinent emergency planning, prepare coping strategies, and communicate the relevant information effectively to residents and communities as a whole (Barham 1996, Chase 1993 Robertson 1989). Such tasks go far beyond the 'classic' firefighting missions of fire authorities. In fact, a shift in the general orientation of fire risk management seems obvious (see, e.g., Smith et al. 1996, Rhodes & Reinholtd 1998). In recent years the Country Fire Authority of Victoria has introduced a novel approach to fire safety, the "Community Fireguard" program, which is based on community involvement and aims at enhancing individual responsibility for fire safety and survival strategies (cf. e.g., CFA 1995, Beckingsale 1994, Beckingsale & Petris 1994, Jones 1987, Whelan 1987).

Whatever fire authorities decide to do, inevitably the question arises whether their efforts are successful. Do they enhance community safety? Is fire preparedness of people at risk improved? Do the strategies employed 'work' with respect to the target group, and what are the preconditions of effective programs? To answer such questions, empirical evaluation research is indispensable (Gaull 1997, Rohrmann 1992, 1998). Given the aim - ultimately, protecting human life - critical result assessments must be part of fire preparedness programs.


Risk Communication: Means, Results, Efficacy

The risk communication process

Within hazard management, providing and utilizing pertinent information in an interactive manner is a core process. In this context, risk communication research is a most relevant area of expertise. This young but rapidly growing field comprises hazard perception, risk information strategies and interactive problem-solving approaches (cf., e.g., Covello et al. 1989, Fischhoff et al. 1993, Handmer et al. 1991, Kasperson & Stallen 1990, Krimsky & Plough 1988, Lundgren 1994, Rohrmann 1995 and the bibliography in Rohrmann et al. 1991). In Table 1, an overview of main risk communication aims as well as typical means and 'channels' is provided.

Table 1: Tasks and Means of the Risk Communication Process

Primary types of risk communication aims
  • Advancing/changing knowledge and attitudes regarding hazards & risk-taking
  • Modifying risk-related behavior of people exposed to hazards
  • Promoting community participation in hazard mitigation
  • Facilitating cooperation & joint conflict resolution regarding controversial risks
  • Developing disaster preparedness and emergency management
Communication means & channels
  • Print material (e.g., fliers & brochures), distributed by institutions/agencies
  • Product information, machine operating instructions, etc
  • Public information services, 'hot lines', etc
  • Educational video/film/computer products
  • Info presented via broadcasting, television, newspapers, journals & the internet
  • Expert presentations (at meetings, trainings, drills etc)
  • Warning sirens (or messages through mobile loudspeakers)

A large body of knowledge has been produced with respect to quite different hazards, both technology-induced (e.g., risks from industrial facilities) and natural ones (e.g., earthquakes), as well as health-risking behavior (e.g. smoking) and diseases (e.g., AIDS). Consequently, the psychology of risk informedness is now much better understood (cf., e.g., Atman et al. 1994, Fischhoff et al 1993, Rohrmann 1998). It seems however that psychological research on risk communication and the social-science literature on fire emergency preparedness have not been systematically related in Australia.

Evaluation research: criteria and designs

All risk communication is conducted with respect to relevant objectives, and the value of the principal aim is hardly in question. Having "good" aims doesn't guarantee success though (even though authorities sometimes seem to assume so…) Rather, systematic empirical investigations are required in order to prove the effectiveness of a campaign - simple experience is not sufficient. "Evaluation" means the scientific assessment of the content, process and effects (consequences, outcomes, impacts) of an intervention (measure, strategy, program) and their assessment according to defined criteria (goals, objectives) (Fink 1993, Patton 1986, Rossi & Freeman 1993; see Rohrmann 1992 with respect to risk communication).

Once the objectives of a particular risk communication have been stated by those responsible for the program, an evaluator needs to operationalize them and investigate whether or not they have been achieved. "Effectiveness" as the overall criterion has to be explicated by characteristics of the content, process and outcomes of the risk communication program employed. A list of pertinent criteria is presented in Table 2.

Table 2: Criteria for Risk Communication Effectiveness

Evaluation aspects (category and examples) Info source

Goal-related Criteria

 Content evaluation      

Correctness and completeness of the information

A
E
 

Comprehensibility of the message

 
E
R

Ethical considerations

 
E
 
Process evaluation      

Difficulties/failures in running the program

A
   

Inclusion of relevant actors/societal groups

A
E
 

Feedback and confirmation possibilities

 
E
R
Outcome evaluation      

Degree of information distribution

 
E
R

Increased/improved problem awareness and knowledge

   
R

Reduction of accident rates or mortality rates

A
E
 
Procedural criteria

Financial efficiency (material costs, personnel)

A
   

Training needs for the personnel involved

A
E
 

Flexibility & adaptability

 
E
 

Note: "A" stands for RC agency (or author of the RC program),
"E" for risk expert or RC expert (independent researchers),
"R" for information receiver or participant of the RC program

The first set of criteria refers to the question whether the content of the message and its presentation is valid for the communication goals; the second set of criteria is related to the process of conducting risk communication programs; the third set deals with the actual outcomes of campaigns; in each case, three examples are given (full list in Rohrmann 1992). In addition to substantial (goal-related) criteria, organizational and procedural aspects are to be included in the evaluation of risk communication programs (cf. bottom Table 3). Such 'secondary' (economic) criteria are vital for any agency, and data on costs are needed for cost/benefit or cost/effectiveness analyses.

Evaluation studies differ considerably in their approach; the main options for a researcher are listed in Table 3. In each case advanced research designs are required. Studies lacking "before" measurements and control groups of not-exposed people are unlikely to yield valid results. Evaluations may be designed 'in-house' or (preferably) conducted by external researchers. The critical "valuation" of the obtained data is the crucial step in an evaluation study. The stated (normative) goal is the principal, but not the only reference for assessing evaluation results. Comparisons with alternative interventions (other hazard information/communication/ education strategies) are important as well.

Table 3: Evaluation of Risk Communication: Basic Considerations

Study design: longitudinal before/after study <2+ points in time>
control group (not exposed to the intervention)
Focus: content-oriented (substantive correctness) and/or
process-oriented (formative/developmental view) a/o
outcome-oriented (summative effectiveness)
Information sources: risk information/communication targets (receivers)
sender/author/agency
Reference for comparisons: normative program goals (as stated by institution)
previous situation
alternative information/communication strategies

(Source: after Rohrmann 1992)

Evaluation results can demonstrate not only whether but also why a program works (or success is lacking) and thus guide the further development and improvement of on-going risk communication.

Evaluating bushfire preparedness programs

So far the purpose and methodology of risk communication evaluation in general has been discussed. However, each concrete evaluation project must be shaped according to the specific nature of the program to be assessed, the characteristics of the people targeted, and the objectives to be achieved.

In the following Table 4, four types of variables are explicated which need to be looked at in order to understand the impacts of educational fire preparedness programs in local communities.

Starting from the bottom of Table 4, the ultimate criteria are of course whether understanding of the problem and degree of preparedness have actually improved (cf. "D" variables). The "C" variables reflect the participants’ appraisal of program components. These judgments are important (particularly for efforts to improve programs) but not a sufficient indicator of program success in itself. The "B" variables relate to the involvement of residents, a crucial precondition for risk communication processes to work. The "A" variables are important in order to assess whether the provided content is in line with the participants expectations and "mindsets". All these variables should be analyzed in relation to each other.

Table 4: Criteria for Assessing Educational Fire Preparedness Programmes

A1 Perceived fire risk level & vulnerability <people & property>
A2 Subjective information needs <employees, residents>
A3 Trust in information-providing sources (e.g., authorities)
A4 Expectations about responsibility for hazard prevention
A5 Reasons reducing/preventing participation in group activities
A6 Comprehension of provided (verbal/visual/audiovisual) material
   
B1 Level of commitment to actively improving hazard preparedness
B2 Actual involvement in preparedness & prevention activities
B3 Involvement pattern for household/office/team members
B4 Clarification of personal responsibilities
   
C1 Appraisal of brochures or videos on preparedness/prevention issues
C2 Appraisal of discussions/group activities/practical exercises
C3 Appraisal of involved personnel (e.g., instructors, facilitators)
C4 Participants' satisfaction with outcomes of program as a whole
   
D1 Level of understanding fire issues (including knowledge check)
D2 Extent of preparedness (technical, organizational, psychological)

 


Fire Risk Communication: Conceptual Framework

In order to analyze the process and outcome of fire hazard information and education efforts, a comprehensive framework is indispensable. Concepts and findings from five relevant research fields were integrated to develop a theoretical model of fire risk communication and its impact on community preparedness: risk communication research (cf. e.g. Covello et al. 1989, Jungermann et al. 1991); the social psychology of attitude change (e.g., Ajzen 1993, Eagley & Chaiken 1993); studies of human behavior in fire (e.g. Canter 1985, 1990, Krusel & Petris 1992); disaster impact management (e.g. Drabek 1990, Raphael 1986), social science theory and research on participation processes (e.g. Munro-Clark 1992, Webler & Renn 1995); for a review see Rohrmann 1995. A graphic presentation is shown in Figure 1 below.

Fifteen constructs (or variable sets) are identified <A to O> as relevant aspects of the fire risk communication process, and the assumed causal links are indicated on a global level.

Figure 1

Figure 1: Fire Risk Communication: Process Framework

The model expresses that the final outcome variable, risk-reducing behavior <D> regarding a hazard <A>, is determined not just by the communicated messages of the information/education program <E> but is the result of a complex evaluation process <B, C and G, H, I>, and influenced by personal characteristics <K, L, M> and manifold context factors, e.g., attributes of the information source and 'channel' features <F> as well as family/friends and the community one belongs to <I>. Also, a longitudinal perspective is implied, as any risk communication has to deal with pre-existing attitudes and behaviors <N, O>.

This conceptual framework guided the specification of evaluative criteria, the explication of substantive hypotheses and the design of measurement instruments for empirical investigations of risk communication impacts.


An Evaluation Study on a "Fireguard" Program

Research objectives

In the following, an empirical evaluation study of a community preparedness program is reported. The project "Evaluation of community-based approaches to bushfire preparedness" (Project EBP) was begun in 1996; it focuses on the "Community Fireguard" (C/F) program of the Country Fire Authority of Victoria (CFA). The principal objectives are: A critical analysis of the content and design of bushfire preparedness programs, based on a socio-psychological perspective; surveying the reception, understanding, acceptance and implementation of measures by both residents and officials from authorities; and a comprehensive assessment of program outcomes on which enhancements of Australian fire safety initiatives can be based. The main research questions for the empirical stages are summarized in Table 5.

Table 5: Project EBP: Main Research Questions

  • What are the genuine information needs of residents in fire-prone areas (as they see them)?
  • What knowledge do people have re bushfire preparedness?
  • What expectations about fire authorities (vs personal responsibilities) exist?
  • To what degree do residents search for, study, comprehend, accept, keep in mind and utilize fire hazard information?
  • How are information materials, activities and involved CFA staff in C/F groups evaluated by participants?
  • What changes in emergency preparedness and protective risk reduction result from participating in C/F?
  • How do CFA officers involved in C/F activities assess the utility of C/F?
  • Are the impacts of community-based bushfire management strategies stable over time

The project is conducted in collaboration with the CFA and will be completed in early 1999.

Project design

The research plan was comprised of theoretical work and both quantitative and qualitative empirical studies, i.e.: surveys with residents exposed to CFA activities and various comparison samples, and interviews and focus groups with CFA personnel and risk research experts. The central part is a "before/after" sub-study with participants in the "Community Fireguard" (C/F) program. In Table 6 the design and sampling for the sub-studies of the project are outlined.

Table 6: Survey Design: Target Groups & Samples

<E> Residents participating in existing C/F program N=110
<X> Bushfire-prone areas; no C/F; future CFG's likely N=126
  out of <X> - interviewed again in Phase II:
<N> Residents participating in new C/F group N=21
<C> comparison group not exposed to C/F, same areas N=36
<F> Residents exposed to fires in 1997 N=30
<P> CFA personnel: officers dealing with the C/F program N=20

Sub-study <E> was designed to investigate the experiences of residents who had actively participated in a local C/F group for 1 to 2 years. The sampling included groups from all facilitators (in 4 regions); within locations, participants were randomly selected. Group <X> ('non-C/F' residents), selected in the same areas, served as a 'base sample' for a group of new C/F participants <"N"> and a comparison group <"C"> without any involvement in the C/F program; these two groups were interviewed twice, before and about 6 months after the C/F program was implemented in the respective area (data N1/N2; C1/C2). This quasi-experimental design allows for intra-individual and inter-individual outcome analyses (e.g., by comparisons N1 vs N2 and N2 vs C2; further possible comparisons are E vs X and F).

The interviews were done face-to-face by a trained team of interviewers and based on a standardized questionnaire (including lists and response scales), constructed with respect to the conceptual framework outlined above.

The list in Table 7 shows the 20 principal topics covered in the 6 questionnaires developed for the substudies of this project. They also apply to the two studies extending beyond an evaluation of the C/F program, namely the survey <P> with selected personnel (including both C/F facilitators and fire brigade officers) and the special study <F> with people who had been exposed to actual bushfires within the last year.

Residents' appraisal of the program

Because of the large amount of data (and the fact that the project is still running), only selected results can be presented here. This section will focus on findings on the C/F program based on the residents' views.

Table 7: List of topics covered in the questionnaires

A Perceived risk level and vulnerability (for area and house)
B Level of understanding of bushfire issues
C Sources of existing knowledge and competence
D Personal experiences with actual bushfires
E Subjective information needs of residents
F Assessment of information sources (authorities)
G Expectations about responsibility (re bushfire hazard)
H Level of preparedness for bushfires
I Motivation for participating in a C/F group
J Actual participation in C/F activities (individual & group)
K Involvement pattern for household members
L Evaluation of discussions/group activities/exercises
M Evaluation of printed materials, videos etc
N Evaluation of involved CFA staff (facilitators, officers)
O Action: taken/intended/induced by C/F (technical/organizational)
P Satisfaction with process and outcomes of C/F program
Q Intentions for future participation in C/F
X Personal attitudes (inc.: life style, social participation)
Y General/other community involvement
Z Demographics

Note: Each topic was explicated by several questions/scales.

Firstly, the demographic data for the main samples in Table 8 show that most respondents have been living in their residential area for 10 to 20 years already; in comparison to the general population, higher ages, educational levels and occupational status are over-represented in the groups targeted in this project.

Table 8: Demographic description of sub-samples

#
Topic/Variable
Sub-Sample
  E X N C F
Z4
Age 51.0 45.0 46.4 46.1 47.0
Z11
Sex (% Male) 51 50 56 44 28
Z2
Length of residence in area 1982 1979 1981 1981 1978
Z1
Length of residence in house 1985 1985 1984 1986 1982
Z8
Education (%)        
    Primary/Secondary 62 45 38 44 47
    Tertiary 38 55 62 56 53
Z9
Work Description (%)        
    Running household 10 11 06 08 09
    Retired 24 08 00 12 16
    Predominantly White-collar 45 60 75 68 63
    Predominantly Blue-collar 14 13 18 12 03

 

The next Table 9 presents selected results for the sub-studies <E> and <X>. (Please note that these samples are not designed to be directly comparable, as they served different research purposes; cf. Table 6).

Table 9: Residents' Views on Bushfire Issues - Groups "E", "X", "F"

   
Sub-Study E
Sub-Study X
Sub-Study F
#
Topic/Variable
Mean
s[M]
Mean
s[M]
Mean
s[M]
A1 Rating of area's bushfire risk (0..100)
80.0
2.0
70.0
2.1
--
 
A2 Likelihood of bushfire threat in next 5 years (1..5)
3.7
0.8
3.2
0.1
3.1
0.2
B1 Understanding bushfire issues (self-rating 0..100)
74.0
1.3
61.0
1.5
65
3.0
B4& Knowledge index re bushfire issues (0..24)
15.5
0.3
13.5
0.3
12.4
0.5
C5 Brochures/pamphlets re bushfires read (%)
95
 
67
 
--
 
C6& Extent of informedness (%)
41
 
21
 
75
 
D1 Personally experienced a bushfire (%)
53
 
42
 
100
 
E1 Need for information re bushfire issues (%)
44
 
49
 
47
 
F1 Best source for info needed is CFG (%)
32
 
0
 
0
 
G2& Responsibility attitude regarding the bushfire risk
 

Accepted as own responsibility (2..10)

8.3
0.2
7.7
0.1
6.9
0.4
 

Seen as CFA's responsibility (3..15)

7.6
0.3
9.0
0.3
8.8
0.5
H1 Overall bushfire preparedness (self-rating) (0..100)
73.0
1.5
61.0
1.9
80
2.0
H2 Specific appraisals of preparedness
 

Technically (1..5)

3.8
0.1
3.2
0.1
4.2
0.1
 

Organisationally (1..5)

3.8
0.2
2.8
0.1
4.1
0.1
 

Psychologically (1..5)

3.8
0.1
3.5
0.10
4.0
0.1
O1& Overall extent of actions taken (0..16)
11.3
0.3
7.1
0.2
7.3
0.4
   
J6 Frequency of joining CFG activities (examples)
 

Attended meeting to discuss fire behavior issues (%)

97
 
--
 
--
 
 

Joined street walk in order to inspect properties (%)

48
 
--
 
--
 
 

Attended a working bee or a burn-off (%)

5
 
--
 
--
 
L1& Mean evaluation of CFG activities (10..50)
42.4
0.8
--
 
--
 
M1& Mean evaluation of materials used in CFG (10..50)
45.0
1.2
--
 
--
 
N1& Mean evaluation of CFA staff involved in CFG (10..50)
44.5
0.8
--
 
--
 
P1 Overall improvement in understanding bushfires (1..5)
3.9
0.1
--
 
--
 
P2 Level of dependence on CFA re bushfire threat (1..5)
3.7
0.1
--
 
--
 
P3 Overall assessment of participating in CFG (1..5)
4.2
0.1
--
 
--
 
P6 Improvement in dealing with b.f. issues as group (1..5)
3.6
0.1
--
 
--
 
P7 Likelihood that CFG continues without CFA (1..5)
3.4
0.1
--
 
--
 
Q1 Likelihood of own future involvement in a CFG (1..5)
4.3
0.1
--
 
--
 

 

The main findings can be summarized as follows:

The lower part of Table 8 comprises the participants' evaluations of the C/F program and its outcomes:

Of course these results can be influenced by socio-psychological expectations - evaluations of a principally valuable activity tend to be biased towards a favorable outcome appraisal. Therefore comparisons based on the "N" and "C" groups of respondents provide a more valid examination; see Table 10.

These results demonstrate:

The lower part of Table 10 contains the "N" group's appraisals of C/F activities and their respective outcomes (topics L1 to P3); these evaluations are predominantly positive and almost as approving as those given by the "E" group (cf. Table 9 above), in spite of the shorter duration of C/F participation.

Perceived shortcomings of the C/F program (as expressed in responses to open-ended questions) seem to be limited to mainly two issues: high demands on time, and difficulties in applying advice to the individual problem case.

Table 10: Residents' Views on Bushfire Issues - Comparison Phase 1--2

 
New C/F groups
Comparison group
N1
N2
C1
C2
#
Topic/Variable
Mean
s[M]
Mean
s[M]
Mean
s[M]
Mean
s[M]
   
A1 Rating of area's bushfire risk (0..100)
81.2
3.0
84.0
3.1
74.0
4.0
82.6
3.5
B1 Understanding bushfire issues (self-rating 0..100)
63.8
3.3
68.1
2.7
62.0
3.2
72.1
2.4
B4& Knowledge index re bushfire issues (0..24)
16.1
0.4
17.6
0.5
13.6
0.6
13.8
0.7
C5 Brochures/pamphlets re bushfires read (%)
94
 
75
 
60
 
46
 
E1 Need for information re bushfire issues (%)
63
 
38
 
48
 
28
 
F1 Best source for info needed is CFG (%)
25
 
28
 
0
 
14
 
G2& Responsibility attitude regarding the bushfire risk
 

Accepted as own responsibility (2..10)

7.8
0.6
8.0
0.4
8.1
0.3
8.4
0.3
 

Seen as CFA's responsibility (3..15)

8.2
0.7
6.8
0.6
8.8
0.5
7.9
0.6
H1 Overall bushfire preparedness (self-rating) (0..100)
56.3
4.2
67.8
3.8
60.0
4.0
65.2
3.5
H2 Specific appraisals of preparedness (1..5)
 

Technically

3.4
0.2
3.9
0.2
3.1
0.2
3.3
0.2
 

Organisationally

3.6
0.2
3.5
0.2
2.9
0.2
3.6
0.1
 

Psychologically

3.6
0.2
3.4
0.2
3.5
0.2
3.9
0.1
O1& Overall extent of actions taken (0..16)
8.1
0.5
10.6
0.6
8.0
0.4
8.2
0.5
   
L1& Mean evaluation of CFG activities (10..50)
--
 
39.6
1.5
--
 
--
 
M1& Mean evaluation of materials used in CFG (10..50)
--
 
43.2
1.4
--
 
--
 
N1& Mean evaluation of CFA staff for CFG work (10..50)
--
 
42.6
1.2
--
 
--
 
P1 Overall improvement in understanding bushfires (1..5)
--
 
3.9
0.3
--
 
--
 
P2 Level of dependence of CFA re: bushfire threat (1..5)
--
 
2.9
0.3
--
 
--
 
P3 Overall assessment of participating in CFG (1..5)
--
 
4.3
0.3
--
 
--
 
P6 Improvement in dealing with problems as group (1..5)
--
 
3.1
0.2
--
 
--
 

Two important results from the table above are also presented in the following graphs. Figure 2 shows the means in phase 1 and 2 for the index O1& "number of actions taken", which increased significantly for C/F participants. A look at the specific items (not in Table 10) reveals that the highest single differences are "joint planning with neighbours" (62/20% in N2/C2) and "writing down planning for bushfire events" (44/08% for N2/C2).

Figure 3 demonstrates the changes in the attitude that CFA (rather than the residents themselves) is responsible for bushfire safety. This view (which reflects CFA policy) is indeed lower for C/F participants, and the intended reduction is stronger for N2 than C2 respondents.These graphs also indicate the considerable variance in the residents' responses (the dotted lines depict the standard deviation for each variable; the solid bar represents the standard error of the mean.)

In general the results are favorable for the C/F approach - yet the data gained so far are not as convincing as expected (and hoped for). However, unfortunately the - methodologically crucial - data collection in phase II was severely confounded by two 'external' events:

Figure 2
Figure 2: Change in Residents' Views on Bushfires issues - Variable O1&

Figure 3
Figure 3: Change in Residents' Views on Bushfires issues - Variable G2&

It seems highly likely that these circumstances increased problem awareness and consequently bushfire preparedness in all investigation areas, thus blurring the specific effects of C/F and reducing the difference between the 'experimental' group "N" and the 'control' group "C". (In a pertinent question in the last data collection, both the N2 and the C2 group indicated an improved understanding of bushfire issues because of a recent fire event; the effect was only moderate, but stronger for the control group).

Also, far less people than expected formed or joined new C/F groups in 1997, so the time the surveyed residents had spent with C/F activities was significantly shorter than originally planned (in most cases considerably less than half a year) and therefore the intended effects could not fully unfold.

Accordingly, the currently available results do not permit (yet) unequivocal conclusions about the strength of the outcomes of the C/F program. Nonetheless, evidence from the "E" study as well as other parts of the project (such as the focus groups conducted in 1997 and 1998 or the survey with personnel; cf. below) clearly indicate that the community-based "Fireguard" approach is beneficial on the whole to the bushfire preparedness of residents in fire risk areas and improves both individual and group risk mitigation.

In order to compensate the validity problems encountered in phase II and to further clarify the efficiency of the C/F approach, a third data collection with the N2/C2 sample is currently in preparation.

Findings from additional inquiries

The survey with CFA personnel (sub-study <P>; cf. Table 5) provided valuable additional information for the evaluation of the C/F approach. Tables 11 A & B contain selected results with respect to their perception of residents' knowledge and attitudes and their appraisal of the C/F program.

Interestingly, the CFA officers rated the residents' preparedness (variable H1) only as 42, clearly lower than the self-assessments shown in Table 10 above. Comments on the residents' information needs and their attitude towards responsibility (aspects E and G in Table 11A) also indicate a somewhat critical perspective.

Table 11A: Views of CFA Personnel on the Community Fireguard Program Part A

Aspect E: Subjective information needs of residents
Question (E2):
"Which of these topics are usually mentioned by residents who enquire with you?" (list; scale 0=never, 1=sometimes, 2=often)
Question (E3):
"Which topics do you find important for advancing the knowledge of residents?" (list; scale 1=medium relevance, 2= high relevance)
Responses to List Items
never mentioned
sometimes mentioned
often mentioned
high relevance
Features and behaviour of bushfires
05
63
32
95
Preparing house and property
00
15
85
95
Firefighting equipment (e.g. hose systems)
00
55
45
35
Fire-safe planting
00
63
37
35
Suitable clothing
05
74
21
65
Joint planning with neighbours
05
75
20
80
Evacuation
00
25
75
70
Aspect G: Expectations About Responsibility
Question (G4):
"What do you think is the main attitude of residents regarding responsibility?" (open)
Main Responses
Not their responsibility 67%; Accept responsibility but need information 11%; Have attitude that fire will not happen to them 11%; other 11%

The results in the second part (Table 11B) show the high satisfaction with C/F outcomes (variables P3, P10); consequently, increased support for the program is suggested (variable P9). The last block in Table 11B deals with the role of C/F facilitators. The replies seem to indicate that a fair degree of guidance and direction is indispensable in order to get resident groups activated and eventually taking responsibility for improving bushfire preparedness.

Table 11B: Views of CFA Personnel on the Community Fireguard Programme Part B

Aspect P: Outcomes Of Community Fireguard Program
Question (P3):
"To sum up - what's your overall assessment of the Community Fireguard program?" (scale 1-5)
Responses:
bad (0%), poor (0%), fair (6%), good (38%), excellent (56%) M = 4.4
Question (P10):
"In your view, to what degree do residents change their behaviour or attitude in terms of dealing with fires as a result of participating in a C/F group?" (list; scale 1-5)
Responses:
not at all (0%), a little (0%), moderately (12%), rather (59%), very much (18%) M = 4.1
Question (P9):
"Are there changes you would like to see in the Community Fireguard program? What do you suggest?" (open)
Main Responses (multiple replies)
more trained facilitators 26%; adaptation to changing needs 11%; more publicity 25%; better support within CFA 11%; improved resourcing 19%
Aspect T : Task & Role of CFG Facilitators
Question (T1):
"What activities do you expect from the facilitators of C/F groups, and what role should they play?" (open)
Main Responses
Direction, support, coordination  45%; education 36%; motivation, activating groups 27%
Question (T5):
"Regarding C/F groups: In reality, who is usually planning and deciding group activities: more the CFA staff or more the residents?" (scale 1-5)
Question (T6):
"And which way do you think would be most beneficial for a C/F group?"
Ratings:
 
influenced by residents
 
influenced by CFA staff
 
 
much more
slightly more
50:50
slightly more
much more
 
T5:
13%
37%
37%
13%
0%
M = 2.5
T6:
33%
56%
11%
0%
0%
M = 1.8
Results based on a survey ("sub-study P") with N=10 C/F facilitators and N=10 officers from regional CFA brigades

 


Assessing Fire Risk Communication: Conclusions

The findings from this evaluation project demonstrate the considerable potential of community-based fire preparedness programs; they also enable a critical review in order to identify avenues for further improvement. The main points are summarized in Table 12.

Table 12: Assessing Fire Risk Communication: Some Conclusions

Appraisal of the Community Fireguard Programme
  • participants evaluate C/F content+conduct favorably acceptance of responsibility is enhanced
  • positive behavioral outcomes (improved bushfire preparedness) very likely
  • info overload and inertia may impede program effectiveness
  • active/stimulating role of facilitators is necessary
Methodological lessons
  • develop sound theoretical framework first!
  • employ state-of-the-art experimental design!
  • don't operationalize behavioral outcomes via cognitive effects!
  • distinguish short-term vs long-term program impacts!
  • tailor evaluations to the characteristics of the local case!
  • discern alternative causes for observed effects/changes!
  • plan ahead how to deal with confounding events in the field!
  • identify restrictions to the results' internal and external validity!
  • incorporate evaluation research in advance!
Important questions for on-going research
  • socio-psychological barriers to involvement and implementation?
  • optimal combination of info material and group work?
  • validity for 'ethnic' (non-anglo) residents?
  • long-term stability of program effects & behavioral change?

Besides the many substantive results, the project also provides significant methodological insights. The two principal messages are: firstly, that the empirical assessment of risk communication programs is both feasible and fruitful, and secondly, that evaluation studies are 'serious business' which can fail if not conducted carefully. The present project seems to be the only 'full' evaluation of fire risk communication outcomes - obviously more studies are desirable in order to consolidate the scientific knowledge on effective strategies and necessary prerequisites. In the middle section of Table 12, some more general conclusions are listed. Theoretical/conceptual efforts and tailoring general principles of evaluation towards the specific program case under study are particularly important considerations. Strict methodological standards are absolutely indispensable, and the validity and scope of findings needs to be clarified by the researcher. (In fact misleading knowledge might be worse than missing knowledge, as Kasperson & Rohrmann 1988 stated in a review).

Clearly evaluation studies should be incorporated into the planning of a program from the very beginning. From a researcher’s viewpoint, it seems important to ensure trustful collaboration with the program's agency, to secure sufficient funds early enough (proper evaluation will be expensive), and finally to participate in the dissemination of evaluation findings, particularly outside 'academic circles' - hopefully enhancing the findings' utilization. Evaluation research should gain from close co-operation with the agency/authority which is responsible for the risk communication program. (This might also help to 'transmit' the complexity of evaluations, as non-researchers tend to underestimate the design, cost and time requirements). However, collaboration must be balanced with the necessary independence of the researcher; otherwise the credibility of the results might be reduced, as there will always some social pressure for positive results (Patton 1986).

Finally, many questions about the preconditions for successful risk information/communication/education are not yet answered; some are listed at the bottom of Table 12. On-going research into these issues will provide requisite knowledge for enhancing and refining hazard mitigation and disaster preparedness campaigns.


Acknowledgments

Close collaboration with CFA officers was crucial for this project; therefore I would like to thank David Beckingsale, Jon Boura, Leon Collet, Stephen Petris and Alan Rhodes for their continuous support Sincere thanks also go to my hard-working research assistants, including Susan Bell, Lucia Boxelaar, Jennifer Makin, Fiona McIntosh and Melinda Norris. Finally, a large number of residents participated patiently in the various sub-studies, which is very much appreciated.


References

Ajzen, I. (1993). Attitude theory and the attitude-behavior relation. In D. Krebs & P. Schmidt (Eds.), New directions in attitude measurement (pp. 41-57). New York: de Gruyter.

Atman, C. J., Bostrom, A., Fischhoff, B., & Morgan, M. G. (1994). Designing risk communications: Completing and correcting mental models of hazardous processes (part 1). Risk Analysis, 14, 779-788.

Barham, R. (1996). Fire engineering and emergency planning. London: Spon.

Beckingsale, D. (1994). Community fireguard and the rural-urban interface. Fire Management Quarterly, 1-8.

Beckingsale, D. & Petris, S. 1994. The CFA community FireGuard program (manuscript). Melbourne: Country Fire Authority.

Canter, D. (1985). Studies of human behaviour in fire: empirical results and their implications for education and design. University of Surrey.

Canter, D. (1990). An overview of human behavior in fires. In D. e. Canter (Ed.), Fires and human behavior (pp. 205-243). London: Fulton.

Chase, R. A. (1993). Protecting people and resources from wildfire: Conflict in the interface. In A. Ewert, D. J.; Magill, A. W. (Ed.), Culture, conflict and communication in the wildland-urban interface. Boulder: Westview.

Country Fire Authority (CFA) (1995). Community Fireguard. Melbourne: CFA.

Country Fire Authority (CFA). (1995). Bushfires - Living with Australia's natural heritage. Melbourne: CFA.

Covello, V. T., Mccallum, D. B., & Pavlova, M. (1989). Effective risk communication. The role and responsibility of government and nongovernment organizations. New York: Plenum.

Drabek, T. E. (1990). Emergency management: Strategies for maintaining organizational integrity. NY: Springer-Verlag.

Eagly, A. H., , & Chaiken, S. (1993). The psychology of attitudes. Fort Worth: Harcourt-Brace.

Fink, A. (1993). Evaluation fundamentals, guiding health programs, research,and policy. London: Sage.

Fischhoff, B., Bostrom, A., & Quadrel, M. J. (1993). Risk perception and communication. Annual Review of Public Health, 14, 183-203.

Gaull, E. (1997). Show me the results. Fire chief, May, 75-81.

Handmer, J., Dutton, B., Guerin, B., & Smithson, M. (1991). New perspectives on uncertainty and risk. Canberra: ANU.

Jones, R. (1987). Community involvement: The missing link. Bush Fire Bulletin, 14-15.

Jungermann, H., Rohrmann, B. & Wiedemann, P. (1991). Risikokontroversen - Konzepte, Konflikte, Kommunikation. Berlin etc.: Springer.

Kasperson, R. M., & Rohrmann, B. (1988). Evaluation of risk communication strategies. In H. Jungermann, R. M. Kasperson, & P. M. Wiedemann (Eds.), Themes and tasks of risk communication Jülich: Research Center Jülich.

Kasperson, R. E., & Stallen, P. M. (Eds.). (1990). Communicating risks to the public. Dordrecht: Kluwer.

Krimsky, S., & Plough, A. (1988). Environmental hazards - Communicating risks as a social process. Dover: Arborn.

Krusel, N., & Petris, S. (1992). Staying alive: lessons learnt from a study of civilian deaths in 1983 Ash Wednesday bushfires. Fire Management Quarterly, 1-17.

Lundgren, R. (1994). Risk communication: A handbook for communicating environmental safety and health risks. Columbus/OH: Batelle Press.

Munro-Clark, M. (Ed.) 1992. Citizen participation in government. Sydney: Hale and Ironmonger.

Patton, M. Q. (1986). Utilization-focused evaluation. London: Sage.

Pyne, S. J. (1991). Burning bush: A fire history of Australia. Melbourne:Allen and Unwin.

Raphael, B. (1986). When disaster strikes - How communities and individuals cope with catastrophe. New York: Basic Books.

Robertson, J. C. (1989). Introduction to fire prevention. New York: Macmillan.

Rhodes, A., & Reinholtd, S. (1998). Beyond technology: A holistic approach to reducing residential fire fatalities. The Australian Journal of Emergency Management, 13, 39-45.

Rohrmann, B. (1992). The evaluation of risk communication effectiveness. Acta Psychologica, 81, 169-192 (a).

Rohrmann, B. (1995). Effective risk communication for fire preparedness: A conceptual framework. The Australian Journal of Emergency Management 10, 43-48.

Rohrmann, B. (1998). Assessing hazard information/communication programs. Australian Psychologist, 33, 1-8.

Rohrmann, B. (1998). The risk notion: Epistemological and empirical considerations. In M. G. Steward & R. E. Melchers (Eds.), Integrated risk assessment: Applications and regulations (pp. 39-46). Rotterdam: Balkama.

Rohrmann, B., Wiedemann, P., & Stegelmann, H. U. (Eds.). (1991). Risk communication - An interdisciplinary bibliography. Juelich: Research Center MUT.

Rossi, P. H., & Freeman, H. E. (1993). Evaluation: A systematic approach. Beverly Hills: Sage.

Smith, P., Nicholson, J., & Collett, L. (1996). Risk management in the fire and emergency services: Enhancing community safety. Melbourne: Proceedings of the "Fire Australia" Conference.

Webler, T., & Renn, O. (1995). A brief primer on participation:Philosophy and practice. In O. Renn, T. Webler, & P. Wiedemann (Eds.), Fairness and competence in citizen participation evaluating models for environmental discourse (pp. 17-34). Hingham/MA: Kluwer.

Whelan, M. (1987). Encouraging community participation in bushfire preparedness: the role of the Municipal Fire Prevention Officer. Monash University: Thesis, Environmental Science.

Yates, J. F. E. (1992). Risk-taking behavior. Chichester: Wiley.


Copyright

Bernd Rohrmann © 1999. The author assigns to the Australasian Journal of Disaster and Trauma Studies at Massey University a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Massey University to publish this document in full on the World Wide Web and for the document to be published on mirrors on the World Wide Web. Any other usage is prohibited without the express permission of the author.


| Home | Current | Back Issues | Reports | Conferences | Books | Links | Information |

Comments to
Trauma.Webmaster@massey.ac.nz
Massey University, New Zealand
URL: http://www.massey.ac.nz/~trauma/
Disclaimer

Last changed November 20, 2001
Copyright © 1999,2001 Massey University