Negotiation, the Journal of Organizational Computing and Electronic Commerce,

JERRY FJERMESTAD AND STARR ROXANNE HILTZ

JERRY FJERMESTAD is an Associate Professor in the School of Management, NewJersey Institute of Technology. Jerry received his B.A. from Pacific Lutheran Univer-sity, an M.S. from Polytechnic University, and an M.B.A. and Ph.D. from RutgersUniversity. His current research interests are in collaborative technology, decisionsupport systems, data warehousing, and enterprise information systems. Jerry haspublished in the Journal of Management Information Systems, Group Decision andNegotiation, the Journal of Organizational Computing and Electronic Commerce,Information and Management, and the Proceedings of the Hawaii International Con-ference on System Sciences.

STARR ROXANNE HILTZ is Distinguished Professor, Computer and Information Sci-ence, New Jersey Institute of Technology, where she also directs the collaborativeSystems Laboratory and the Virtual University project. Her research interests encom-pass the acceptance and impacts of computer systems, centering on computer-medi-ated communication, particularly Group Support Systems and educational applications.Publications include six books and articles in a variety of journals and conferenceproceedings, including Communications of the ACM, Journal of Management Information

Systems, Management Information Systems Quarterly, Management Science, Human Commu-nications Research, the American Journal of Sociology, and the Proceedings of the Hawaii

International Conference on System Sciences.

ABSTRACT: This paper presents a descriptive evaluation of 54 case and field studiesfrom 79 published papers spanning two decades of group support systems (GSS)research. It organizes the methodology and results of these studies into a four-factorframework consisting of contextual factors, intervening factors, adaptation factors,and outcome factors. The tables will provide the GSS researcher with a summary ofwhat has been studied. The appendices provide a detailed description of the method-ology and the results.

KEY WORDS AND PHRASES: case studies, descriptive evaluation, field studies, groupsupport system, research integration.

IN THE ERA OF THE INTERNET, millions of people are glued to their workstations forhours a day, and even “old style” Fortune 500 companies are adapting some characteris-tics of the virtual organization. Group Support Systems (GSS) provide the collaboration

This research is supported by a grant from the National Science Foundation (CISE–TO 9732354)and by the New Jersey Center for Pervasive Information Technology (NJCPIT).

Journal of Management Information Systems / Winter 2000, Vol. 17, No. 3, pp. 113–157.

© 2000 M.E. Sharpe, Inc.0742–1222 / 2000 $9.50 + 0.00.

114 FJERMESTAD AND HILTZ

tools that many of these organizations use. This paper summarizes the published caseand field studies on GSS use (by “real” groups in “real” meetings) to date, and providesan aggregate analysis of the methodology and results.

This study can be viewed as an updating of prior papers [4, 11] which comparedlaboratory and field research on GSS conducted in same-time, same-place decisionrooms. It includes many more studies, of course. The prior papers found ten fieldstudies that had been published in journals through the summer of 1990, whereas thisone, a decade later, is based on 54 field studies.

Studies Included

WE LOCATED 54 DIFFERENT CASE AND FIELD STUDIES, published through mid-2000,that met our criteria for this analysis. First, the study had to be published in an EnglishLanguage refereed journal or conference proceeding. For example, unpublished dis-sertations or non-refereed conference presentations are not included. Second, theywere studies of one or more specific groups, which we defined as comprising at leastthree members. Third, they used a computer-based system with at least minimal fea-tures designed to support group communication and decision-making processes. Thestudy had to be an action research, case study, or field study that required the group towork on a specific task, and that posed some sort of research question and collectedsome data to help to answer this question. The task in most cases was a “real world”task with no right or wrong solution. In three cases the task was part of the require-ments for a course.

The results of some studies were presented in more than one paper. If the design ofthe study and description of the organization, subjects, and task were the same, thedifferent papers were determined to be on the same study, and were given only onenumber. We thus have a total of 79 papers on the 54 different studies.

The Theoretical Framework

TO ORGANIZE THE INFORMATION IN THE STUDIES, we used the comprehensive theo-retical framework previously developed to integrate and analyze all of the informa-tion for experimental studies of GSS. This integrated framework was developed onthe basis of various contingency theory approaches to explaining GSS success, toprovide complete coverage of factors present in the literature as a whole, and hasbeen previously published [6, 7] and will not be reproduced here.

Categorization of the Studies and Results

THE MAJOR ASPECTS OF THE METHODOLOGY and outcomes of field studies on GSS,using the study as the unit of analysis, have been put into a database and organizedinto charts. Appendix 1 shows what studies are included and the references. Appen-dix 2 summarizes the methodology and other parameters for each study, and Appen-

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 115

Table 1. Factors Model: Contextual Factors (Unit of Measure is Case)

4.1 TECHNOLOGY

Task Support: ToolsAgendaComputer Model

Alternative GenGroup OutlinerCross ImpactTopic

CommenterRankingEBSIDEFVotingIssue

OrganizerNGTStandard PackageMCDMPolicyNone

CategorizerCost Benefit

1134111423317101823451

Other

Group WritingNot ReportedProcess StructureGroup ProximityDispersedDecision RoomAnonymityAnonymity (A)Identified (I)FacilitationFacilitatorNo FacilitatorChauffeurTime DispersionSynchronousAsyncronousLevelsLevel 1Level 2

9453222341194597471212

Communication ModeCMCDSSGSS

9144

Design–GSS SystemDecision Conference1DecisionAnalytics1EIES4Email2EMS1Facilitator1GroupSystems30Plexcenter1PC Work Station1GroupWise2SAMM5TeamFocus2Teamate1VisionQuest2

dix 3 summarizes the results. Tables 1 to 3 are summary counts of the variables from

these charts.

What Has Been Studied: Contextual Factors

Table 1 shows the counts for the technology subfactor. Table 2 highlights the countsfor group and context subfactors of the contextual factors and the interveningsubfactors.

Technology

Communication Mode

Group Support Systems have been classified into three primary types (Table 1): “DSS”(Decision Support Systems) GSS, or “CMC” (Computer-Mediated Communication).The majority of the studies (81.5 percent) used a GSS.

CMC refers to a system designed primarily to support text-based and generallyasynchronous (or anytime-anywhere, through computer networks) group discussion,

116 FJERMESTAD AND HILTZ

Table 2. Factors Model: Contextual and Intervening Factors (Unit of Measures isCase)

4.2 GROUPGroup

Composition(Subject Type)ProfessionalsManagementSenior

ManagementMilitarySoftwareEngineersStudentsLeadershipLeader

Not Reportedor no LeaderEmergentLeader

343841412411

CommunicationGovernment(U.S.)

GovernmentNon-U.S.Medical/HospitalUtilityMilitary/DefenseOil

ConsultingOtherUniversityCourseInsuranceAccounting/BankNationHong KongEuropeanDenmarkGermanMexicanNetherlandsSouth AfricaUSAAustraliaNew ZealandMixed

INTERVENINGMethodActionResearchCase StudyField StudyQuasi-ExperimentalField StudyNumberof Sessions123456 to 910 to 1329

45333541211118134131296326229312

MultipleNRTrainingNot

ReportedTrainingMentionedNone120+ minSessionLength

< Day101 < 4 Days71 < 3 Weeks6MultipleSessions9Asynchronous9Not

Reported12Group Size(Subjectsper group)range to 10or less14range to 2019range to 304range to 402range to 60 plus3Not Reported12Data Collection

102715

Questionnaires36Interviews26Observations21Session Logs19Case Reports3Metrics1Self Completion1Project Grade1Comparisons2Comments1Ideas generated1Content analysis1Not Reported3Ballots1

2918

32621

Task

ImplementationAlternative2BPR11CompetitiveAdvantage2ContractNegotiation1EconomicDevelopment2Idea

Generation1Judgment2Manufacturing1Planning7Models1SoftwareDevelopment2StrategicPlanning19CourseProject2FoundationTask14.4 CONTEXTOrganizationTypes*UniversityManufacturingComputer

763

2

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 117

Table 3. Factors Model Outcome Factors (Unit of Measures Is Measure)

1. EfficiencyMeasuresImproved28Did not improve1No difference1No Measures242. Effectiveness MeasuresImproved39Did not improve2No difference3No Measures103. Satisfaction MeasuresProcess SatisfactionImproved20No difference2No Measures32Outcome SatisfactionImproved11Did not improve2

Improved:

No difference1

No Measures33ParticipationImproved16Did not improve3No difference0No Measures354. Consensus MeasuresImproved8Did not improve3No difference0No Measures435. Usability MeasuresImproved12Did not improve2No difference1No Measures39

Process support

Process structureTask structure

Information exchangeRole perceptionsCommunicationNumber of ideas

The ability to deal with taskcomplexityCohesiveness

Flexibility

Divergent and convergentEnriched communicationImproved focus

Increased number of ideasReduced stress

Knowledge and knowledge sharingHigh trust groups are effective andhave active participation

such as a computer conferencing or bulletin board system, that may or may not haveGSS tools included. A total of nine studies (16.7 percent) used a CMC system.Systems

Eleven different systems were used. GroupSystems (developed at the University ofArizona) and its predecessors accounted for 55 percent (30/54) of the GSS systems;SAMM from the University of Minnesota was used in five studies; and EIES fromThe New Jersey Institute of Technology was used in four instances.Tools

The tools included in GroupSystems and its predecessors are the most frequentlyemployed. The most frequently used task support tool is Brainstorming (23 studies)

118 FJERMESTAD AND HILTZ

for idea generation, followed by Voting (17), Topic Commenter (11), Issue Organizer(10), and Categorizer (5). These tools were primarily used by the groups to elicit newideas and organize them. It is interesting to note that only a few specialized tools wereused: IDEF (Integrated Computer-Aided Manufacturing Definition) tools (3), Nomi-nal Group Technique (1), and Multicriteria decision-making (2).Process Structures

Eighty three percent of the studies were synchronous decision room–based. Of thenine studies that were asynchronous, six were used in organizational settings. Theother three were used in an academic environment. Anonymity was employed in 59percent of the studies.Level

The “level” of the GSS or CMC system is a rough coding of its sophistication in termsof GSS features, and follows the descriptions of “level 1” and “level 2” systems by[5]. The majority of the systems used (87 percent) are level 2 systems.Facilitation

The case and field studies primarily used facilitation methods (62.9 percent, or 34/54of the studies).

Group

For the case and field studies all of the group variables were treated as moderatorvariables. It was surprising that most field studies did not explicitly report whetherthe groups were established or ad hoc.Group Composition

The case and field studies utilized mostly (92 percent) professionals. Managers, se-nior managers, or professional staff were utilized in 45 studies, and military or De-partment of Defense personnel in another four instances.Leadership

Most organizational project groups have leaders. However, most of the studies (41instances) did not report if there were assigned leaders for the task groups.

Task

Task Type Implementation

Fourteen different categories of task implementations were used in the case and fieldstudies. Nineteen (35 percent) of the tasks were strategic planning and 11 were busi-ness process reengineering tasks.

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 119

Context

This includes environmental and organizational variables. The case and field studiestypically did not report many important bits of information, such as the environment,time pressure, and culture. The most frequent was the U.S. government, with ninestudies, followed by seven from university settings and six from manufacturing. Therewere also six from foreign governments [9, 10, 13].Culture/Nationality

Though dominated by U.S. studies, the approximately one-fourth from a variety ofother cultures is a very good start toward the ability to assume that generalizationsabout use and impacts of GSS do hold for at least the “developed” world. Elevenstudies, including eight from the Netherlands, were European-based. There were alsothree from New Zealand, one from Australia, and two from Hong Kong. In terms ofthe less developed nations, however—which actually represent the majority of theworld’s population—there were only one from Mexico and one from Africa.

Intervening Factors

The intervening variables include two major categories: methods and summary con-structs. (Summary constructs are beyond the scope of this paper; see [6].) Methodsrepresent the basic manipulation and measurement techniques that are available tothe researcher, including study design, task implementation, session length, numberof sessions, and training (Table 2).Method Study Design (type)

Fifty percent (27 of 54) of the studies were described as case studies and another 28percent (15 of 54) were field studies. Ten studies were action research. There weretwo quasi-experimental field studies.Training

Twenty-nine studies did not report on this important detail of the methodology at all.Another 18 mention that some sort of training was given, but no details are provided.Number of Sessions

Slightly over one-third (21 out of 54) of the studies did not report the number ofsessions the groups participated in. Fifty-four percent of the case and field studies didhave at least two or more sessions reported.

120 FJERMESTAD AND HILTZ

Session Length

In the case and field studies, 59 percent of the groups spent extensive time on theirtask: Either they had multiple sessions (9), or between 1 and 4 days (7), or between 1and 3 weeks (6), to complete their tasks. Nine of the studies were asynchronous,extending over weeks, months, or even years.Group Size (Subjects per group)

The ranges for the number of subjects per group are quite variable. They go from fiveor six subjects per group up to 60 plus.Data Collection

Most of the studies use multiple methods to collect data. Thirty percent of the studiesuse questionnaires, 22 percent use post-case interviews, and 16 percent use sessionlogs to aid in the analysis.

Outcome Factors

Table 3 shows the results of the outcome factors for the 54 case and field studies.Most of these results are based on subjective perceptions from questionnaires or in-terviews, or the judgment of the researchers. However, some of the efficiency andeffectiveness measures are quantified.Efficiency

Sixty-two percent (28 out of 54) of the case and field studies suggest that efficiencywas improved over manual or face-to-face methods. Ninety-three percent (28 out of30) of those that measured efficiency outcomes report improvements. Many of thetime savings reported are quite impressive. For example, Bikson [1] describes GSSuse in 102 sessions at the World Bank as “vastly more efficient” than other meetings.Dean et al. [2] quantified efficiency in a business process reengineering task usingIDEF tools, finding that participants produced “251 percent more activities and 175percent more ICOMS per day.” In another study using IDEF, Dennis et al. [3] foundthat completed models took only one week for the GSS groups, versus 6 weeks fortraditional processes.

Effectiveness: Eighty-nine percent (39 out of 44) of the studies that measured ef-fectiveness report that effectiveness was improved using GSS technology in compari-son to other methods. Though most of these measures are based on subjectiveimpressions of participants, some were able to quantify the increased effectiveness.For example, Post’s [8] study of 64 sessions in a major manufacturing company claimsthat GSS saved $432,260 while improving the quality of decisions. Van Genuchten et

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 121

Table 4. Characteristics of Successful and Unsuccessful GSS Implementations

Successful ImplementationsFacilitatorLeadershipMany sessions

Training on the technologyComplex tasks

Idea generation task for Decision Room GSSDecision making tasks for Asynchronous CMCPermit verbal and electronic communicationUser defined approaches

Culturally sensitive implementationsAnonymityHigh trust

Unsuccessful ImplementationsNo Facilitator

A dominating or unenthusiastic leaderFew sessions

No training on the technologyTrivial tasks

Discourage verbal communicationLimited approachesMisappropriated AnonymityLow trust

al. [12] counted code defects found during software inspection meetings and foundthe number to be “considerably” higher than for traditional meetings

Satisfaction: Three separate measures of satisfaction are included: process satis-faction, outcome satisfaction, and participation. The results clearly suggest that groupsare more satisfied with the technology and processes of GSS compared to manual orface-to-face meetings. When analysis is offered of reasons for overall improvementsin satisfaction, the most frequent themes are that it is due to an improvement in pro-cess (more participation, due to simultaneous input and/or anonymity features of themedium) and/or to perceived greater quality of the results.

Consensus was measured only in 11 instances, eight of which report positive results.Usability of the systems is perceived positively in 12 of the 15 studies that mea-sured it.

Overall Outcomes combines all of the results measures listed above. In compari-son to the experimental studies, where the overall positive effects of GSS technologywere only 16.6 percent, the overall positive effects reported for the case and fieldstudies is 86.5 percent (134 out of 155).

Characteristics of Successful and Unsuccessful Implementations

In order to aid the GSS researcher and organizational manager we compiled Table 4,which highlights the characteristics of successful versus unsuccessful GSS imple-mentations. This list is not all-inclusive, but a guide based upon 54 case and fieldstudies that we evaluated.

Discussion and Conclusions

THE OBJECTIVE OF THIS PAPER is to provide the GSS researcher and organizationalmanager with an up-to-date descriptive evaluation of the GSS research in organiza-tions. The results suggest that many different types of organizations have exploredusing GSS technology to improve many aspects of business decision-making. Most

122 FJERMESTAD AND HILTZ

of these have been in a decision room environment using a facilitator. Several recentstudies have used asynchronous GSS technology without facilitators to achieve thesame ends. These results are very promising. The groups report that the technologyimproves efficiency, effectiveness, consensus, usability, and satisfaction above that ofmanual methods.

Acknowledgments: The authors wish to express their gratitude to the special issue Guest Edi-tors, Robert Briggs, Jay Numamaker, Ralph Sprague, and Gert-Jan de Vreede, and to the Edi-tor-in-Chief, Vladimir Zwass, for their valuable comments in revising this paper.

REFERENCES

1. Bikson, T.K. Groupware at the world bank. In C.U. Ciborra (ed.), Groupware and Team-work Invisible Aid or Technical Hindrance? Chichester, UK: John Wiley & Sons, 1996, pp.145–183.

2. Dean, D.L.; Lee, J.D.; Orwig, R.E.; and Vogel, D.R. Technological support for groupprocess modeling. Journal of Management Information Systems, 11, 3 (1995), 43–63.

3. Dennis, A.R.; Daniels, R.M.; Hayes, G.; and Nunamaker, J.F. Methodology-driven use ofautomated support in business process re-engineering. Journal of Management InformationSystems, 10, 3 (1993–1994), 117–138.

4. Dennis, A.R.; Nunamaker, J.F. Jr.; and Vogel, D.R. A comparison of laboratory and fieldresearch in the study of electronic meeting systems. Journal of Management Information Sys-tems, 7, 3 (1990–1991), 107–135.

5. DeSanctis, G. and Gallupe, R.B. A foundation for the study of group decision supportsystems. Management Science, 33, 5 (1987), 589–609.

6. Fjermestad, J. An integrated framework for group support systems. Journal of Organiza-tional Computing and Electronic Commerce, 8, 2 (1998), 83–107.

7. Fjermestad, J., and Hiltz, S.R. An assessment of group support systems experimentalresearch: methodology and results. Journal of Management Information Systems, 15, 3 (1998–1999), 7–149.

8. Post, B.Q. A business case framework for group support technology. Journal of Manage-ment Information Systems, 9, 3 (1993), 7–26.

9. Sheffield, J., and Gallupe, R.B. Using electronic meeting technology to support economicpolicy development in new Zealand: short-term results. Journal of Management InformationSystems, 10, 3 (1994), 97–116.

10. Sheffield, J., and Gallupe, R.B. Using group support systems to improve New Zealandeconomy, part II: follow-up results. Journal of Management Information Systems, 11, 3 (1995),135–153.

11. Valacich, J.S.; Dennis, A.R.; and Nunamaker, J.F. Electronic meeting support: theGroupSystems concept. International Journal of Man–Machine Studies (1992), 261–282.12. Van Genuchten, M.; Cornelissen, W.; and Van Dunk, C. Supporting inspections with anelectronic meeting system. Journal of Management Information Systems, 14, 3 (1998), 165–178.

13. Vreede, G.J. de. Collaborative business engineering with animated electronic meetings.Journal of Management Information Systems, 14, 3 (1998), 171–164.

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 123

Appendix 1: The Studies Included in the Analysis

1. Aldelman, L. Real-time computer support for decision analysis in a group setting: an-other class of decision support systems. Interfaces, 14, 2 (1984), 75–83.

2. Adkins, M.; Sheare, R.; Nunamaker, J.F.; Romero, J.; and Simcox, F. Experiences usinggroup support systems to improve strategic planning in the Air Force. Proceedings of the Thirty-First Hawaii International Conference on Systems Sciences, 1 (1998), 515–524.

3. Alavi, M. An assessment of electronic meeting systems in a corporate setting. Informa-tion & Management, 25 (1993), 175–182.

4. Bikson, T.K. Groupware at the world bank. In C.U. Ciborra (ed.), Groupware and Team-work: Invisible Aid or Technical Hindrance? Chichester, UK: John Wiley & Sons, 1996, pp.145–183.

5. Briggs, R.O.; Adkins, M.; Mittleman, D.; Kruse, J.; Miller, S.; and Nunamaker, J.F., Jr.A technology transition model derived from field investigation of GSS use aboard the U.S.S.Coronado. Journal of Management Information Systems, 15, 3 (Winter 1998–1999), 151–196.6. Caouette, M.J., and O’Connor, B.N. The impact of group support systems on corporateteams’ stages of development. Journal of Organizational Computing and Electronic Com-merce, 8, 1 (1998), 57–81.

7. Carmel, E.; Herniter, B.C.; and Nunamaker, J.F. Labor-management contract negotia-tions in an electronic meeting room: a case study. Group Decision and Negotiation, 2 (1993),27–60.

8. Corbitt, G.F; Christopolus, M.; and Wright, L. New approaches to business processredesign: a case study of collaborative group technology and service mapping. Group Decisionand Negotiation, 9, 2 (2000), 97–107.

9. Davison, R. The role of groupware in requirements specification. Group Decision andNegotiation, 9, 2 (2000), 149–160.

10. Davison, R., and Vogel, D. Group support systems in Hong Kong: an action researchproject. Information Systems Journal, 10, 1 (2000), 21–40.

11. Dean, D.L.; Lee, J.D.; Orwig, R.E.; and Vogel, D.R. Technological support for groupprocess modeling. Journal of Management Information Systems, 11, 3 (1995), 43–63. Dean,D.; Orwig, R.; Lee, J.; and Vogel, D. Modeling with a group modeling tool: group support,model quality, and validation. Proceedings of the Twenty-Seventh Hawaii International Con-ference on Systems Sciences, 3 (1994), 214–223.

12. Dennis, A.R. Electronic support for large groups. Journal of Organizational Computing,4, 2 (1994), 177–197.

13. Dennis, A.R.; Daniels, R.M.; Hayes, G.; and Nunamaker, J.F. Methodology-driven useof automated support in business process re-engineering. Journal of Management InformationSystems, 10, 3 (1993–1994), 117–138. Dennis, A.R.; Hayes, G.S.; and Daniels, R.M. Re-Engineering business process modeling. Proceedings of the Twenty-Seventh Hawaii Interna-tional Conference on Systems Sciences, 4 (1994), 244–253.

14. Dennis, A.R.; Hayes, G.S.; and Daniels, R.M. Business process modeling with groupsupport systems. Journal of Management Information Systems, 15, 4 (1999), 115–142.

15. Dennis, A.R.; Heminger, A.R.; Nunamaker, J.F.; and Vogel, D.R. Bringing automatedsupport to large groups: the Burr–Brown experience. Information & Management, 18 (1990),111–121.

16. Dennis, A.R.; Nunamaker, J.F.; and Paranka, D. Supporting the search for competitiveadvantage. Journal of Management Information Systems, 8, 1 (1991), 5–36.

17. Dennis A.R.; Tyran, C.K.; Vogel, D.R.; and Nunamaker, J.F. Group support systems forstrategic planning. Journal of Management Information Systems, 14, 1 (1997), 155–184.18. DeSanctis, G.; Poole, M.S.; Dickson, G.W.; and Jackson, B.M. Interpretive analysis ofteam use of group technologies. Journal of Organizational Computing, 3, 1 (1993), 1–29.19. DeSanctis, G.; Poole, M.S.; Lewis, H.; and Desharnais, G. Using computing in qualityteam meetings: initial observations from the IRS–Minnesota project. Journal of ManagementInformation Systems, 8, 3 (1992), 7–26. DeSanctis, G.; Poole, M.S.; Lewis, H.; and Desharnais,G. Using computing in quality team meetings: initial observations from the IRS–Minnesotaproject. Proceedings of the Twenty Fourth Annual Hawaii International Conference on SystemSciences, 3 (1991).

124 FJERMESTAD AND HILTZ

20. George, J.F.; Nunamaker, J.F.; and Valacich, J.S. Electronic meeting systems as innova-tion: a study of the innovation process. Information & Management, 22 (1992), 187–195.21. Hiltz, S.R., and Turoff, M. Computer networking among executives: a case study. Jour-nal of Organizational Computing, 1, 4 (1991), 357–376.

22. Iacono, C.S., and Weisband, S. Developing trust in virtual teams. Proceedings of theThirtieth Hawaii International Conference on Systems Sciences, 2 (1997), 412–420.

23. Jarvenpaa, S.L., and Leidner, D.E. Communication and Trust in Global Virtual Teams.Journal Computer Mediated Communications, 3, 4 (June 1998). http://www.ascusc.org/jcmc.24. Kock, N. Can communication medium limitations foster better group outcomes? Anaction research study. Information & Management, 34 (1998), 295–305.

25. Kock, N. Fostering interdepartmental knowledge communication through groupware: aprocess improvement perspective. Proceedings of the International ACM SIGGROUP Confer-ence on Supporting Group Work, November (1997), 29–37. Kock, N.F., and McQueen, R.J.Asynchronous groupware support effects on process improvement groups: an action researchstudy. Proceedings of the Seventeenth International Conference on Information Systems (1996),339–355. Kock, N., and McQueen, R.J. Groupware support as a moderator of interdepartmen-tal knowledge communication in process improvement groups: an action research study. Infor-mation Systems Journal, 8 (1998), 183–198. Kock, N.F., and McQueen, R.J. An action researchstudy of effects of asynchronous groupware support on productivity and outcome quality inprocess redesign groups. Journal of Organizational Computing and Electronic Commerce, 8,2 (1998), 149–168

26. Krcmar, H.; Lewe, H.; and Schwab, G. Empirical CATeam research in meetings. Pro-ceedings of the Twenty-Seventh Hawaii International Conference on System Sciences, IV (1994),31–40.

28. Markus, L.M. Asynchronous technologies in small face-to-face groups. Information Tech-nology & People, 6, 1 (1992), 29–48.

29. McCart, A.T., and Rohrbaugh, J. Evaluating group decision support system effective-ness: a performance study of decision conferencing. Decision Support Systems, 5 (1989), 243–253.

30. Morales, B.; Moreira, H.; and Vogel, D. Group Support for Regional Development inMexico. Proceedings of the Twenty-Eight Hawaii International Conference on System Sci-ences, IV (1995), 212–239.

31. Muller, P.C., and de Vreede, G.J. Improving interaction in new product development: acase study. Proceedings of the INFORMS Sessions on GDSS as Support for DesigningOrganizzations and Infromation Systems, III (1995), 19–45. Muller, P.C., and de Vreede, G.J.Supporting the conceptualization of a new Telecommunication service with group decisionsupport. Proceedings of the GroupSystems Sixth Annual Users’ Conference (1995).

32. Niederman, F., and Bryson, J. Influence of computer-based meeting support on processand outcomes for a divisional coordinating group. Group Decision and Negotiation, 7 (1998),293–325.

33. Nunamaker, J.F.; Applegate, L.M.; and Konsynski, B.R. Facilitating group creativity:experience with a group decision support system. Journal of Management Information Sys-tems, 3, 4 (1987), 5–19. Nunamaker, J.F.; Applegate, L.M.; and Konsynski, B.R. Facilitatinggroup creativity: experience with a group decision support system. Proceedings of the Twenti-eth Annual Hawaii International Conference on System Sciences, 1 (1987), 422–430.

34. Post, B.Q. A business case framework for group support technology. Journal of Manage-ment Information Systems, 9, 3 (1993), 7–26. Post, B.Q. A business case framework for groupsupport technology. Proceedings of the Twenty-Fifth Hawaii International Conference on Sys-tem Sciences, 4 (1992), 34–45.

35. Quaddus, M.A.; Atkinson, D.J.; and Levy, M. An application of decision conferencing tostrategic planning for a voluntary organization. Interfaces, 22, 6 (1992), 61–71.

36. Sheffield, J., and Gallupe, R.B. Using electronic meeting technology to support eco-nomic policy development in new Zealand: short-term results. Journal of Management Infor-mation Systems, 10, 3 (1994), 97–116. Sheffield, J., and Gallupe, R.B. Using group supportsystems to improve New Zealand economy part I: short-term results. Proceedings of the Twenty-Sixth Annual Hawaii International Conference on System Sciences, 4 (1993), 469–477. Sheffield,J., and Gallupe, R.B. Using group support systems to improve New Zealand economy part II:

GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 125

follow-up results. Proceedings of the Twenty-Seventh Annual Hawaii International Confer-ence on System Sciences, 4 (1994), 416–427. Sheffield, J., and Gallupe, R.B. Using groupsupport systems to improve New Zealand economy part II: follow-up results. Journal of Man-agement Information Systems, 11, 3 (1995), 135–153.

37. van den Herik, K.W., and de Vreede, G.J. Experiences with facilitating policy meetingswith group support systems. International Journal of Technology and Management. In press.van den Herik, K.W., and de Vreede, G.J. GSS for cooperative policymaking: no trivial matter.Proceedings of the International ACM SIGGROUO Conference on Supporting Group Work,(1997), 148–157.

38. van Genuchten, M.; Cornelissen, W.; and van Dijk, C. Supporting inspections with anelectronic meeting system. Journal of Management Information Systems, 14, 3 (1998), 165–178.

39. Vician, C.; DeSanctis, G.; Poole, M.S.; and Jackson, B.M. Using group technologies tosupport the design of “lights out” computing systems: a case study. In K.E. Kendall, K. Lyytinen,and J.I. DeGross (eds.), The Impact of Computer Supported Technologies on Information Sys-tems Development. Amsterdam: North Holland, 1992, pp. 151–178.

40. Vogel, D.R., and Nunamaker, J.F. Automated planning support using computers to en-hance group decision making. Administrative Radiology, September (1989), 54–59.

41. de Vreede, G.J. Collaborative business engineering with animated electronic meetings.Journal of Management Information Systems, 14, 3 (1998), 141–164. de Vreede, G.J. GroupModeling for understanding. Journal of Decision Systems, 6, 3 (1997), 197–220. de Vreede,G.J. Participated modeling for understanding: facilitating organizational change with GSS.Proceedings of the Twenty-Ninth Hawaii International Conference on Systems Sciences, 3(1996), 398–407.

42. de Vreede, G.J.; Briggs, R.O.; van Duin, R.; and Enserink, B. Athletics in electronicbrainstorming: asynchronous electronic brainstorming in very large groups. Proceedings ofthe Thirty-Third Hawaii Interantional Conference on Systems Sciences (2000).

43. de Vreede, G.J., and de Bruijn, H. Exploring the boundaries of successful GSS applica-tions: supporting inter-organizational policy networks. DataBase, in press. de Bruijn, H., andde Vreede, G.J. Exploring the boundaries of successful GSS application: supporting inter-organizational policy networks. Proceedings of the Thirty-Second Hawaii International Con-ference on Systems Sciences (1999).

44. de Vreede, G.J., and Dickson, G.W. Using GSS to support designing organizationalprocesses and information systems: an action research study on collaborative business engi-neering. Group Decision and Negotiation, 9, 2 (2000), 161–183. de Vreede, G.J.; Sol, H.G.;and Dickson, G.W. A group problem solving approach to business process redesign: combat-ing organized crime in Amsterdam. Proceedings of the 28th Hawaiian International Confer-ence on Systems Sciences (1995), 493–502. de Vreede, G.J.; den Hengst, S.O.; and Sol, H.G.Facilitating user involvement in information system design and development with GDSS—theorganized crime case. In L. Olfman (ed.), Proceedings of the ACM SIGCPR Conference, Nash-ville Tennessee, 6–8 April, 1995. de Vreede, G.J., and Sol, H.G. GroupSystems V as robocopIV? Combating organized crime with GroupSystems. In Proceedings of the GroupSystems’Fifth Annual Users Conference, 14–16 March, Tucson, AZ, 1994. de Vreede, G.J., and Sol,H.G. Combating organized crime with groupware facilitating user involvement in informationsystem development. In D. Coleman and P.R. Huckle (eds.), Proceedings of the GroupWare’94 Europe Conference, 6–8 June, London, 1994.

45. de Vreede, G.J.; Jones, N.; and Mgaya, R. Exploring the application and acceptance ofgroup support systems in Africa. Journal of Management Information Systems, 15, 3 (1998–1999), 197–224. de Vreede, G.J.; Jones, N.; and Mgaya, R. A new driving force behind capac-ity building in Africa: group support systems. Proceedings of the Thirty-first Hawaii InternationalConference on System Sciences, 6, (1998), 705–714.

46. de Vreede, G.J., and Muller, P. Why some GSS meetings just don’t work: exploringsuccess factors of electronic meetings. Proceedings of the 7th European Conference on Infor-mation Systems, III, (1997), 1266–1285.

47. de Vreede, G.J., and van Wijk, W. A field study into the organizational application ofgroup support systems. Proceedings of the ACM SIGCPR Conference, (1997), 151–159.

126 FJERMESTAD AND HILTZ

48. Walczuch, R.M.; Watson, R.T.; Bostrom, R.P.; and Day, J. Supporting reengineeringusing group support systems: a case study. International Journal of Information Management,15, 2 (1995), 97–114.

49. Zigurs, I.; DeSanctis, G.; and Billingsley, J. Adoption patterns and attitudinal develop-ment in computer-supported meetings: an exploratory study with SAMM. Journal of Manage-ment Information Systems, 7, 4 (1991), 51–70. Zigurs, I.; DeSanctis, G., and Billingsley, J.Exploring attitudinal development in computer-supported groups. Proceedings of the Twenty-Second Hawaii International Conference on System Sciences, 3 (1989), 353–357.

50. Zigurs, I., and Kozar, K.A. An exploratory study of roles in computer-supported groups.MIS Quarterly, 18, 3 (1994), 277–297.

EIES Field Studies

51. Johnson-Lenz, P.; Johnson-Lenz, T; and Hessman, J.F. JEDIC/EIES ComputerConferencing for Standardization Activities. In M.M. Henderson and M.J. MacNaughton (eds.).Electronic Communication: Technology and Impacts. AAAS Selected Symposium 52. Boulder,CO, Westview Press, for the AAAS (1980), 97–102. Johnson-Lenz, P., and Johnson-Lenz, T.(1980). JEDEC/EIES Project: Standardization in Microcomputer/LSI Products via ElectronicInformation Exchange. Final Report to the National Science Foundation. Unpublished.

52. Johnson-Lenz, P., and Johnson-Lenz, T., LegiTEch/EIES: Information Exchange AmongState Legislative Researchers. In M.M. Henderson and M.J. MacNaughton (eds.). ElectronicCommunication: Technology and Impacts. AAAS Selected Symposium 52. Boulder, CO,Westview Press, for the AAAS, 1980, pp. 103–111. Lamont, V.C. Computer conferencing: TheLegiTech Experience. In L.A. Parker and C.H. Olgren (eds). Teleconferencing and InteractiveMedia, 1980, pp. 457–461. Extension Center for Interactive Programs, University of Wiscon-sin, Madison. Stevens. C.H. Many-to-Many Communication Through Inquiry Networking.World Future Soc. Bull., 14 (1980), 31–35.

53. Bernstein, L.M.; Siegel, E.R.; and Goldstein, C.M. The Hepatitis Knowledge Base: APrototype Information Transfer System. Annals of Internal Medicine, 93, 2 (1980) 169–181.Siegel, E.R. Use of Computer Conferencing to Validate and Update NLM’s Hepatitis DataBase. In Henderson, M.M. and MacNaughton, M.J., eds. Electronic Communication: Technol-ogy and Impacts. AAAS Selected Symposium 52. Boulder, CO, Westview Press, for the AAAS,1980, 87–95.

54. IBM Studies:

1. Grohowski, R.; McGoff, C.; Vogel, D.; Martz, B.; and Nunamaker, J. Implement-ing Electronic Meeting Systems at IBM: lessons learned and success factors. MIS Quar-terly, 14, 4 (1990), 369–382.

2. Martz, W.B.; Vogel, D.R.; and Nunamaker, J.F. Electronic meeting systems: re-sults from the field. Decision Support Systems, 8 (1992), 141–158.

3. McGoff, C.; Hunt, A.; Vogel, D.; and Nunamaker, J. IBM’s experiences withGroupSystems. Interfaces. 20, 6 (1990), 39–52.

4. Nunamaker, J.; Vogel, D.; Heminger, A.; Martz, B.; Grohowski, R.; and McGoff,C. Experiences at IBM with group support systems: a field study. Decision SupportSystems, 5 (1989), 183–196. Nunamaker, J.F.; Vogel, D.R.; Heminger, A.; Grohowski,R.; and McGoff, C. Group support systems in practice: experience at IBM, Proceedingsof the Twenty-Second Hawaii International Conference on Systems Sciences, 3 (1989),378–386.

5. Vogel, D.R.; Nunamaker, J.F.; Martz, W.B.; Grohowski, R.; and McGoff, C. Elec-tronic meeting systems experience at IBM. Journal of Management Information Sys-tems, 6, 3 (1990), 25–43.

Appendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology

Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Adelman, 1984 DSS: Decision-Analytic, Case Study Data Military, U.S. Marine 1 group; Alternative design Two 2-day sessions. Level 2, Decision room, Collection: Corps Professionals. analysis, cost benefit Tools: multi-attribute, Cost Comparisons between analysis benefit, Training? non-DSS and DSS Adkins, Sheare, GSS: GroupSystems, Field Study; Data Organization military/ 10 groups; Group Strategic planning 3 Months Nunamaker, Decision room, Level 2, Collection: Post government—Air characteristics: 3 staff Romero, & Anonymity, Facilitator, Tools: questionnaire. Quality Force base in Idaho. groups & 7 squadrons Simcox, 1998 EBS, Topic Commenter, ratings of plans by Size 1000s (large). experimental; 17 Group Outliner, Categorizer; experts. Variables Departments observed squadrons traditional Training not described, measured—quality of staff and operational FtF; subjects per Processes NGT. plan, time, & (“wing command”) group—range not satisfaction with stated Total subjects process, commitment 365, military to plan. personnel. Alavi, 1993 GSS: Vision Quest, Decision Field Study; Data Organization head-23 groups; Group Idea generation and 1 session per group room, Level 2, Anonymity, Collection: post quarters of a Mid-characteristics—varied evaluation (generative) Length of study 3 Facilitator, Leader, Tools: meeting Atlantic Fortune 500 between peer and Tasks from different months electronic brainwriting and questionnaires, company; hierarchical; ad hoc domains (marketing, rating, Training: minimal, for interviews with Departments and permanent project human resources) group leaders only. meeting leaders, observed—various, teams; subjects per analysis of EMS including marketing, group 3 to 15; mean meeting transcripts. personnel, operations size 8.4; total subjects management, and 167; first time users of legal. EMS. Bikson, 1996 GSS: GroupSystems, Level Case Study Data Organization: World 102 GSS sessions. The business needs: Length was not 2, Decision room, Facilitator collection: interviews, Bank. Departments—The participants were decrease decision reported. The GSS and technographer, memoranda, Organizational design, from country time and action cycle sessions extended Anonymity. Tools : EBS, newsletters, session planning and services operations and from times while responding over a 9-month period. Categorizer, Group outliner, logs, reports. (ORG) and information personnel and to increasingly Vote. Training: Yes. technology and administrative complex and urgent facilities (ITF). services. The GSS resource allocation room was used at 50% problems. Meeting of capacity. planning, attitude survey, brainstorming, focus groups. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 127Appendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Briggs, Adkins, Mittleman, Kruse, Miller, & Nunamaker, 1999 128 FJERMESTAD AND HILTZTechnology/Media Used Study Type GSS: GroupSystems, Level 2, Decision room, Facilitator, Leader, Tools: complete set; Training: none required. Theory/Data Collection Methods Action Research Theory: Technology Acceptance Model (TAM). Data collection: testimonies, interviews, observations, and experience. Quasi-experimental Field Study; Tuckman's stages of development: forming, storming, norming, performing and adjourning questions: Meeting sessions recorded and transcribed and coded; Interviews; field experiment 2 (modes) by 2 (problems), by two orders of mode; repeated measures Stage model of Negotiation; Action Research; Data Collection: questionnaires, interviews, direct observation, document collection Case Study Data collection: session logs, questionnaires, interviews. Organizational Context/Nation US Navy fleet command: Strategic and operational decision making. Groups Multiple groups of different size and composition. Task Strategic, tactical and operations planning tasks Sessions Single and multiple sessions over 4 years. Caouette & O'Connor, 1998 GSS: GroupSystems, Decision room, Level 2, Anonymity, Facilitator, Leader, Tools? Training? Organization New York financial guarantee insurance company; Size small (100 employees), 5 years old. 2 groups; 8 subjects per group; total subjects 16; executives One macro (strategic) planning task, 1 micro planning Two half-day sessions, the same day. Carmel, Herniter, & Nunamaker, 1993 GSS:: GroupSystems, Level 2, Decision room, Anonymity, Chauffeured, Leaders, Tools: Win-Win; EBS, Topic commenter, Ranking, group memory; Training: yes; Process: Negotiations Support—Win-Win GSS: GroupSystems, Level 2, Decision room, Anonymity, Tools: EBS, Topic commenter, group writing, Vote, Training: ? Corbitt, Christopolus, & Wright, 2000 Health Center Major issues—Management side: competitiveness, recruitment & retention, malpractice insurance, And focus on the doctors. Union: wages, jobs, focus on medical support staff. Government agency Environmental remediation 2 groups—management and labor Union 11 participants; Management #? Union members had limited computer experience Contract Negotiation task. 13 sessions, 57 total hours over 1 month. 2 groups consisting of production and staff support workers; 6 to 17 subjects per session Business process reengineering 4 sessions once a week for 5 weeks; 2.5 to 7.3 hours in length. Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Davison, 2000 GSS: GroupSystems, Level Case study; Data Academic organization 1 group, 6 subjects, Generating system Email use initially to 2, Decision room, Tools: collection: Session consisting of 41 academic requirements for a derive basic EBS, Topic commenter, logs, questionnaires, members from the bibliographic functionality; GSS vote, Training: Yes. ballots department of information system used to support information systems. detailed idea Hong Kong. generation. 2 GSS sessions, one week apart, 20 and 50 min. Davison & GSS: GroupSystems; Level Action research. Hong Kong One group of 7 Revise the client billing 11 meetings over 6 Vogel 2000 2, Decision room, Facilitator is the Accounting firm headed by the CIO. system months. Anonymity, Tools: Not researcher. Data (medium sized, Subjects in 20s and reported, Training? collection included international) 30s. (Asynchronous email, observation, telephone in between). discussions with participants via FtF, telephone and email; meetings with the leader/CIO; questionnaire after several of the meetings. Dean, Lee, GSS: EMS-IDEF, Level 2, Quasi-experimental Organization—U.S. 8 groups; Subjects per Task: modeling of Session length = Orwig, & Vogel, Decision room, Facilitator, field study. Department of groups Average 9.4 business activities for between one and 3 1994; 1995 Tools: IDEF, EBS, Defense (FtF), 19.7 (GSS); Business Process weeks. Alternative generator, Professionals Reengineering. Training yes, on modeling method and tool, Media processes IDEF modeling method, a structured analysis and design technique. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 129130 FJERMESTAD AND HILTZAppendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Dennis, 1994 Technology/Media Used Study Type GSS: GroupSystems, Level 2, Decision room, Anonymity, Facilitator, Tools: Basic set, Training: No. Theory/Data Collection Methods Case Study design with a semi-Independent variable—meeting style (interactive, supportive, chauffeured, non-GSS) measured as electronic intensity, the higher the score the greater the electronic use. Data collection: Questionnaires. Field Study Comm Mode: GSS (EMS-IDEF), FtF (traditional) Field Study: comparing 9 FtF and 9 GSS treatment groups. Data collection: Interviews and case projects. Field Study. Automated participation logging, post session questionnaire, follow up interview with CEO and 2 managers 3 months later Organizational Context/Nation 5 Organizations US Army, University of Arizona Hughes Aircraft IBM Bell South Groups 10 groups from 5 different companies; 8 to 24 subjects per group; 152 total subjects; professionals Task Real organizational tasks; Judgmental decision making; Type 4 Sessions Multiple sessions ranging from half a day to 14 days. Dennis, Daniels, Hayes, & Nunamaker, 1994 Dennis, Hayes, & Daniels, 1999 GSS: EMS-IDEF, Level 2, Decision room, Facilitator, Tools: IDEF, Training: GSS several hours, FtF several days GSS: GroupSystems, Level 2, Decision room, Anonymity, Facilitator, Tools: IDEF, Training—Yes. GSS: GroupSystems, Level 2 decision room-24 station, Facilitator and 4 assistants, Tools: EBS, Issue Identification and Analysis, Topic Commenter, File Reader and Voting Training? Processes: 3 pre-planning meetings Large Multinational firm DoD projects 23 Groups; 11 and 12 groups per cell; 6 to 60 subjects per group; 283 total subjects; Professionals 18 groups; 9 groups per cell; 6 to 75 subjects per group; 304 total subjects; military officers-majors 1 Group; 26 subjects; Group characteristics) hierarchical, including CEO; subjects per group 31 total subjects 31; senior managers Actual models developed for DOD, Corps of Engineers; Type 1 to 8 Installation management and logistics projects. GSS 4.5 to 13 days; FtF 20 to 100 days. Not reported Dennis, Heminger, Nunamaker, & Vogel, 1990 Organization: Burr Brown, 1500 employees; Manufacturing—electronic parts Strategic planning Six GSS sessions over 3 days;, 3 FtF pre-planning sessions. Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Dennis, GSS EMS, Level 2, Decision Case study with no 5 Organizations Utility 5 groups; 12 to 30 Real business tasks: 1 to 3 sessions for Nunamaker, & room, Anonymity Facilitator, manipulated variables. Company Financial subjects per group; Competitive each group, 1 to 2 day Paranka, 1991 Tools: EBS, Topic The intent was to Corporation 107 total subjects; advantage strategies; sessions each. commenter, Idea organizer, examine strategic Management College Senior managers Type 2, 3, 4, 5, 6 Training? planning & competitive analysis using EMS technology and tools. Dennis, Tryan, GSS: GroupSystems, Level Field Study. Theory: 30 organizations first 30 groups; 11 to 38 73% were involved Sessions went from a Vogel, & 2, Decision room, Venkatraman & time use on strategic subjects per group with strategy half day to 4 days. Nunamaker, Anonymity, Tools: EBS, Ramanujam Model planning. formulation, 37% in 1997 Topic commenter, Issue Data collection: Case Manufacturing, goal formulation, 30% analyzer, Vote, Idea reports, interviews consulting, hospital, in environmental organizer, alternative Measures: GSS university, analysis, and 20% in evaluator, Policy formation, capabilities-Task government, strategy evaluation stakeholder-SIAS. Training? support, process Restaurant, real support, task structure, estate, Utility, bank. process structure DeSanctis, GSS: SAMM, Level 2, Field Study; Theory: Texaco, Oil company 3 groups; 14, 8, 7 Tasks: New Team 1: once a week Poole, Dickson, Decision room, Anonymity, AST; Data collection: Information technology subjects per group; 29 technologies planning, for 2 hours; Team 2 & Jackson, 1993 leader, Tools: EBS, Vote, questionnaires, department total subjects End user support, data once every 3 weeks agenda, Training? observations, center automation; for 1 to 2 hours; Team interviews, video tape Task complexity—3 once a week for 1 High to moderate. hour. DeSanctis, GSS: SAMM, Level 2, Case study with no IRS/NTEU joint quality 10 groups plus others; Real business task: Multiple sessions, Poole, Lewis, & Decision room, Tools: Basic manipulated variables. improvement process group size not Quality improvement; unknown length. Desharnais, set, Training? Assessment of quality reported; total Type 2, 3, 4, 5, 6 1991;1992 improvement subjects-unknown; processes at the IRS. professionals George, GSS: GroupSystems, Level Case study Data Government Indian 2 separate groups: 12 No task was reported. 2 session; 2 different Nunamaker, & 2, Decision room, Tools: collection: interviews. Health Service; Staff dental program The groups used the groups; .length not Valacich, 1992 basic set, Training: Yes. Purpose of system and area meetings. directors and 12 local GSS for a routine reported. was organizational department members meeting innovation. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 131Appendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Genuchten, Cornelissen, & Dijk, 1998 132 FJERMESTAD AND HILTZTechnology/Media Used Study Type GSS: Group Systems, Level 2, Decision room, Facilitator, no Leader, Tools: Categorizer, Training? Theory/Data Collection Methods Organizational Context/Nation Groups 14 groups; 4 subjects per group; professional software engineers. Task Logging meeting for \"Fagan inspections\" of software development documents, detecting defects in code; type: intellective. Case 1: Long term transportation planning. Case 2: Environmental planning, Sessions Not reported Herik & Vreede, 1997, in-press GSS: GroupSystemsV, Level 2, Decision room, Anonymity, Facilitator, Tools: topic commenter, idea organizer for brainstorming, voting, Training: ? Hiltz & Turoff, 1991 CMC: EIES, Level 1 Distributed, Asynchronous, Facilitators, Leaders, Tools: None, Training: 2 days. Also, 3 FtF meetings (pre training and orientation; 3 months interim meeting; final DC meeting. CMC: Email, Level 1, Distributed, Asynchronous, Video conferencing, Tools: none; Training ? Iacono & Weisband, 1997 Case study. Data Organizational Context collection: comments Philips Medical during logging, post Systems and the Baan meeting questionnaire, Co. Netherlands. counts of errors identified or found later. Action Research. Danish Ministry of Methods: participant Housing, Spatial survey, interviews with Planning and the session initiator and Environment (VROM) selected other participants; participant observation as facilitator; system logs. Case Study: Mailed Senior executives and pre & post researchers questionnaires, interim online questionnaire, participant observation, system usage statistics. Theories: Critical Success Factors and Media Richness Case study Data Collection: 1. transcripts content analyzed 2. projects graded for whole group 3. individuals graded for frequency, number and quality of email posts 4. Survey questionnaires pre task Analyses were categorized by performance on a group project Organizations: 3 U.S. universities Case 1: Ruimpad, 6 from VROM + 3 from traffic board; had been working together for 1.5 years. Case 2: 50 people involved in heterogeneous groups of 11—13. 168 participants in 7 groups of 21—32 members. Case 1: Multiple sessions, length not reported; Case 2: Multiple sessions over 3 weeks. Formulate and reach consensus on private sector recommendations for the White House Conference On Productivity. 4 months 14 groups; 3 subjects per group; 42 total subjects; Graduates and undergraduates. task: week one: introduce themselves. Week 2: Select a project topic in social issues of computing and organize themselves. Week 3: research, write, and present a 5 page policy paper. Type 4,6 3 week asynchronous session Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Jarvenpaa, & CMC: Email, Level 1, Field study: Theories: “Virtual teams,” as Teams of 4 to 6 in 1—exchange Two one week warm-Leidner, 1998 Asynchronous + TIP (McGrath); course assignment for which each member information about up/practices sessions; Synchronous CMC, using Cultural values IS graduate courses, resided in a different themselves;;2, gain One four week listserver email and personal (Hofstede); SIDE including 350 masters country; exercise experience with the asynchronous session. email, Tools: none, Training: (Social students from 29 counted at least 20% World wide web; 3. none. No assigned leader, Identification/Deindividuniversities who of course grade. 75 Propose and develop emergent leaders. uation theory, (Lea participated over 6 teams started; 29 had a Web site providing a &Spears 1992) Data: weeks in 1996. 2 or members who new service for IS group email archives, Students receive completed both World Net. 2 online course credit; best questionnaires. These questionnaires; team wins $600 29 were categorized Content analysis of 12 as HI or LO on trust teams. before and after; the 3 most extreme teams in each cell were chosen for content analysis. Kock, 1998 CMC: Groupwise—Novell, Action Research Data University process 5 groups; 7 to 13 Task: process Asynchronous and FtF Level 1, Self appointed collection: structured improvement groups subjects per group; 50 improvement. over 7 months. leader, Tools: none—and unstructured from 15 different total subjects; attachments, Training ? interviews, and departments. New professionals—faculty transcripts from Zealand and staff. electronic postings. Kock & CMC: Groupwise—Novell Action Research Two New Zealand 7 groups; subjects per Task: process Length of study/ group McQueen, 1998 Level 1. Anonymity, participant Organizations 1. a group 5 to 15: 64 total improvement (3 interaction 14 to 33 Facilitator, Leader, Tools, observation, school of studies of a subjects; stages: definition, days; total observation Training ? unstructured and university; 2. Ministry professionals. All analysis and redesign) 2 to 6 months structured interviews, of agriculture and subjects had prior FtF and transcripts of the fisheries university, participation in BPR email discussions, government Depts. meetings. analysis of observed—2 university organizational depts.; MQM, 18 documents. management of quality depts. in agriculture organization. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 133Appendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Krcmar, Lewe, & Schwabe, 1994 134 FJERMESTAD AND HILTZTechnology/Media Used Study Type GSS: GroupSystems, Decision room, Anonymity, Facilitator, Tools: EBS, Outliner, idea organization, idea evaluation, voting, alternative evaluation, Training ? GSS: Facilitator, Decision room, Level 2, Anonymity, Facilitator: Technical consultant, Tools: NGT, Idea generation, ranking, cross impact analysis, Training: Yes—4 hours. GSS: Decision Conferencing, Level 2, Decision room, Anonymity, Facilitator: 3 facilitators—process facilitator, analyst to support computer modeling, and “correspondent” who documents discussion and process, no Leader, Tools: computer model, Training? CMC: Teammate, Level: 1 Anonymity—No, Facilitator-No, Tools: none Training: yes, extensive Theory/Data Collection Methods Case Study Data collection: questionnaires, interviews, system monitoring Case study. Data Collection: questionnaires Organizational Context/Nation Social club, Business BPR, and an academic group Germany Groups 8 to 15 subjects per meeting Task Case 1:Social club to allocate funds to worthwhile people; Case 2: BPR; Case 3: Academic publishing Economic development of the region Sessions 1 to 8 hours in one day. Up to 3 days of sessions Lewis, Keleman, & Garcia, 1990 Organization: Whatcom County, Washington; A project consisting of 22 local public and not for profit organizations. Organization: New York State Government Departments 3 groups; 5 subjects per group; 15 total subjects; Type: mid career professionals Number of sessions: 2 Session length: 4 hrs. McCart & Rohrbaugh, 1989 Case study. Theory: T. Parsons AGIL four functions theory of social systems; “Competing Values Approach” Data collection: Self completion questionnaire Case Study. Data collection: internal and external, interviews, transcripts. 14 groups; chosen from those hosted over the 1982-1985 period; Group Size varied; professionals. Task: Planning Original meetings 2 days; this study a follow up years later Markus, 1992 Morales, Moreira, & Vogel, 1995 GSS: GroupSystems, Level 2, Decision room, Anonymity, Facilitator, Tools: Basic set, Training: ? Case Study; Theory-AST; Data collection: Data: post questionnaires, facilitator observations. Student teams produce a report which describes a strategic problem or opportunity facing a company, an analysis of it, and recommendations Development planning groups in Mexico, held at the Monterrey Institute of Technology. 4 groups; 3 to 4 subjects per group; 15 total subjects;: MBA students Students do a strategic consultation for a company. Asynchronous and synchronous communication over 6 months. Mexicans; 293 senior business and government officials; N of groups not specified. One group was size 18 Regional development project—Strategic. Session length 2hours and 15 minutes. Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Muller & Vreede, GSS: GroupSystems, Level Case study utilizing Organization: PIT Marketers & Strategic planning: 5 sessions; no other 1995; 1995 2, Decision room, Co-operative computer Telecom; Internal engineers, (product \"added value information available. Anonymity, Tools: Idea augmented concept departments: developers), services:\" organizer, Vote, Group development method; marketing, sales, R & consumers, & PIT dictionary, Group matrix, Data collection: D; external: employees Training ? questionnaires. customers, users, dealers, etc. Netherlands Niederman & GSS: SAMM, Level 2, Single Case A large operating 1 group; 6 to 8 Problem formulation 6 sessions; the first 2 Bryson, 1998 Decision room, Facilitation, experiment Data foundation which members; tasks sessions used FtF, the Tools: idea generation, idea collection: provides management professionals. next 4 used GSS. One evaluation, group questionnaires, consulting to non-profit session a month messaging, Training: Yes communication logs, organizations. and interviews Nunamaker, GSS: Early GroupSystems, Case studies: Data Organizations: 7 groups; 6 to 22, Strategic planning 3 sessions per group Applegate & Level 2, Decision room, collection: Government, average of15 subjects over 3.5 months. Konsynski, Anonymity, Tools: EBS, Idea Questionnaires, Computer per group; 106 total Average session 1987;1987 structuring & analysis, vote; observations. manufacturing, State subjects; high level length 4 hours. Training? University managers Post, 1993; GSS: TeamFocus, Level 2, Case study. The intent Major manufacturing 64 groups; average Real business tasks: 64 sessions; several 1992 Decision room, Facilitator, was to collect corporation group size 10.2; 654 Planning, strategy, are multiple sessions. Anonymity, Tools: basic set, effectiveness and total subjects; require definition; Type Average time 4.7 Training: yes. efficiency data to Professional. 2,3,4,5,6 hours. evaluate business cases. Quaddus, GSS: Single PC work Case study: Action Independent Living 1 group; 11 subjects; Strategic planning—Preconference Atkinson, & station, Level 2, Decision research type—Center, Western top executives future direction and preparation session; Levy, 1992 room, Facilitator, Analysts, participants evaluate Australia implementation; Type two day session. Tools: MADM, Training: On current situation and 5. tool. future vision. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 135Appendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Sheffield & Gallupe. 1993;1994; 1995; 1995 136 FJERMESTAD AND HILTZTechnology/Media Used Study Type GSS GroupSystems. Level 2, Decision room, Anonymity, Facilitator, Tools: Topic commenter, vote; Training: ? Theory/Data Collection Methods Case Study. The intent was to collect data on several economic policy making meetings. Implementation activities from follow-up interviews are also included. Research model: Input-processes-outputs. Case study. Data collection: interviews, observations, session logs, questionnaires. Case study. Data collection: ideas generated, comments Organizational Context/Nation Organization: New Zealand's Competitive advantage project Groups 12 groups; 14 to 33 subjects per group; 254 total subjects; Professionals Task New Zealand's economy; Mixed motive; Type 5 Sessions 5 sessions, 30 minutes to 3 hours for each session in one 8 hour day. Vician, DeSanctis, Poole, & Jackson, 1992 GSS: SAMM, level 2, Decision room, Anonymity, Leader, Tools: agenda, idea gathering, idea evaluation, decision models, communication, Training: ? GSS: PlexCenter, Level 2, Decision room, Facilitator, Anonymity, Tools: EBS, Issue analyzer, Topic ranking, SIAS, Training: 15 min. GSS: GroupSystems, Level 2, Decision room, TeamGraphics; Arena simulation language Anonymity, Facilitator: Tools: Topic Commenter, idea organizer, group outliner; Training: ? GSS: GroupSystems, Level 2, Decision room, Facilitator, Tools: EBS, Categorizer, Training ? Texaco, Inc, a Fortune 50 company. An automation team with corporate ITD. This was part of an overall TQM project. Health care group. 14 teams consisting of 5 members Tactical: implement automation tools in the corporation's data centers. Weekly meeting over 15 months. Vogel & Nunamaker, 1989 13 management & administrative & CEO Strategic Planning: systems and processes open to the organization. Strategic and organizational change (BPR) 2 sessions; 3.5 hours each one week apart. Vreede, 1996; 1998 Action Research. Prototype, surveys Government organization: Criminal Investigation department (CID) of the Amsterdam Municipal Police Force. Netherlands. Construction project: choice between two designs. Netherlands. 1 Group; Subjects per session—4,4,8,12,12 Total subjects: 27; unique stakeholders; Professionals; 40 participants 10 groups; 100 participants 5 sessions; 2 at 3 hours and 3 at 6 hours Vreede, Briggs, Duin, & Enserink, 2000 Theory: Brainstorming productivity: blank slate (decathlon) versus sequential ideas (relay). Field study Questionnaires, session logs, Strategic and tactical planning. 10 sessions, length not reported Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Vreede & Bruijn, GSS: GroupSystems, Level Action research Data Multiple organizations 9 groups; 4 to 15 Strategic and 1 to 4 sessions; one in-Press; Bruijn 2, Decision room, Facilitator, collection: Netherlands subjects per group; 61 operational tasks. half day to 1 day in & Vreede, 1999 Anonymity, Tools: Basic set, questionnaires, total subjects; length. Training? interviews and Professionals observations. Vreede & GSS: GroupSystems, Level Action research Data Amsterdam Municipal 8—14 subjects Collaborative business 7 sessions, seven Dickson, 2000 2, Decision room, collection: session Police Force, representing the engineering: develop hours each over 3 Anonymity, Tools: EBS, idea logs, questionnaires, Netherlands. information the approach to months. The entire organization, Vote, and interviews department reorganize the project took one year. Alternative evaluation. Criminal investigations Several development Training ? department to better sessions were also deal with organized used. crime. Vreede, Jones, GSS: GroupSystems, Level Technology Culture—East and 11 groups Participants Idea generation and 11 meetings over two & Mgaya, 1998; 2, Decision room, Tools: Acceptance Model and South Africa, former included stakeholders evaluation on years, one per group. 1998-1999. categorizer, vote, Training: ? interpretive/grounded British colonies from a variety of “capacity building” Some were 2-3 days GSS were usually portable, theory; Field Studies. Malawi, Zimbabwe, governmental bodies, participatory long. Often shared machines Method—qualitative + and Tanzania. civilians, and other development projects, quantitative, including relevant sectors (e.g., including open and semi-World bank). environmental structured interviews, planning, new logs of meetings, computer system observations during planning, World Bank the meetings and projects, etc. Treated interactions with strategic and stakeholders; post operational issues. meeting satisfaction questionnaires in Tanzania. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 137138 FJERMESTAD AND HILTZAppendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Vreede & Muller, 1997 Technology/Media Used Study Type GSS: GroupSystems, Level 2, Decision room, Anonymity, Facilitator, Tools: not reported, Training:? Theory/Data Collection Methods Case Study Three cases selected as having 'bad\" (low success) meetings. Data collection: Questionnaires, session logs, and interviews. Organizational Context/Nation Three cases: 3DOME—a consortium from academia and business; The Vice Case—a multi disciplinary police team; The Climate—a diverse group of industry & government representatives Netherlands National Nederlanden Insurance, a large organization. Netherlands. Groups 3Dome—5 to 9 subjects; Vice—8 subjects; Climate—10 subjects Task 3Dome—Internet service; Vice—municipal policy; Climate—Climate control Sessions 3Dome—6 meetings; Vice—1 meeting; Climate—2 meetings; Session length not reported. Vreede & Wijk 1997 GSS: GroupSystems, Level 2, Decision room, Facilitator, Tools: Not reported, Training? Walczuch, Watson, Bostrom, & Day, 1995 GSS: Vision Quest, Level 2, Decision room, Anonymity, Facilitator, Tools: EBS and voting, and PRISM, a single user implementation of Interpretive Structural Modeling Training—practice (\"fun\") problem Action/field research. Surveys; interviews with meeting initiator and some participants; expert estimation (e.g., of hours that would have been required w/o GSS); direct observation; system logs. Data were collected before, immediately after, and well after the session. Case study, Data collection: Questionnaires, Observation, Nine structured follow up interviews, including repeat of pre-meeting Cohesion Questionnaire Subjects: total of 91 employees or independent agents; mean group size 10.1. Mostly first time users of GSS. Variety involving medium to high structure, high importance 9 single sessions, 2 in decision room, 7 in portable condition Organizational Context U. of Georgia Housing Dept., Size professional staff of 22 1 group; 14 subjects; managers and professionals; from two different divisions of the housing organization Task: identify problems with organizational processes and find process improvements 3 meetings over 3 months Technology/Media Used Theory/Data Organizational Authors Study Type Collection Methods Context/Nation Groups Task Sessions Zigurs, GSS: SAMM, Level 1 Case study. The intent Communication 8 groups; 4 or 5 The University Goal 8 two hour sessions DeSanctis, & Decision room, Anonymity, was to observe and course assignment subjects per group. 37 Task as practice; The over two months on Billingsley, 1991 Tools: Ranking, Voting; measure system total subjects. Foundation Task, two tasks Training: Practice problem. usage. Undergraduates in a Strategic planning; communication Type 4 course. Zigurs & Kozar, GSS: TeamFocus, Level 2, Case study. The intent Organization IBM in 10 groups; 7 to 15 A \"real\" organization 1 session, length not 1994 Decision room, Tools: EBS, was to measure and Boulder, Colorado subjects per group; task; Judgmental reported. voting, topic Commenter, observe the GSS 100 total subjects; decision making; Type policy formation, Idea environment on role professional 4. organizer; Training? perceptions of initiators and participants. EIES Field The original EIES field trials, sponsored by the National Science Foundation, followed eight scientific research communities through two years of system use. Most of the Trials communities did not have specific tasks that they set out to accomplish, but rather were devoted to the discussion of theoretical and methodological issues in their disciplines. The three online communities that had a specific task are included here. Johnson-Lenz, CMC: EIES, Level 2, Field Study: JEDEC Study: 77 total invited Design and agree on 20 months duration; Johnson-Lenz, Distributed, Asynchronous, Observation and Representatives of members in 8 new product standards mean of 74 sessions & Hessman, Facilitator/leaders, Tools: online Baseline and electronics industry overlapping activities; to be recommended totaling 22 hours 1980 DSS, Terms, and voting, follow-up companies 58 participated for official adoption by online. Training: Yes, Quarterly face questionnaires, ballot. to face meetings. interviews; system use statistics Johnson-Lenz, CMC: EIES, Level 2, Field study: LegTech-EIES Study: One, 24 in initial Provide scientific Participants involved Johnson-Lenz, Distributed, Asynchronous, Observation and Government and Not period; later expanded information to state for 6—18 months at 1980; Lamont, Facilitator, Tools: Special questionnaires for Profits to 80. State legislative legislators, on request. time of study. 1980; Stevens, Topics templates and roles scientific advisors and 1980 structures to limit, filter and liaisons from scientific order communication, societies. Training?: GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 139140 FJERMESTAD AND HILTZAppendix 2: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Methodology (Continued)

Authors Bernstein, Siegel, & Goldstein, 1980; Siegel, 1980 Technology/Media Used Study Type CMC EIES; Level 2, asynchronous, Facilitator. Tools: tailored “Terms” software to search and to vote, with simplified interface Theory/Data Collection Methods Field study: observation and questionnaires Organizational Context/Nation NLM-Hepatitis Study: Medical researchers, working on National Library of Medicine project Groups One, ten M.D. experts plus facilitator and NLM staff Task Update and validate by consensus, contents of NLM database (“knowledge bank”) on viral hepatitis, including reviews of 850 new papers. Manufacturing tasks (shop floor control) Sessions 7 months. Hours online varied from 4.5 to 66. IBM Studies Grobowski, McGoff, Vogel, Martz, & Nunamaker, 1990 McGoff, Hunt, Vogel, & Nunamaker, 1990 Martz, Vogel, & Nunamaker, 1992 Nunamaker, Vogel, Heminger, Grobowski, & McGoff, 1989 Vogel, Nunamaker, Martz, Groboski, & McGoff, 1990 GSS: GroupSystems, Level 2, Decision room, Facilitator, Anonymity, Tools: EBS, Issue analyzer, Vote, Policy formation, Training Field study Data Collection: questionnaires, interviews, observations, comparisons of projected and actual man hours. Organization: IBM manufacturing plant Size: 6,000 employees; upstate New York 59 cases; 8.27 subjects per session Total subjects: 490 Professional Number of sessions: 199 session hours Session length: avg 3.37 hours; Number of sessions: 29. Appendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results

Author Results Comments Conclusions Adelman, 1984 The DSS facilitated: The results suggest that the group based DSS aided in improving effectiveness, and 1. by allowing normative approaches to decision making efficiency of the group. The task was implemented one week after the second set of 2. permitting real time sensitivity analysis meetings. The DSS tools improved discussions by letting the decision makers focus on 3. providing a mechanism for group discussion the model and not each other, resulting in better discussion. Adkins, Sheare, GSS supported strategic planning improved the quality of the plans, GSS allows a much larger number of GSS technology improved the quality of Nunamaker, reduced time (8 vs. 17.7 hours) , and increased satisfaction with the personnel to be actively involved in the the strategic plans, reduced the time to Romero, & process. There was no significant increase in the level of planning process and thus results in a completion, and increased the process Simcox, 1998 commitment to implementation between the GSS and non-GSS higher quality product. GSS groups also satisfaction. groups. had a greater number of ideas generated than did non-GSS groups Alavi, 1993 Idea generation time: GSS took less A substantial level of discussion and The GSS users felt that relative to Number of ideas: GSS more information sharing was observed. In traditional FtF meetings, GSS leads to the Idea quality: GSS better addition to the exchange of ideas and generation of more high quality ideas, in a Prioritizing time GSS took less information electronically, a relatively high shorter time period. The GSS participants Comfort participating: GSS better level of verbal communications took place. were also more satisfied with the meeting Stress: GSS less stress The electronic channels seemed to process and had a higher level of Usability: GSS better augment and stimulate verbal discussion, cohesiveness. Use the GSS again: GSS better rather than replacing it. Participants said Cohesiveness: GSS better they felt more comfortable in offering their The group worked well: GSS better ideas and perceived less stress during The results are evaluated in comparison to a traditional meeting. GSS than in FtF meetings. Bikson, 1996 1. Meeting objectives were met A high level champion was not part of the GSS meetings were vastly more efficient 2. A facilitator is important implementation- it came from the ORG than non-supported meetings and improve 3. The participants believed they had more impact on the department. Repeated business the quality of the results. Positive outcomes compared to other methods accounted for a significant demand for the outcomes result from pre-meeting planning 4. Improved knowledge, knowledge sharing, and understanding GSS room. and training, a good meeting plan and well 5. Increased participation The GSS had limited used in decision defined objectives, and a facilitator. 6. Anonymity- positive effect on divergent thinking: broadens making due to: the culture at the World participation Bank; very complex analysis and decision 7. Parallel input was an important feature making processes at the Bank; They 8. GSS sessions save time, add to comprehensiveness, suggested spaced rather than massed accuracy, and quality sessions to avoid potential buy-in regret. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 141142 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Briggs, Adkins, Mittleman, Kruse, Miller, & Nunamaker, 1999 Caouette & O'Connor, 1998 Lessons Learned 1. Infrequent ad-hoc use hinders self-sustenance 2. Find a specific, repeated process to support with GSS 3. For continued ad-hoc use a GSS specialist is required 4. Keep the process simple Team A was committed to the task, and started with GSS; its members were very positive about the value of GSS. In the afternoon, without GSS, it somewhat fell apart. Team B started manually. Its leader did not understand the technology or her role. Team B did not perceive any added value from the system. 1. Do the stages occur in the same order and intensity when teams are augmented with GSS? Yes, the GSS can help a group get started. 2. Does GSS allow teams to handle conflict better? Yes, GSS reduces conflict time. 3. How do group characteristics impact team development, with and without GSS? The groups developed differently. Leadership is an issue. 4. Does task complexity interact with perceptions of the value of GSS? Yes. System usage total 23% (16% in EBS and 7% in Contract Log); 77% of the time the two groups used verbal communication. There is evidence that the GSS enriched the communication process and cooperative work. Comments Pre-meeting plans are vital; Compare GSS results to alternatives; Typing does not interfere with the results; Usefulness is important, but pay attention to: social, political, physical and cognitive factors; Usefulness varies with time pressure. \"Effective decision making may be dependent on a group's ability to work as a team. Therefore, the process that groups go through in becoming a productive, functional team is an important issue to explore in organizations that rely on high-level GSS as a group communication tool.\" GSS can help a group get started (forming), but only when the group considers the task to be solved important. When any team gets off to a good start, good work begets more good work (performing). The sessions were categorized into three stages: strategy, Issues, and bargaining. The Union and management teams each had separate strategy meetings. Conclusions The results suggest that ad-hoc problem solving hinders the transition process. Furthermore, GSS sustainability requires structured repeatable processes. Longer term aim is to have distributed portable use. GSS reduced the amount of time and the number of times that teams were in conflict (storming). Teams using GSS spent more time ensuring that they were together on the task (norming). Closure was also more apparent (adjourning). The two teams developed quite differently and GSS impacted all stages, most noticeably the storming stage. The commitment of the teams to the task, group composition and leadership were identified as moderating factors. Carmel, Herniter, & Nunamaker, 1993 The results of the case study suggest that the four components (GSS, mediators, structured process, and setting) contributed toward the contract negotiation. No single component dominated. Author Results Comments Conclusions Corbitt, 1. Accomplished more with a GSS than FtF methods: Yes The authors suggest that the GSS tools The results suggest that by using a GSS to Christopolus, & 2. GSS is effective for BPR make the BPR activities effective and support BPR the design is accomplished Wright, 2000 3. Honesty, commitment, openness improved across sessions successful. more effectively and in a shorter period of 4. GSS increases the number of ideas time than with previous methods. 5. Computer confidence improves across sessions 6. Improved efficiency. Davison, 2000 1. 54 comments on 7 topics after session 1 The use of Email to collect and System failure due to: lack of an 2. 101 comments on 2 topics after session 2 communicate ideas worked well. authoritative manager; an extremely 3. Increased participation The project was abandoned after two GSS complex task. It was also suggested that sessions. Considerable task focus the lack of an interdisciplinary team problems were encountered (including a librarian) may have helped. Davison & Vogel 1. Participants not highly motivated (they could not charge their Participants challenged the anonymity of The GSS functioned as a team memory. 2000 time to any project or client). the GSS; were afraid of loss of “Culture of cautiousness hampered the 2. The participants were not interested in the task. confidentiality. . CIO “misappropriated” it, situation; female members in particular 3. Anonymity was used incorrectly. using it as one form of dominating were perceived as not willing to contribute 4. Facilitator and CIO were at odds. proceedings, entering many ideas and ideas if they were unsure of their accuracy. 5. Team members perceived system as good for idea generation, “sometimes submitted wild or provocative ‘GSS can be effectively used for but GSS not so good for developing consensus or deciding ideas so as to see what he could get away longitudinal meeting contexts so long as upon fine points of detail with.” Facilitator/researcher had conflicts the GSS facilitator employs the technology 6. GSS did exert a positive influence on the meeting process. with the CIO over meeting dominance. flexibly. Dean, Lee, 1. What features of group modeling are important to effectively This paper talks about the evolution of the tool as a result of the evaluation of the field Orwig, & Vogel, support group modeling meetings? EMS supported groups trials. 1994; 1995 produced 251% more activities and 175% more ICOMS per The EMS supported modeling appears to increase the number of subject matter experts day. Individuals in EMS supported groups were also more who can be directly involved, and allows models to be built significantly faster because of productive. parallel contributions and increased efficiency. The evidence suggests that this results in 2. Can GSS modeling tools and methods allow a larger number more extensive and complete model descriptions. On a number of style indicators, MMS of participants to contribute resulting in faster and more based models are comparable but there are a few areas where manually supported efficient model development? A greater number of participants models scored higher on style and semantic quality, particularly in model integration. By do get involved. decreasing the time needed, EMS supported modeling also reduces the cost required to 3. How does the quality of structured models developed with develop business models. Larger number of active participants should improve “buy in” GSS-IDEF compare to traditional meeting support? Quality and perceived legitimacy of the model and thus help in using them as vehicles for measures: generally not significantly different. implementing improvements. Dennis, 1994 Method The results suggest for larger groups The greater the use of electronic Perceived effectiveness: GSS > FtF performing idea generation tasks, communication as a meeting component Perceived satisfaction: GSS > FtF meetings with a greater proportion of the more effective, efficient and satisfying Perceived efficiency: GSS > FtF electronic communication were perceived it is. to be more effective, efficient, and satisfying. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 143144 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Dennis, Daniels, Hayes, & Nunamaker, 1994; 1994 Comm Mode Productivity: GSS > FtF Number of ICOMs: GSS > FtF Development time: FtF > GSS Content quality: Ns Style quality: Ns Comments Conclusions Dennis, Hayes, & Daniels, 1999 Efficiency: GSS > FtF (took less time) Quality: No significant differences Perceived Quality: GSS > FtF Dennis, Heminger, Nunamaker, & Vogel, 1990 Dennis, Nunamaker, & Paranka, 1991 1.Can GSS support planning by a large group? Yes 2. Can GSS provide both structure and flexibility? Yes 3. Can GSS process be efficient? Yes 4. Can GSS be effective? Yes 5. Can GSS be satisfying to the participants? Yes 22 of 26 believed that the automated process was better than a manual one. Flexibility was introduced to change the agenda and tools as needed. Participants generally found the process effective, citing anonymity, which \"allowed people to ask questions that would not have been asked if names were tagged to the questions\participation among many more participants than would otherwise have been possible. All participants were in the neutral to agree range on whether the EMS supported process is better than the manual process. Observation and survey results. GSS meetings are better than FtF – Yes; GSS helps in idea generation over FtF- Yes; GSS more efficient than FtF- Yes; GSS has higher satisfaction than FtF- Yes. GSS groups averaged 17 subjects per The use of GSS techniques and processes group versus 8 for the FtF groups. The can significantly decrease the time, and traditional IDEF models took 6 weeks to thus the cost of complex actual IDEF complete versus 1 week for the GSS models. GSS-IDEF participants spend groups. Both methods used facilitators; more time concerned with content than on the GSS facilitators did not act as an model appearance. intermediary. The EBS portion of the process enabled all participants to contribute equally. Groups using GSS-based modeling processes and tools develop models 4.5 times faster than groups using traditional process and tools. Traditional models take a median of 6 weeks to complete versus one week for GSS groups. The GSS modeling tool and process enables the groups to better integrate the model components among the subgroups. Parallelism of the GSS provided the group members the opportunity to propose changes to the model. The GSS groups also did not waste time discussing minor issues. Initial comments are entered quickly This study shows that GDSS can be (about 12 comments per minute during successful for large groups. It successfully first 10 minutes of an hour session), provided both structure and flexibility; was dropping to 4 effective and efficient, and satisfying to comments per minute. users. The company elected to use the system again in subsequent years. Groups seem to require that there is a sense of commitment towards the outcomes of the meeting. The results suggest that EMS has the ability to help in the rapid collection of ideas. The structuring techniques both for conduct of the meeting and the analysis of the issue contribute to the success of the meeting. Author Results Comments Conclusions Dennis, Tryan, Information Production All but one organization used process The results suggest that organizations that Vogel, & & Identification (IPI) support tools. Anonymity was seen as the make extensive use of process support Nunamaker, Process support- positive most important by 14/30 of the (parallelism, anonymity, & group memory) 1997 Process structure- positive organizations. The participants suggested rather than chauffeured show improved IPI Task support- No effect that anonymity improved the quality of the and ICI. Organizations that made more Task structure- No effect meeting. 17 organizations stated that use of process structures (i.e. meeting Information Communication & Integration (ICI) process structure (role of facilitator) was agenda) improved IPI. Organizations that Process support- positive important. 20 organizations used task made greater use of task structures Process structure- Marginal positive structure tools (Topic Commenter). Task improved ICI and FL. Task support- No effect support tools (external databases, etc.) Task structure- Positive were not used very much. Many Flexibility & Leadership (FL) organizations brought in their reports. Process support- No effect Process structure- No effect Task support- No effect Task structure- Positive DeSanctis, Team 1 Team 2 Team 3 All of the teams used the GSS and used it primarily for task purposes. The teams used Poole, Dickson, Inputs: good task tech fit Low task tech fit Low fit the GSS to display agendas, record information and these tools provided continuity from & Jackson, 1993 Openness to use SAMM one meeting to the next. Group Process interaction: Appropriation- high variable high Decision quality was improved when the GSS was used. Appropriations varied across Distribution uneven uneven uneven the teams and were dictated by the nature of each team's task and commitment to TQM. High use of EBS Contingency theory would predict that task-tech fit accounts for GSS effectiveness. Used low level communication However, team 3 had the lowest Task-tech fit and the highest success with the GSS. Low participation low high Outcomes: Efficiency- mixed low moderate Effectiveness- Good some gains good to high Consensus- poor Comfort- good GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 145146 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results DeSanctis, 1. Poole, Lewis, & Desharnais, 1991; 1992 2. 3. 4. 5. 6. George, Nunamaker, & Valacich, 1992 1. 2. Extent of use: Teams who started using the GSS early were more likely to continue to use the system; Meetings per month: 20.6. Types of features used: ; Level 1 and Level 2 technology was used; Facilitators accompanied most sessions; Level 2 technology was used more than level 1. Satisfaction with SAMM: Comfort & enjoyment; Provide the right support Initiation of use: Group leaders and outside facilitators Instrumental use: SAMM primarily was used for task purposes. Use sentiments: There was a balance between positive and negative sentiment towards SAMM. The GSS helped to increase participation levels, because it allowed the group to work better together. The GSS improved the outcome. Comments Patterns of use varied. Infrequent users had lower satisfaction and comfort than more frequent self-initiating user. Conclusions Over the seven month period of time system use was relatively high, primarily for task and process related activities. The initiation to use SAMM came mostly from the group members themselves. Self reports of satisfaction with the technology were high. This paper is an early case study using GSS technology. The authors discuss what it took to implement the GSS and that was a strong champion and no cost. The results suggest that this was a success because the systems was adopted and implemented for a short time and later returned. Key factors were: lack of use, lack of a continued champion Author Results Comments Conclusions Genuchten, Dependent measures ( Descriptive or relationship results) It was found that the group resented EMS support of software defects logging Cornelissen, & Effectiveness is number of defects found per inspected page. spending time as each person typed in meetings can improve the effectiveness Dijk, 1998 Meeting effectiveness is the number of (additional) defects found per their individual errors identified. From the and efficiency of the process. page during the logging meeting. The effectiveness of the EMS second meeting on, they sent their The fact that inspections are routine supported inspections is considerably higher than for traditional individual defect list in ahead of time, and meetings with a clear structure meetings. The preparation effectiveness was also higher- it was entered into the system before the differentiates them from the typical EMS preparation effort was higher when list had to be submitted in EMS began. The meeting was meeting. This imposes new requirements advance. Ratio of preparation to meeting effectiveness was much considered less stressful than traditional such as; facilitator is superfluous; higher for EMS meetings (e.g., 13 vs. 3 to 1 ratios). Efficiency is the logging meetings. participants know what to do. Also, such number of defects found per person hour invested in the inspection. small frequent meetings cannot afford a Efficiency much higher with EMS. A Fagan inspection is a structured review separate facilitator. Also setup time for aimed at detecting defects in development tools for EMS has to be minimal for routine Yield is the % of defects found during the inspections vs. the % that documents or code. It consists of meetings. Less favorable reactions of slipped through and were detected during later tests or in the field . individual preparation in which each subjects than in previous lab experiments However, subjective opinions of the EMS were less favorable than in participant independently lists apparent with EMS may be due to the fact that previous lab studies. e.g., mean of 3.7 out of 5.0 for \"satisfied.\" defects; a logging meeting for group to inspections are already a very structured merge, compare, and add to list, then process; in other kinds of meetings, the fixing of the documents. The effectiveness EMS often brings the benefit of improved and efficiency of the logging meeting are preparation and structure. An inspection typically low compared to the individual as a structured process could benefit from preparation, when logging meetings are fixed-format input fields incorporated into traditional FtF. the EMS, rather than free form input. ** Herik & Case 1: the project organization was too complex. Voting on the Policy making is defined as “the process of constructing a politically supported plan to Vreede, in press items did not seem to really build consensus on what is important. achieve desired societal change.” The participants were very dissatisfied with the available time for Anonymity seemed to hinder discussion in this long-term group. discussion. “We have generated more information, but it is considered less carefully.” But, the final report presented two years “Group support for policy making is a difficult endeavor.” The process of idea generation, later, did include the models that were used in the GSS sessions. visual modeling, and the availability of anonymity appear to be highly successful in a multiple stakeholders environment, whereas a lack of expression (media richness) the Case 2: GSS was felt to enhance the efficiency and creativity of the voting facilities and reaching consensus when views conflict, are evaluated less meeting, but was felt to be unfit to support discussion, either favorably. Longitudinal research reporting long term impacts is necessary. electronically or verbally. Satisfaction with using GSS= 4.1 /5, but Policy agendas should contain both divergent and convergent activities. GSS, though support of communication only 2.9/ 5. Opinions on the use of suitable for ‘quick and dirty’ idea generation, is less useful for the in-depth discussion anonymity were mixed but generally positive. Six months later, this required for complex policy debate. Voting tools do not generate consensus in project was folded into another one. heterogeneous groups. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 147Appendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Hiltz & Turoff, 1991 All groups scored well on “critical success factors” and all completed interim and final reports on time. Participants felt satisfied with the system (perceived it as stimulating and enjoyable); that the quality of the output was high, and that they saved time compared to what would have been required to produce the reports and recommendations otherwise. Team performance: 6 high performing and 8 low performing teams. Diversity of teams unrelated to success. Access: Having 2 or more members with a modem at home significantly related to report quality (p<.05). Ability (GPA) and quality: no relationship. Age and quality: high performing teams older (26.3 vs. 23.8, p < 05). Previous experience: no relationship. Both Work process and Work Content initiations and responses positively correlated with team performance; as were total initiations and responses. High performing teams had more \"fun\" interactions. Peaks of interaction driven by project deadlines for each phase. High performing teams had more early initiations to find partners and quickly form teams (day 3 vs. day 7 peak) , and more effort in the final phase. Teams with high trust encouraged each other, took initiatives, were enthusiastic in language, and responded to each other, early on. Inequitable, irregular, and unpredictable communication hindered trust. Teams ending with high trust gave substantive and timely responses to one another. Of 14 teams that started with Low trust, 10 stayed Low; of 15 that started with Hi trust, 10 stayed Hi. Appointed leaders of Hi-Lo teams engaged in negative rather than positive reinforcement. Choice of medium: Electronic > FtF and phone 76.2% versus 23.8% Medium limitations: Electronic had higher ambiguity than FtF and phone. Adaptation: Electronic > FtF and phone Response times: Electronic contributions were slower than FtF contributions. Size of communication: Electronic > FtF Decision quality: Electronic > FtF 148 FJERMESTAD AND HILTZComments Conclusions Perceived information richness of the medium was strongly correlated with perceived productivity enhancements as a result of system use. Iacono & Weisband, 1997 Jarvenpaa, & Leidner, 1998 \"Once involved in an interaction, participants must actively respond by signaling their involvement and by doing what is requested of them. We conceptualize this active interaction as initiations and responses. Initiating an interaction by asking a specific question or making a proposal implicates the receiver in generating a relevant response... But to initiate interaction requires trust. Each individual initiation is an enactment of swift trust, contributing to the collective perception that trust is reasonable, inspiring more trust and more initiations from other members of the group. If initiation of interactions is part of doing trust work, so, too, is generating the relevant responses . A response indicates to the initiator (and everyone involved in the interaction) that the receiver has done her obligatory part. Consequently, the making of responses signals and inspires trust in the group\" (p. 413). \"Continuous interaction among team members fosters trust and predicts team performance.\" High performing teams quickly began to form teams, and were able to handle several activities at once. Most intense efforts were several days before each deadline instead of at the last minute (or not at all) (p. 419). “Swift trust” is based initially on clear role definitions to form initial expectations, and is then built and maintained by “a highly active, proactive, enthusiastic, generative style of action; action strengthens trust in the group in a self-fulfilling fashion.” Kock, 1998 There was a perceived increase in member contribution quality through the electronic media. The author suggests that adaptation to electronic communication led group members to prepare longer and better thought out contributions than in FtF settings. The results suggest that the process improvement groups implemented their proposals which resulted in increased quality and productivity. This resulted because the groups adapted to a leaner medium (electronic CMC). The results also suggest the CMC can be used for equivocal tasks. Author Results Comments Conclusions Kock & Cost: 78% felt that email conferencing decreases the cost of running There was considerable lag between leader's questions and group response, ranging McQueen, 1998; process improvement (PI groups, through (a) reduction in disruption from under an hour to more than 8 days; mean was 73 hours. This means group leaders Kock, 1997 of member functional activities in their jobs; (b) a drastic reduction in would typically have to wait about 3 days for replies enabling them to proceed further. transportation and communication expenses in groups involving The usefulness of asynch may increase with the number of depts. involved in a group. members based in different cities; and (c) considerable reduction in Five of the groups were successful in generating and either fully or partially implementing member participation time (65 to 92%). process redesign proposals. \"While not having negative perceived effects on group Effectiveness: 33% felt conferencing increased effectiveness, 33% effectiveness, asynchronous groupware support was perceived as increasing process no difference compared to FtF, 17% thought it Decreased. Those adoption, hierarchy suppression, departmental heterogeneity, and contribution length, who thought effectiveness increased cited improvement in the and decreasing discussion duration, cost, and interaction in process improvement quality of individual contributions fostered by the written groups.\" While synchronous GDSS groups are often associated with an increase in the asynchronous medium. Group interaction: fewer turns but longer quantity of ideas generated by the group, asynch seems to be related to an increase in contribution length per turn (mean of 260 words online vs. 40 per group process adoption and discussion focus, increase in quality of contributions, and a turn FtF). drastic decrease in group set up costs. 1. CMC support enables group discussions to be carried out without effecting individual timetables 2. CMC support reduces the influence of distance 3. CMC reduces the influence of previous interdepartmental conflict 4. CMC improves the way members interact- more sincere 5. CMC encourages members to write better. Krcmar, Lewe, & Case 1: Social club- GSS tools not used, Facilitation is essential in convergent group phases. The facilitator must act very Schwabe, 1994 Case 2: BPR-Anonymity very important; over time (1 day) the group carefully- participant are not in favor of heavy facilitation. The usefulness became more committed; More ideas were generated; difficulty Anonymity was mixed. Parallel processing was the most important benefit. The most structuring the ideas; Improved productivity. consistent observed benefit was equality of participation. Case 3: Review- The facilitator was the most import process; Anonymity was important; Parallel work most important; More equal participation Lewis, Keleman, Ease of use Positive The authors suggest that consultants/ The results suggest that the GSS was & Garcia, 1990 Comfort Positive facilitators should be available when favorably viewed by the groups. Efficient Positive dealing with unstructured problems, and Effective Positive when the groups are not experienced with Productive Positive the process, tools, and group dynamics. Better than FtF Positive Training is also crucial to the groups Process Satisfaction Positive performance. There are significant Solution Satisfaction Positive interactions between the group, task, and Willingness to use it again Positive settings. Recommend GSS Positive GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 149150 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results McCart & Rohrbaugh, 1989 On follow up, 5/14 groups were very positive about benefits, 5 not positive, and 4 intermediate. Two major benefits of decision conferencing: location away from office, and presence of facilitator. Five characteristics explained variance in perceived benefits: 1. providing an opportunity for open and extended discussion 2. Building a computer based decision model 3. Reviewing computer output on implications of alternative choices 4. Construction of an action plan to be implemented and 5. Firmly believing that the problem would be resolved by the end of the conference. Decision conferences were not successful when these five benefits had not been fully realized. Perceived usefulness of the model was the strongest predictor. 2 of the 4 teams did not use internal electronic messaging. Interviews suggest that the primary reason for joining the project was to get the word processing software. The two teams which adopted the technology, primarily communicated between two people. Comments Conclusions Differences in perceived conference success were related to (1) the proportion of participants who believed the conference resulted in a decision and (2) the level of benefits derived from full support of the structure or preference technology, including full discussion of the decision models and an action plan. Decision conferences fail when these benefits are not provided. For instance, the group many squander so much time building a model that they do not use it. An inappropriate devotion to the use of information technology may substantially reduce the opportunity for open and extended discussion. Markus, 1992 Synchronous use of the technology came about due the need of one group to convert from 5 ¼ to 3.5 inch disk media. This is AST. Adoption and usage patterns differed across the four groups. Not all groups adopted the technology, for those that did not all members participated. External messaging was used more that internal messaging. Groups utilized the technology to overcome geographic distance, media incompatibilities, and poor group relations. Overall, the results suggest that Mexicans can benefit from GSS. GSS assisted the participants to enhance inter-group communication and integrate information. GSS application in Mexican contexts is often more flexible than with US groups. Because of relatively high power distance effects, Mexican cultures would be less comfortable in anonymous GSS contribution. Morales, Moreira, & Vogel, 1995 GSS applicability to participants job: more agree 47.3% FtF communication is better with a GSS: More disagree 49% GSS helps the group integrate better: more agree 86.7% Group participation is better with a GSS: More agree 44% Quality of results was good with the GSS: More agree 96% Mexican participants tend to be more expressive and tended to comment on opinions of others more than Americans, whereas GSS tends to encourage brevity. GSS tends to promote collective behavior and should thus evoke comfort among Mexican groups because they are on the “collectivism” vs. ‘individualism” end of the cultural values scales. Author Results Comments Conclusions Nunamaker, 1. What are the dynamics of an EBS session? The users focus on the task and the GSS The results suggest EBS process Applegate, & Entering comments- 57%; waiting for screen- 12.6%; reading with very little interaction between the outweighs other obstacles. In addition, the Konsynski, screen- 27%; group interaction- 3.4% participants. EBS appears to decrease users report high levels of satisfaction with 1987:1987 2. How does the technology facilitate the idea generation inhibition. The GSS allowed for more the outcomes and rate the GSS as an process? equal participation. important tool for idea generation. Comments suggest that it is improved. 3. How does the technology inhibit the idea generation process? The interface, typing, and waiting for the next screen all inhibited the process. 4. What is the reported satisfaction of the planners using the GSS? High levels of both outcome and process satisfaction were reported. Post, 1992; 1993 Observations and measures The results of the surveys suggest that the The results suggest that GSS technology Effectiveness- dollars saved: $432,260 organization can reap significant benefits appears to be most valuable when it is Efficiency- labors hours saved: 71% from the technology. deployed and integrated into Consensus generating: Yes organizational decision making Improved decision making: Yes environment that is both dynamic and Comfort: Yes. complex. Willing to use the system again: Yes. Quaddus, The multiattribute decision model (MADM) was used to compare The authors suggest that the major benefits for the group were: Atkinson, & with weights the differences between current strategies and Levy, 1992 reorganized strategies. The results revealed that 9 reorganized • Agreement on strategic direction strategies out of 15 strategies were preferred. An action plan to • The process enabled the groups to remain focused implement these strategies was then developed. • Conflicting opinions were more easily dealt with • The important issues were easily surfaced • A shared understanding developed • Facilitation played a major role Sheffield & Observations from Case Study The authors suggest that the meeting The main finding of the follow-up study Gallupe. 1993; Effectiveness: Yes technology supported the economic was that the initial meetings were a 1994; 1995; Effectiveness of facilitation: Yes development process where meeting catalyst for cooperative action for the 1995 Effectiveness of technology: Yes urgency and efficiency were of prime industry participants. The participants felt Reducing barriers: Yes importance. All observations are that the intensity of the meeting process, Participation: Equal perceived. the EMS, the model, and the large number Information exchanged: Improved of participants resulted in long-term Meeting outcomes: Effective success. Average effectiveness: Effective GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 151Appendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Vician, Systematic processes: Improved over time DeSanctis, Openness of communication: Improved over time Poole, & Comfort with technology: Improved over time Jackson, 1992 Member attention and Interest: The GSS support this GSS improved negotiations- ranking and rating activities GSS improved participation GSS enhanced communication GGS help in sense of accomplishment over time Vogel & Effectiveness: Idea generation-11 pages, 650 lines of ideas in 40 Nunamaker, min. 1989 Issue analyzer-over 100 focus items identified in 45 min. Issue consolidation-13 items condensed from 100 in 45 min. Efficiency- The groups suggested that this would normally take 2 days to complete. Vreede, 1996; GSS Technology- Positive results: 1998 Quantity of session results Greater productivity than manual meetings High quality Usability Collaborative Design Process- Positive results Satisfied Agreement Enjoy Usability Vreede, Briggs, Problem Identification Tasks 1. Productivity: Sequential > Blank slate Duin, & Enserink, 2000 2. Unique ideas: Ns 3. Relevant elaborations: Sequential > Blank slate Solution Generation Tasks 1. Productivity: Sequential > blank slate 2. Unique ideas: Ns 3. Relevant elaborations: Sequential > Blank slate 152 FJERMESTAD AND HILTZComments The authors suggest that the team members seem to have gained a favorable sense of accomplishment as time progressed. Typically the scores were low in the early period (after 5 meetings). Conclusions The effects of the GSS and TQM approach were subtle. The technology does not bring revolutionary advancements toward the goal. Leadership is important: 1) commitment on the part of the leader to the principles of participation; 2) perseverance in continual learning and application of the technologies over time. The authors concluded that: 1. Anonymous session are important-promotes honest and thoughtful. responses from the group members. 2. Efficiency-parallel input of ideas. 3. Flexibility-the software can configured to meet the needs of the group. 4. Enhanced group dynamics- equality of participation. 5. Electronic memory-all comments are recorded. This technology was compared to other methods and the results were all positive: Stakeholders built conceptual models and models for change using the technology. The GSS technology made the collaborative activities more efficient. GSS technology and animation technologies are complementary in BPR. Groups working on problem identification tasks produced significantly higher numbers of ideas than groups working on solution generation tasks. Blank slate participants (decathlon) suggested that there were fewer exchanges of ideas and elaboration. Sequential (relay) participants had no negative comments. No differences in product or process satisfaction were reported. Sequential (relay) groups make significantly more elaborations than do blank slate (decathlon) groups and produce slightly more unique ideas. Relay participants were significantly more satisfied in terms of interest accommodations than decathlon participants. The facilitator is a strong stimulus for meeting satisfaction. Author Results Comments Conclusions Vician, Systematic processes: Improved over time The authors suggest that the team The effects of the GSS and TQM approach DeSanctis, Openness of communication: Improved over time members seem to have gained a favorable were subtle. The technology does not Poole, & Comfort with technology: Improved over time sense of accomplishment as time bring revolutionary advancements toward Jackson, 1992 Member attention and Interest: The GSS support this progressed. Typically the scores were low the goal. Leadership is important: 1) GSS improved negotiations- ranking and rating activities in the early period (after 5 meetings). commitment on the part of the leader to GSS improved participation the principles of participation; 2) GSS enhanced communication perseverance in continual learning and GGS help in sense of accomplishment over time application of the technologies over time. Vogel & Effectiveness: Idea generation-11 pages, 650 lines of ideas in 40 The authors concluded that: Nunamaker, min. 1. Anonymous session are important-promotes honest and thoughtful. responses from 1989 Issue analyzer-over 100 focus items identified in 45 the group members. min. 2. Efficiency-parallel input of ideas. Issue consolidation-13 items condensed from 100 in 3. Flexibility-the software can configured to meet the needs of the group. 45 min. 4. Enhanced group dynamics- equality of participation. Efficiency- The groups suggested that this would normally take 2 5. Electronic memory-all comments are recorded. days to complete. Vreede, 1996; GSS Technology- Positive results: This technology was compared to other Stakeholders built conceptual models and 1998 Quantity of session results methods and the results were all positive: models for change using the technology. Greater productivity than manual meetings The GSS technology made the High quality collaborative activities more efficient. GSS Usability technology and animation technologies are complementary in BPR. Collaborative Design Process- Positive results Satisfied Agreement Enjoy Usability Vreede, Briggs, Problem Identification Tasks Groups working on problem identification Sequential (relay) groups make Duin, & 1. Productivity: Sequential > Blank slate tasks produced significantly higher significantly more elaborations than do Enserink, 2000 2. Unique ideas: Ns numbers of ideas than groups working on blank slate (decathlon) groups and 3. Relevant elaborations: Sequential > Blank slate solution generation tasks. Blank slate produce slightly more unique ideas. Relay Solution Generation Tasks participants (decathlon) suggested that participants were significantly more 1. Productivity: Sequential > blank slate there were fewer exchanges of ideas and satisfied in terms of interest 2. Unique ideas: Ns elaboration. Sequential (relay) participants accommodations than decathlon 3. Relevant elaborations: Sequential > Blank slate had no negative comments. participants. The facilitator is a strong No differences in product or process stimulus for meeting satisfaction. satisfaction were reported. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 153154 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Vreede & Bruijn, in-press; Bruijn & Vreede, 1999 Testing GSS Assumptions: 1. the meeting process should be fair- Yes 2. the meeting process should be open- Yes 3. the meeting process should be rational- Little 4. groups should be guided by a facilitator- Yes 5. groups should exchange as much information as possible- Mixed 6. people are cooperative to each other & the meeting process- Little 1. GSS sessions improved insights 2. GSS sessions improved consensus 3. GSS sessions improved coordination & participation 4. Positive perception with respect to the information and knowledge elicited during the sessions. 5. High levels of a willingness to work together again (Usability). 6. GSS improved productivity over manual methods 7. Anonymity a key ingredient to the sessions Participants were generally very satisfied with the meting process, the technology, and the meeting outcomes. Over 97% of Malawi and Zimbabwe meeting participants said they would recommend this technology; ratings of over 4 out of 5 in Tanzania. Useful results were achieved that were endorsed by participants. In terms of the Technology Acceptance Model, data suggest that top management endorsement, computer literacy, and satisfaction with use stimulate GSS acceptance, whereas a preference for oral communication combined with referent power issues had negative impact. Comments Conclusions Vreede & Dickson, 2000 The authors suggested that the results on the GSS assumptions may be dependent upon the phase of the decision making process (orientation, separation, package deal). The orientation phase appears to be a good candidate for GSS application; the separation phase, poor; and package deal phase has potential for GSS application but with caution. Codes: Yes- there is support for the basic GSS assumption Little- there is only a small amount of support Mixed- there is equal support for Yes or Little The system design took 9 months and the The global objective was achieved in planning & analysis took 3 months. The addition to an effective design. The GSS stake holders were satisfied with process, supported efficient data collection and outcomes and the GSS. model construction activities. Vreede, Jones, & Mgaya, 1998-1999 Logistic difficulties emphasized: frequent power surges or outages, extreme heat shut down equipment, humidity grew mold inside computers. Referent power means that people do not hold a position in an organization based on their skills, but on their contacts. Often, decisions are not based on information or rational interests but on personal relations and favors. Characteristic of this part of Africa. Anonymity interferes with this. Having shared computers with one person doing the input “was a great success” when some participants had no computer or keyboarding skills. In multi-day meetings, handing out hard copy of he work done the previous day was enthusiastically received. Cultural factors influenced use. Relevant external factors that extend the TAM for the African cultures studied include endorsement of top management, computer literacy, oral communication preference (some typed in English but prefer native language for oral discussion), referent power, and satisfaction with use. Author Results Comments Conclusions Vreede & Muller, 1. Anonymity- impeded the quality of information exchange These were 3 cases selected to be poor. The results suggest that there are several 1997 2. GSS good for brainstorming, but not good for decision making The objective was to find out what they factors which contribute to GSS meeting 3. GSS usability- low had in common. The results suggest that failures: overloaded agendas, little 4. Process satisfaction- some what positive outcome quality is the strongest indicator discussion time, diverging perceptions of 5. Focused- low of a bad meeting. Explanations: diverging meeting goals, knowledge gaps among the 6. Information overload- Full agenda perceptions of the meetings goals, participants, and applying the technology 7. Communication- negative perceptions knowledge gap among group members; without special provisions in conflict 8. Participation- low/poor the use of anonymity may have hindered situations. 9. Facilitator- positive impact personal negotiating capabilities. 10. Commitment- low/poor 11. Quality lower than manual methods Vreede & Wijk, Use of GSS increased productivity by 55% both in terms of man The intensity of an electronic meeting Critical factors to the success of GSS 1997 hours and project time span. GSS judged to have increased quality seemed to fatigue participants; as a result, meetings included the translation of of outcomes. Productivity gains seemed to increase in larger some seemed unmotivated to fill out post-meeting objectives into a structured projects. Groups liked working anonymously and in parallel. meeting questionnaires. agenda, active motivation of group members to participate, and enough times for groups to digest intermediate meeting results. Walczuch, No change in cohesion. Details are provided of the step by step 1. The appropriate mix of group support Watson, A positive attitude toward the process prevailed. \"The newness of processes used in each of the three systems technologies and manual Bostrom, & Day, the technology seems to have stimulated group members interest in meetings. techniques is instrumental in achieving 1995 the meetings.\" Most managers satisfied with the process, especially meeting success. the anonymity and the simultaneity of the GSS systems. 2. Constant review of the overall meeting design contributes to the success of the meting; 1. negative and positive aspects of a meeting should be balanced to provide feedback and encouragement to the group. Zigurs, Attitudes across groups: Attitudes vacillate across time and task The discarded groups appear to be more Usage of GSS varies over time. Groups DeSanctis, & characteristics. Attitudes and quality: Significant correlations with ineffective and model Hirokawa & Poole's follow a cycle of experimenting- learning- Billingsley, 1991 perceived quality. Task participation: Significant correlations with theories. GSS learning follows a cyclic experimenting. Different groups emerge quality and negative socio-emotional behavior. System usage: approach. over time: adopters and discarders. decreases across time. GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 155156 FJERMESTAD AND HILTZAppendix 3: Group Support Systems: A Descriptive Evaluation of Case and Field Studies—Results (Continued)

Author Results Zigurs & Kozar, 1994 1. Role perceptions of initiators and participants: § Initiators had no difficulty in identifying the expertise of the participants. § There is a lack of agreement between initiators' and participants' ideas about roles. § Participants had difficulty with identifying their correct roles. § Participants filled a significantly lesser variety of roles than they expected to. § The GSS actually assumed many of the roles that the participants expected to fill. 2. Prescribed roles and perceived effectiveness § There were no significant correlations between perceived effectiveness of outcomes and task roles. § There were no significant differences between effectiveness of process and task roles. Comments The data suggests considerable mismatches between the role expectation of the initiators and participants. This inconsistency might explain why meetings fail to \"behave\" and evolve the way they were planned. The role of the recorder was most often mentioned for the GSS. However, the GSS only recorded the typed input and not the verbal communications. Conclusions The technology was perceived by participants as assuming roles, some of which were roles they themselves expected to fill. EIES Field Trials Johnson-Lenz, Johnson-Lenz, & Hessman, 1980 Johnson-Lenz, Johnson-Lenz, 1980; Lamont, 1980; Stevens, 1980 System use sped up the standardization process, contributed to better standards by making more information available and providing more discussion opportunities than just quarterly face to face meetings, and better prepared members for the quarterly meetings. Various members started “spin off” conferences on special topics. Success of the group directly correlated with time online of the leader/facilitator. System use improved speed and quality of decisions. Success of the group directly correlated with time online of the leader/facilitator. CMC said to make information exchange much more convenient for participants in 25 states across 5 time zones. “The combination of access to resources and structured communications capabilities for information exchanges makes a (tailorable CMC) system very promising for meeting the information needs of policy makers on all levels.” Author Results Comments Conclusions Bernstein, Reviews and new updated database were completed. Little non-Success of the group directly correlated The technology was perceived by Siegel, & task communication took place among the doctors. with time online of the leader/facilitator. participants as assuming roles, some of Goldstein, 1980; Deemed successful; modifications can be which were roles they themselves Siegel, 1980 effectuated in an efficient and timely expected to fill. manner through CMC. IBM Studies Grobowski, Lessons learned: The results suggest that the electronic environment is a more appropriate channel for McGoff, Vogel, Anonymity is beneficial group communication than the manual one. Martz, & GSS processes can aid varied group compositions Nunamaker, Higher levels of participation 55.5% man hour savings were reported 1990 Fewer meetings over less time 92% average calendar reduction McGoff, Hunt, Participants say focused More formal, more recently established, and less cohesive groups tended to achieve Vogel, & Pre-meeting planning is important higher levels of man hour savings than did similar groups without GSS technology. Nunamaker, Post-meeting distribution of session logs is crucial There are interactions between training, facilitator and participant. 1990 Meeting room environment should match the characteristics of the Martz, Vogel, & group Nunamaker, Software systems must be flexible 1992 An infrastructure of staff and support is crucial Nunamaker, The GSS should provide for an organizational memory Vogel, Results Heminger, Dependent measures (descriptive or relationship results) Grobowski, & Strongly agree: McGoff, 1989 Computer better than manual Vogel, Idea generation Nunamaker, Issue identification Martz, Groboski, Goal achievement & McGoff, 1990 Process is fair Effectiveness: GSS > manual methods Participation: equalized Efficiency: GSS < Manual in terms of man hours Satisfaction: Utilization rate of the system: high Process satisfaction: high Problem solving process Quality - positive Willingness to participate in future sessions - positive GSS- less non-task interaction Process satisfaction- positive GROUP SUPPORT SYSTEMS: A DESCRIPTIVE EVALUATION OF CASE AND FIELD STUDIES 157