Technology (IT) enabled crowdsourcing: A conceptual framework

: IT-enabled crowdsourcing is defined as technology-enabled outsourcing of tasks through an open call to the masses via the internet. Crowdsourcing is an IT artifact that has gone beyond the traditional boundaries of an organization to a much broader context. Over the past decade, research and practice on crowdsourcing have continued to grow, evolve, and revolutionize the way work gets done. Although numerous studies have been conducted in this area, our understanding of the main components involved in crowdsourcing processes remains limited. The goal of the current study is to conduct a structured literature review and synthesize the available crowdsourcing literature and applications in one coherent conceptual framework. The framework identifies the main elements involved in the crowdsourcing process and its characteristics. This framework extends the field of Information Systems (IS) and would help us better understand this phenomenon. Furthermore, the results of this study could potentially fill the knowledge gap in the crowdsourcing literature by identifying the main characteristics of a crowdsourcing process as a legitimate, IT-enabled form of problem-solving. Our results would also help organizations to leverage crowdsourcing more efficiently.


Introduction
Social networking systems allow us to connect easily with one another to communicate, learn, educate, conduct business and solve problems.Advances in connective and collaborative technological environment have enabled individuals to get involved in internet-mediated social participation and have transformed users from passive browsers to active contributors.Crowdsourcing is one such phenomenon.Despite a significant increase of effort in crowdsourcing research and practice, the concepts and components of crowdsourcing activities remain unclear at best.

Background
The concept of seeking assistance beyond one's capabilities from the 'crowd' is not new.In 1714, the British government asked the crowd to develop a reliable way to compute longitude and offered a monetary prize for the winner.In 1858, a group of scholars created the first Oxford English Dictionary and appealed to volunteers to write entries according to their area of expertise (Hossain & Kauranen, 2014).The dictionary was assembled by a 'crowd.'However, it wasn't until the rise of the Internet that harnessing the power of crowds and the phenomenon that is now known as crowdsourcing took off.Internet-enabled technologies enable large heterogeneous groups of people from all around the world to communicate and collaborate and set off a wide range of "open" and "crowd-sourced" practices and approaches (Benkler, 2016).Levy (1997) is among the first scholars who pondered the emergence of a collective intelligence system as individuals contribute to the "knowledge community" through the internet.Surowiecki (2004) investigated several cases of crowd wisdom applications where the success of solutions depends on a large body of solvers.He proposed that by providing the right circumstances, groups are often smarter than the smartest people in them.In other words, under the right conditions, crowds produce better solutions than those offered by experts; under the right conditions, size and diversity beat ability.Howe (2006), coined the term crowdsourcing as the distribution of work to the crowd via internet.
Since then, different terminologies were used to describe the phenomenon, such as collective intelligence, crowd wisdom, and mass collaboration (Doan, Ramakrishnan, & Halevy, 2011).Other terms can also be found in the literature; including collective wisdom (Hwang, 2009) and crowd work (Kittur et al., 2013).In this study, the term crowdsourcing is used because it fully captures the concept and has been widely used by many studies in the field (Estellés-Arolas & González-Ladrón- De-Guevara, 2012;Howe, 2006).One should note that there are differences between crowdsourcing and other associated concepts such as outsourcing and open sourcing.While outsourcing allocates work to a defined organizational entity, crowdsourcing allocates work to a random collection of individuals.It makes it possible to harness volunteers who might not otherwise be able to contribute (Saxton, Oh, & Kishore, 2013).There are also differences between crowdsourcing and open sourcing.While open sourcing is about a community sharing code for the common good and therefore involves many contributors and many beneficiaries (Grams, 2010), crowdsourcing involves many contributors and few beneficiaries.
In the crowdsourcing literature, various perspectives of crowdsourcing have been presented: a process that involves several key actors and operations (Hetmank, 2014), a paradigm that provides principles to real-world problems (Buettner, 2015), or a platform with specific functions and features which can implement the paradigm and support the corresponding processes (Gray, Shoaib, Kulkarni, & Suri, 2016;Kucherbaev, Daniel, Tranquillini, & Marchese, 2016).In this study, crowdsourcing is examined as a process.
In the last decade, practices and procedures in crowd-sourced systems have continued to grow, evolve and revolutionize the way work gets done (Dwarakanath et al., 2015;Irshad et al., 2017;Vernez et al., 2017).The term crowdsourcing has been used for various activities which makes it challenging to clearly define what crowdsourcing really is and what its characteristics are.To improve crowdsourcing performance, it is necessary to have a clear definition of crowdsourcing and to understand the process characteristics.
Many researchers have proposed their own definitions for crowdsourcing (Brabham, 2008a(Brabham, , 2008b(Brabham, , 2012;;Howe, 2006Howe, , 2009;;Vukovic & Bartolini, 2010).These definitions are focused on area-specific aspects of crowdsourcing applications.Hetmank et al. (2009) conducted a systematic literature review in the domain of crowdsourcing systems.Seventeen definitions of crowdsourcing systems were found and categorized into four perspectives: the organizational, the technical, the functional, and the human-centric.
Estellés-Arolas and González-Ladrón-De-Guevara (2012) provided a more general definition of crowdsourcing and established the essential characteristics of any crowdsourcing initiative.They synthesized 40 definitions extracted from 209 crowdsourcing articles.As a result, they proposed a description covering any given crowdsourcing activity, which was characterized by the following elements: a defined crowd, an outlined task, an explicit compensation for the crowd, an identified initiator, defined benefits for the initiator, an online process, the open call, and internet usage: "Crowdsourcing is a type of participative online activity in which an individual, an institution, a non-profit organization, or company proposes to a group of individuals of varying knowledge, heterogeneity, and number, via a flexible open call, the voluntary undertaking of a task.(Estellés-Arolas & González-Ladrón-De-Guevara, 2012) (p.197)" We concur with the above definition and apply it throughout this study.

Development of a conceptual framework for crowdsourcing
Crowdsourcing is a new way of doing business and to better understand this phenomenon, a broad understanding of its process, its mechanisms, and its main components is essential.For this reason, we focused on studies that address crowdsourcing conceptual frameworks, literature reviews, typologies and taxonomies.To develop a crowdsourcing conceptual framework, an extensive literature review was conducted where the article collection process was based on a structured review.For the structured literature review, we followed practices for literature reviews accepted in the Information Systems (IS) discipline (Schryen, 2013;Webster & Watson, 2002).
We used EBSCO Business Host database (https://www.ebsco.com/)as our source of data.Focusing the search on a widely available database ensured that the procedure is replicable, rigorous, and transparent.We also searched the top ten journals that are identified as high-quality mainstream IS journals (https://www.scimagojr.com/).The top IS journals include MIS Quarterly, IS Research, Journal of MIS, European Journal of IS, IS Journal, Journal of the Association for IS, Journal of Strategic IS, Communications of the Association for IS, Information and Management, and Decision Support Systems.The review also included proceedings papers from three major IS conferences, namely the International Conference on Information Systems (ICIS), the Hawaii International Conference on System Sciences (HICSS), and the Americas Conference on Information Systems (AMCIS).The literature search includes papers from 2006 to 2018.Articles were collected, and the search terms included such words as "crowdsourcing", "crowdsource", and "crowd-sourced", along with the keywords: "conceptual framework", "typology", "taxonomy", and "review."The search resulted in 150 hits.We screened the papers for inclusion and relevance, using the criteria that the research is clearly focused on IT-enabled crowdsourcing conceptualization, literature review, typology, or taxonomy.
Although several papers were found on the subject, few of them focused on the understanding of the integration of all the main components involved in a crowdsourcing project.We reviewed those papers that specifically presented a taxonomy of CS systems or a conceptual framework to come up with the main components of a crowdsourcing project.We then integrated the factors introduced in the I-model (Zhang & Benjamin, 2007) with the concept of "genes" as the building blocks of the CI genome (Malone, Laubacher, & Dellarocas, 2010) to identify and categorize the components of a crowdsourcing process.The combination of these models and ideas form the theoretical foundation of our work.
In one of the earliest studies in this area, Rouse (2009) proposed a taxonomy of crowdsourcing considering three dimensions of supplier capabilities: nature of the task, distribution of benefits, and forms of motivation.In other studies, Doan, Ramakrishnan, and Halevy (2011) provided a global picture of crowdsourcing systems based on the nature of collaboration, architecture, crowd recruitment requirement, and nature of the task.They identified nine dimensions to describe any crowdsourcing activity and addressed challenges facing the recruiting and retaining users in a crowdsourcing project.In another study, Hosseini, Phalp, Taylor, and Ali (2014) analyzed the literature and proposed a taxonomy of crowdsourcing based on the four pillars of crowdsourcing: the crowdsourcer, the crowd, the crowdsourced task and the crowdsourcing platform.Yuen, King, and Leung (2011) proposed that the literature on crowdsourcing can be categorized into application, algorithm, performance, and dataset.They grouped crowdsourcing applications into four categories of voting system, an information sharing system, game, and creative system.Cullina and Morgan (2015) provided a taxonomy to address the issue of measuring the crowdsourcing process.This research-in-progress paper examined crowdsourcing at the operational level with a view towards (i) identifying the parts of the process (ii) identifying what can be measured and (iii) categorizing operational metrics to facilitate deployment in practice.In this study the focus is on the measurements of the crowdsourcing process and not identifying its different components.Geiger and colleagues (Geiger, Seedorf, Schulze, Nickerson, & Schader, 2011) identified four types of crowdsourcing mechanisms exclusively within the organizational context based on following dimensions: task, value derived from contributions, differentiation between contributors, technology, and remuneration for contributions.
Subsequently, Zhao and Zhu (2012) applied the Malone, Laubacher, and Dellarocas (2010) model to study motivation factors in crowdsourcing contests.Later, Zhao and Zhu (2014) did an extensive review of the crowdsourcing research and identified crowdsourcing research opportunities from three perspectives: participant, organization, and system where participant's perspective includes crowd's motivation and behavior, organization's perspective includes activities such as adoption, implementation, governance, and evaluation of crowdsourcing, and system's perspective includes the incentive mechanism design and other related technology issues.Pedersen et al. (2013) provided a review of the crowdsourcing research in the IS field.The authors developed a crowdsourcing conceptual model based on the traditional "Input-Process-Output" model.Elements of this conceptual model include: Problem, People (Problem Owner, Individual, and Crowd), Governance, Process, Technology, and Outcome.This review is focused in the field of IS and invites future extension of crowdsourcing knowledge and understanding based on the proposed conceptual model.Similarly, Ghezzi, Gabelloni, Martini, and Natalicchio (2017) reviewed the crowdsourcing process in management journals applying the "Input-Process-Output" framework to interpret research on crowdsourcing.Further, Assis Neto and Santos (Assis Neto & Santos, 2018) reviewed workflow and quality management in crowdsourcing projects.The previous models of crowdsourcing proposed in the referenced studies are summarized in Table 1 in the appendix.This table shows the differences in these studies with the current research and how the components of the crowdsourcing process mentioned in these papers were integrated into the proposed framework.Although several papers were found on the subject, few of them focused on the understanding of the interaction and integration of all the main components involved in a crowdsourcing project.
A few papers provided us with an initial set of dimensions which we have integrated into a general conceptual framework for the IT-enabled crowdsourcing process.These are described next.Zhang & Benjamin (2007) proposed the information model or I-model which identifies the main components of any information-related field.These components are domain (context), people, information, technology, and organization that can continuously interact and integrate with each other.
Subsequently, Malone, Laubacher, and Dellarocas (2010) proposed the idea that to build a collective intelligence (CI) system to accomplish an organization's desired task, managers have to ask the following four questions: "what is being done (goal)?Who is doing it (staffing)?Why are they doing it (incentive)?And, how the task will be done (structure/process)?" The authors introduced the concept of CI genes as a particular answer to each one of the above key questions thus allowing managers to design their required organizational CI system through recombining these genes.We integrated the factors introduced in the I-model (Zhang & Benjamin, 2007) with the concept of "genes" as the building blocks of the CI genome (Malone, Laubacher, & Dellarocas, 2010) to identify and categorize the components of a crowdsourcing process.
Crowdsourcing is a type of open-sourced information systems where the sources of information or "intelligence" is the crowd.
The first step in designing a crowdsourcing process is clarifying the domain of the project.This part of our conceptual framework is based on the I-model.Extending the idea of "who is doing the task" in a CI system, and "people" in the I-model, we distinguished the following three groups of people involved: initiators of the process, benefactors of the outputs of the process, and participants in the crowdsourcing process or the crowd.The type of the task and its characteristics should also be identified as well as individual participant's motivation to perform the task based on the idea of CI genome (Malone, Laubacher, & Dellarocas, 2010).Since the focus of this study is on the IT-based crowdsourcing process, the technology of the platform and its characteristics were also recognized in the model (Zhang & Benjamin, 2007).
The I-model includes organization and structure, encompassing activities that define the strategy, policy and management of any information-related fields (Zhang & Benjamin, 2007).These activities are implied in our conceptual framework as part of the two categories of "who" initiates the process and "who" benefits from the results.It is expected that the process initiators or managers will come up with the strategies to measure the quality of the process and the output as well as work flow management.
We present our conceptual framework in Fig. 1.After identifying the domain of the crowdsourcing project, based on the framework developed in our work, we extended the main dimensions of any crowdsourcing process proposed by Malone and Laubacher (Malone, Laubacher, & Dellarocas, 2010) as follows: Who? (Who initiates the process?Who benefits from it?Who performs the task?) Why? (Why does the crowd participate in the process?)What? (What is the task?) How? (How does the crowd perform the task?(i.e., the technological platform characteristics).Different combinations of answers to these questions describe different types of crowdsourcing processes.In the next sections, a comprehensive review of the literature is conducted to map the characteristics of a crowdsourcing process based on the possible answers to each of these key questions (Who-Why-What-How).

Domain
Crowdsourcing can be applied in two contexts: business and non-business.The former includes companies, for-profit organizations or marketplaces (Poetz & Schreier, 2012;Whitla, 2009) and the latter includes non-profit organizations or institutions, such as public libraries or government (Heer & Bostock, 2010).Crowdsourcing is a model for problem-solving, not merely a model for doing business (Afuah & Tucci, 2012;Doan, Ramakrishnan, & Halevy, 2011).Researchers study both business and non-business applications of crowdsourcing.It is essential to identify and clarify the domain for the successful adoption and implementation of crowdsourcing.
Perhaps the most crucial component of a crowdsourcing process is the initiator (Yan & Wang, 2013).The initiator can be a company (e.g., Coca-Cola's "Shaping a Better Future" challenge, Doritos' "Crash the Super Bowl" contest, etc.), or a public organization (e.g.Smithsonian Institution's "Digital Volunteer" program, the Federal Bureau of Investigation's crowdsourcing website launched in April 2011, etc.).Individuals of any background can also turn to a crowd to solve their problems.For instance, Howe used crowdsourcing to design the cover of one of his books (Howe, 2009).Crowdsourcing suggests a business model for companies, but more than that, it is a potential problem-solving mechanism (Brabham, 2008a).The initiator of the process can be any entity that can carry out the process which could be for-profit or non-profit organizations, a government institution, or an individual.

Who benefits from the process?
Crowdsourcing process can benefit three groups (Rouse, 2009).First, processes that provide private benefits to meet the commercial goals of the initiator(s) (e.g., Amazon Mechanical Turk).Second, social projects that are designed to benefit the public or a community of some type that uses the power of the crowd in the service of humanity (e.g., Galaxy Zoo, Data Kind, etc.).Third, projects that provide mixed benefits for both individuals and communities (e. g., customers offering suggestions for product improvement can benefit the firm as well as many customers if the idea is valuable to many customers).

Who performs the task?
The crowd is the dynamically formed group of individuals who voluntarily participate in crowdsourcing systems to share their ideas, experiences, knowledge, work, or money (Zhao & Zhu, 2012).The attraction of the right crowd and their sustainable contribution are the keys to crowdsourcing success and require an understanding of the characteristics of the individual members of the crowd.Also, the project initiator should have appropriate information about the crowd as the lack of information about the crowd might impact the management and the quality of the process (Assis Neto & Santos, 2018).In this study, we will review the source of the crowd and their skill levels.
Source of the crowd: Depending on the initiator's tolerance for risk and the need for heterogeneity, one source of the crowd may be superior to the other.Crowd heterogeneity is the result of the source of the crowd, whether it is the general public or a specific community.The origin of the crowd could be one of the following: existing specific communities where specific knowledge and expertise are resident; general public where any given interested party can participate; a combination of the two where a project initiator starts an open call to recruit the crowd but later filter and select the participants (Chilton, Horton, Miller, & Azenkot, 2010;Jeppesen & Lakhani, 2010).
As initiators reach out to external sources, the diversity and size of the crowd increases but so do potential risks and noise.Often, initiatives that deal with confidential information, as well as large corporations concerned about intellectual property leakage or loss of competitive strategies prefer internal communities over external crowd.Also, for some tasks such as translation, heterogeneity will not be so substantial.On the other hand, some projects require the wisdom and creativity of a heterogeneous group, where each person brings their knowledge.In these cases, an increase in the number of individuals who attempt to solve a problem increases the diversity of the ideas generated by the crowd, which increases the likelihood of getting to a novel, effective, and implementable proposal.In choosing the number of the crowd, initiators should also consider the availability of necessary tools and resources to store, filter, evaluate, and analyze the data created through the crowdsourcing process (La Vecchia & Cisternino, 2010).
Crowd's level of skill: crowdsourcing is built based on access to the Internet, which connects a diverse group of individuals who have many kinds of expertise, abilities, and problem-solving skills.Skill, broadly defined, includes the various capabilities that are relevant to the performance of a task.Skill plays a crucial role in the performance of tasks.Lukyanenko, Parsons, and Wiersma (2014) defined crowd information quality (crowd IQ) and studied its impact on user-generated content.Depending on the initiator's needs and nature of the crowdsourcing task, skills can be: general, specialized, or situational.While simple and repetitive work such as tagging an image requires general skills, asking for the crowd's creativity and wisdom may need specialized knowledge.For instance, asking the crowd to suggest improvements to the current product design, require specialized skills about the market, the product, the materials, the manufacturing process, etc. (Assis Neto & Santos, 2018).Furthermore, for evaluations and voting jobs, the crowd should have situational skills such as knowing about the time, place, event, etc.For tasks that approach specific individuals with particular skill sets, there could be initial training before the start of the process.An example can be found in the work of Feng et al. (2016).The authors proposed an educational game to train the crowd to perform biomedical image analysis.To improve task performance, it is essential to understand exactly what skills the person brings (or does not bring) to the task.

Why does the crowd participate in the process?
Incentives have been studied in the related areas of open innovation, outsourcing, and open source software (OSS).However, the voluntary nature of participation in crowdsourcing activities may require the project initiator to provide a different set of incentives for participants, and therefore, there is a need for additional studies on motivation in the crowdsourcing process (Zhao & Zhu, 2012).
The distinction between intrinsic and extrinsic motivation is the fundamental idea underlying several psychologists' theories of motivations and incentives (Leimeister, Huber, Bretschneider, & Krcmar, 2009).Intrinsic motivation refers to "doing something because it is inherently interesting or enjoyable" (Ryan & Deci, 2000).It arouses participants' inner motives such as natural feelings of competence, satisfaction, or fulfillment.It occurs when an individual engages in a particular behavior because it is personally rewarding.An intrinsically motivated individual may participate in an activity because of the fun and challenges associated with it rather than for external motives.External motivation, on the other hand, refers to "doing something because it leads to a separable outcome" (Ryan & Deci, 2000).It involves engaging in a behavior because of external incentives, such as recognition by others, or direct or indirect award or monetary compensation.
Other theories explore motivation through a smooth transition between internal and external motivation (Ryan & Deci, 2000).These theories have been developed through a set of five subtheories: External motivation which can be receiving rewards such as monetary compensation and better job opportunities (Brabham, 2008a(Brabham, , 2012;;Stewart, Lubensky, & Huerta, 2010); introjected motivation which can be getting recognition among peers (Brabham, 2010); identified motivation which can be feeling of greater freedom and volition since the behavior is more compatible with one's personal goals and identity (Ke & Zhang, 2009); integrated motivation which can be a sense of virtual community where the activities are considered as meaningful and significant (Brabham, 2010;Jin, Li, Zhong, & Zhai, 2015); and Intrinsic motivation which can be facilitating several intrinsic motivations such as perceived enjoyment and fun, curiosity and interest, developing individual skills and self-affirmation, etc. (Stewart, Lubensky, & Huerta, 2010).
In crowdsourcing projects, direct compensation such as monetary or financial benefits is an external motivation that drives a participant to work hard to get expected rewards (Zheng, Li, & Hou, 2011) .The direct compensation can be of relatively minor value, such as a free product or a small cash prize or payment that is likely to be used by the crowd to make a living.On the other hand, one of the intrinsic motivations for individuals to participate in a crowdsourcing process is learning motivation.Learning motivation, also called feelings of personal mastery, gaining additional knowledge or skills, competence, and fulfillment (Hars & Ou, 2002;Please, Leimeister, Huber, Bretschneider, & Krcmar, 2009) is often discussed in the open source context.Csikszentmihalyi (1991) argues that "the best moments usually occur when a person's body or mind is stretched to its limits in a voluntary effort to accomplish something difficult or worthwhile."Learning theory proposes a maximum level of challenge for each specific level of skill.If the task difficulty increases and passes a certain level, participants would feel a lack of control over the environment and become anxious and frustrated.Also, if the challenges are too little, the individual loses interest.While designing a crowdsourcing project, it is essential to consider this optimum level of task difficulty and its impact on participants' motivation to participate.
Another intrinsic incentive that can motivate individuals to participate in a crowdsourcing project is networking.The literature shows that in an online environment, the motivation to build personal and professional relationships among the members will contribute to creating a sense of belonging and therefore increase participant's level of effort.Additionally, Hars and Ou (2002) found self-advertisement or self-marketing as one of the main motivations to participate in open source projects for those seeking new job opportunities.Leimeister, Huber, Bretschneider, and Krcmar (2009) described this motivation as an opportunity for demonstrating capabilities and skills; a form of self-advertising for those seeking new job opportunities.
Enjoyment is another intrinsic motivation for individuals to participate in a crowdsourcing project.The pleasure experienced by participants may increase a person's tendency to repeat that task and strengthen their feeling of active participation and improve their performance.Task enjoyment is considered to compel the initiation of activity and increase the persistence of task performance (Bandura, 1978).Individuals usually engage in tasks because, for them, the activity is considered enjoyable (Lumpkin & Achen, 2018).In the domain of virtual innovation, it has been shown that enjoyment motivates online community members to contribute to tasks (Lumpkin & Achen, 2018).Participants who are fueled by enjoyment experience a rewarding activity.Other types of intrinsic incentives include: Altruism, knowledge sharing, etc. (Hosseini, Shahri, Phalp, Taylor, & Ali, 2015) Depending on the task, a right mix of incentives is necessary to motivate the right crowd to participate in the crowdsourcing process.The incentive mechanism may include only intrinsic, only extrinsic, or a combination of both intrinsic and extrinsic incentives.

What is the task?
The crowdsourcing approach can be applied in various contexts and for different tasks.Previous research has shown that human decision-making strategies change to adapt to task requirements (Payne, Bettman, & Johnson, 1992).Therefore, it is important to understand and identify the functions and characteristics of the task in a crowdsourcing project.Specifically, in the next sections, tasks' functions, participation modes, and complexity are examined.

Functions
It is important that the crowd-sourced tasks have clear functions or objectives.The crowd needs to carry out the resolution of a problem through the process.In crowdsourcing activities, tasks' functions seem to gain an unprecedented power due to the fading of time, space and even organizational boundaries (Brabham, 2008a).There are different classifications of the functions of crowdsourcing applications in the literature.Howe (2006)  It should be noted that crowdsourcing is a complex mechanism and often involves more than one of these functions.

Participation modes
The crowdsourcing process can either be integrative or selective (Schenk & Guittard, 2011).
Crowdsourcing process can be categorized based on the value of individual's contributions.In some cases, individuals' contributions are valuable only when combined with other contributions (integrative crowdsourcing).Since the issue is to pool complementary input from the crowd, individual elements have very little value per se, but the amount of additional information brings value to the process.In this case, crowdsourcing enables the initiators to gather a variety of content and offers access to multiple and complementary sources of information.Integrative crowdsourcing can be relevant to crowd labor.Initiators seeking to implement integrative crowdsourcing should be aware of integration challenges.The data collected from various sources might be incompatible or redundant if no rules or guidelines are designed.
In other categories of crowdsourcing, initiators have to choose an input from the set of options that the crowd has provided (selective crowdsourcing).Selective crowdsourcing may be a way to find candidate solutions if the initiator has a specific need.It enables access to individuals' problem-solving skills.For instance, a firm facing an R&D problem may rely on competencies from the crowd in order to solve the problem.In selective crowdsourcing generally, only the best solution is rewarded.The objective of the crowdsourcing process is a crucial factor in determining the selective or integrative nature of crowdsourcing process.

Complexity
Task Complexity is measured regarding analyzability of the instructions provided while analyzability refers to the availability of concrete knowledge about task activities and the degree of complexity of the search process in performing the task (Chang, Chang, Paper, 2003).Task complexity increases by the decrease in analyzability.Crowdsourcing tasks can be classified into three categories: Simple, Moderate, and Complex (Rouse, 2009;Schenk & Guittard, 2011).
Simple tasks are structured tasks that can be broken into a series of steps and often have a single acceptable answer or a defined range of acceptable solutions.These are routine and often timeconsuming tasks that can be performed by an individual with low or moderate level of skill and training.Examples include tagging images, identifying handwriting, and some community research projects.Moderate tasks involve a higher level of difficulty and can be more challenging to evaluate.Examples include designing a T-shirt or logo, user-generated advertisement, photography, or performance of more complex tasks in a shared scientific effort.
Finally, complex tasks are less structured, non-routine tasks and can only be performed effectively by crowd members with in-depth knowledge and experience and are hard to evaluate.
Examples include generating product ideas, predicting market trends, or solving complex problems.Task complexity is among the most critical factors that define a crowdsourcing process.

How does the crowd perform the task? (Technology platform)
Technological advances in the internet-enabled virtual environment have led to advances in the crowdsourcing platforms.Crowdsourcing initiators may design and manage their platforms.Or, the technology platform can be developed by a third-party company and operated by project initiator.Free communication and collaboration platforms such as email, Skype, and Dropbox can also be used in accomplishing crowdsourcing projects.The technology platform and its capabilities differ based on the initiator's goal and the characteristics of the task.Based on the theory of task-technology fit (TTF) by (Davis, 1989), IT is more likely to have a positive impact on individual's performance if the capabilities of the IT match the tasks that they perform.
Internet-based technologies enable the crowd to form, facilitate, and optimize the continued interaction and ultimate solution to a crowdsourcing problem.Crowdsourcing is enhanced by several factors relating to today's Internet: global reach, increased interactivity and collaboration capabilities, increased speed, anonymity, and the ability to use media from other communication modes.The Internet is a worldwide medium that reduces the constraints of physical distances and provides a platform for a large number of individuals from all over the world to interact and collaborate with each other.Advances in virtual environments reduce the cost and increase the speed and persistence of individuals' engagement in the process.
Media richness is a factor that needs to be considered while choosing the technology platform for a specific crowdsourcing project.Media richness is defined based on the following characteristics: the use of multiple media, feedback immediacy, and multiplicity of cues.
Feedback immediacy can be described as timeliness of providing feedback through media while multitude of cues is the ability of a media to convey information via multiple signals and channels, including physical presence, voice tones, body gestures, words, numbers, and graphic symbols (Kahai & Cooper, 2003;Lim, Matros, & Turocy, 2014).
Media richness decreases moving from face-to-face communication to video and to audio and to text communication.In order to measure media richness, Clark and Brennan (1991) described several characteristics that determine the nature of communication including co-presence (members occupy the same physical location), visibility (members can see one another), audibility (members can hear one another), co-temporality (communication is received at the approximate time it is sent), simultaneity (members can send and receive messages simultaneously), sequentially (members' speaking turns to stay in sequence).Face-to-face communications enable all of the media richness characteristics mentioned above.Real-time video conference provides all of the attributes except for the co-presence.In a video conference setting, distributed members exchange live video as well as audio and text.Audio calls lack both the co-presence and visibility.Electronic dialogue environment, where users exchange messages via text in real-time, lack of co-presence, clarity, and audibility.Text-only communications and emails do not provide any of the media richness characteristics listed above.Overall, the internet-enabled technologies facilitate the crowdsourcing projects by enabling individuals from all around the globe to engage, interact, communicate, and collaborate more efficiently with higher speed, and less cost.However, it is essential for crowdsourcing initiators to pick the right technology platform with the right characteristics that fit the crowdsourcing problem at hand.
Table 1 shows the elaborated version of the conceptual framework with potential answers to each of the key questions: Who -Why -What -How.It summarizes the above sections, describing different dimensions of a crowdsourcing process.

Crowdsourcing applications examples
To understand the different components in our framework, we describe and compare real-life crowdsourcing projects (Table 2).We applied the conceptual framework presented in the current study to identify the components and characteristics of these crowdsourcing applications.Kaggle (https://www.kaggle.com/),founded in 2009, is a crowdsourcing platform that provides consulting services.It is designed for predictive modeling and analytics competitions in which statisticians and data miners from all over the world compete to create the best models.The initiators are for-profit companies or individual users who post their data and a description of the problem and offer monetary and non-monetary prizes.The nature of the task is complex, and various strategies and techniques can be applied to any predictive modeling task.
Galaxy Zoo, first launched in 2007, is part of the Zooniverse, a group of "people-powered" research (https://www.zooniverse.org/).Galaxy Zoo is a set of crowd-sourced projects which invites people from all around the world to assist in the identification and classification of more than 900,000 galaxies based on a large number of available galaxy images.This project relies on the help of members of the public with general skill sets to help in scientific research.This project benefits the community by a better understanding of galaxies and categorizing them into different classifications.The project is based on the participation of volunteers who are intrinsically motivated to participate.
The X-Culture projects (https://x-culture.org/)are crowd-sourced business consulting competitions initiated by for-profit organizations.Individuals (who are mostly students) from all around the world participate semi-annually in solving highly complex indivisible business challenges during an eight-week period.Participants need to have specialized skills.The participation model is selective where the best solutions are awarded intrinsically as well as externally by the monetary prize, certificate, employment networking opportunities, etc. Initiators communicate with the crowd through webinars posted on the platform website and emails sent from the platform Admin.Crowds are supposed to use any free/paid communication and collaboration tools.
Amazon Mechanical Turk (https://www.mturk.com/),launched in 2005, is a crowdsourcing marketplace designed to provide businesses and individuals access to a diverse, on-demand workforce.The tasks posted on this platform are usually simple and repetitive tasks with a high variety of potential outcomes (e.g., identifying objects in a photo or video, performing data deduplication, transcribing audio recordings or researching data details).The crowdsourcing process can be designed as a competition or a collaboration.Participants complete tasks in exchange for a monetary payment set by employers.
My Starbucks Idea (https://ideas.starbucks.com/) is an open innovation platform launched in 2008 with the goal of increasing engagement with customers and giving customers insight into what the company is doing.After an idea is submitted, other members of the crowd can vote or comment on it.Top voted ideas on the website and the ones picked by a team of experts could then be adapted and implemented by the company.My Starbucks Idea has benefited both Starbucks and the customers.My Starbucks Idea helps the company stay aware of the customers' needs and it creates loyalty, customer engagement, transparency, and trust.It also provides a platform for customers to share their voices with the company and maybe see the implementations of their ideas.
Understanding the main components of these projects will make it easier to study further the process which facilitates the improvements in participants' performance.

Summary and discussion
The conceptual framework integrates and extends the current crowdsourcing literature and applications to advance the field.A crowdsourcing project can be initiated by for-profit or nonprofit organizations, government-related institutes, or individual persons.The outcome of a crowdsourcing project can be helpful for a community, for individuals (organizations or persons), or it can be designed to benefit both a community and individuals.The crowd can be selected from different sources and with various backgrounds and skill sets.Individuals can be recruited from the general public, a specific community, or even a combination of both.
Participants could be required to have a range of skill sets from general, to specialized, and to situational.Incentives of participants in crowdsourcing vary from intrinsic to extrinsic motivations.Depending on the task, crowdsourcing initiators must distribute the right mix of incentives to motivate the right crowd to participate.The recent advances in the communication and collaboration technologies and virtual environments have enabled crowd-based projects more than ever.The technology platforms used for a crowdsourcing project can be developed and managed by the project initiator or a third party.Free online tools and applications can be used to start and maintain an IT-enabled crowdsourcing project.The capabilities of the IT should match the tasks that they perform and there should be a clear understanding and guideline for the tools used in a crowdsourcing project.The crowdsourcing approach can be applied in various contexts and for different reasons.It is essential to understand and identify the functions and characteristics of the task.In this study, the most critical dimensions of tasks in a crowdsourcing project are defined as functions, complexity, and participation modes.Crowdsourcing task functions are classified into four broad categories of crowd creation, crowd wisdom, crowd labor, and crowdfunding.Crowdsourcing tasks can be classified into three types: Simple, Moderate, and Complex.The process can either be integrative or selective.Task characteristics and its attributes shape most of the components involved in the process.
Before the start of any crowdsourcing projects, initiators need to identify the different factors and the key players involved in the process to optimize the quality of outcome.The current study aims to review the current crowdsourcing literature and applications to identify and synthesize the key features of the process and their characteristics into a conceptual framework.
It is important to emphasize the importance of interrelations between different components of the conceptual framework.As a demonstrative example of such interrelationship, one can design the incentive of a crowdsourcing project based on task complexity.For simple tasks, incentives are typically non-financial (Ye & Kankanhalli, 2013).Participation in these tasks is usually voluntary or micro-paid.If the task is voluntary (such as Galaxy Zoo projects), the crowdsourcing project's initiators should rely on other incentives, such as trying to make the task fun, fulfilling solvers' needs, and invoking their sense of achievement by emphasizing the tasks' importance (Kaufmann, Schulze, & Veit, 2011).However, for time-consuming and repetitive tasks (such as the jobs posted on Amazon Mechanical Turk), initiators are required to provide small monetary incentives.
For moderate difficulty tasks, incentives are usually both monetary and non-monetary.
Participants in this type of task are self-motivated to differentiate themselves, to provide novel solutions, and to protect rather than share their knowledge.For example, in an InnoCentive project (InnoCentive is an open innovation company, https://www.innocentive.com/),substantial financial rewards are considered to motivate individuals from different domains to crack the challenges that cannot be solved by a company's internal talents.
For complex tasks, participants are likely to expect monetary rewards for their efforts and time involved.However, for complex tasks, there is a risk that the substantial time and effort invested in problem-solving may be wasted if the solution does not get accepted.Also, for completing the complex tasks, participants are usually required to have specialized skills and knowledge about a specific area.Therefore, providing both financial and non-financial incentives for the tasks in this category is found to motivate the crowd to participate and continue the process (Ye & Kankanhalli, 2013).For example, enjoyment in addressing challenges or a sense of achievement may compensate for the risks involved in participation.Participants may also get motivated by peer reputation enhanced by the task completion (e.g., TaskCn, a Chinese crowdsourcing website in which a user offers a monetary award for a question or task and other users provide solutions to compete for the award, http://www.taskcn.com/).Methods like gamification can also be applied to this category of crowdsourcing projects.It is also important to note that for tasks in this category, it may not be feasible to obtain final desirable solutions through one-time process (Morgan & Wang, 2010).These tasks may require participants' further collaboration for refining and implementing the initial proposal and initiators are generally required to provide feedback throughout the process.
While it is beyond the scope of our framework, it should be noted that ethical issues associated with crowdsourcing are an important topic of research and other researchers have examined the ethical issues associated with designing a crowdsourcing project in detail.For example, Fort and colleagues (Fort, Adda, & Cohen, 2011) noted the ethical implications related to the low amount of pay for the participants on Amazon Mechanical Turk.They discussed the importance of considering the ethical dimension for resource management and system evaluation in crowdsourcing projects.

Implications
This study provides several contributions to the crowdsourcing literature and applications.Our approach is different from other models such as the "input-process-output" model of classifying crowdsourcing components which may suggest one-directional relationships between the building blocks of a crowdsourcing project.

Practical implications
For crowdsourcing applications, identifying the key roles and activities involved in a crowdsourced project is crucial.Being able to differentiate various crowdsourcing applications based on the dimensions defined in the conceptual framework, helps crowdsourcing project initiators and developers to design the process better and manage it.Moreover, to improve crowdsourcing performance, the first step is to understand its characteristics that might influence the crowd's experience throughout the crowdsourcing process.
It is important that the crowd's skill sets, the incentives, and the technology platform are identified based on the crowdsourcing conceptual framework, before the project is initiated.The project initiator should also characterize the crowd members' required level of skill and background, and also establish the right form of incentive.The initiator should also identify the participation mode and provide task instructions.The crowdsourcing task instructions should be clear and analyzable.The initiator should pick the platforms to execute the project which can be designed and managed by the initiator or third-party companies.Our developed framework can serve as a guide and roadmap for project initiators to consider and plan for the above-mentioned factors in order to maximize the potential for success of their CS project.

Theoretical implications
For theory, the primary contribution of this study is in the identification of the main components of crowdsourcing as a legitimate, IT-enabled form of problem-solving to help understand this new phenomenon better.The framework provides a strong foundation to develop theories related to each of the distinct areas of a crowdsourcing process: Who, Why, What and How and the domain.Overall, this research provides a better understanding of an IT-enabled CS process and examines the characteristics of each of its main components that might influence crowd's participation behavior and performance in a business context.The study provides a basis for further analysis to understand the performance of crowdsourcing projects.Future research can examine the conceptual framework that we developed in the present study for different tasks and contexts.By contexts, we mean specific organizational and environmental conditions surrounding the crowdsourcing task.The starting point may be to more finely categorize various tasks and contexts.Then for each task and context, the followings research questions can be addressed: what kind of people will do it best (Who?), what needs to be done to attract people to the project (Why?), and what technology platform will enhance participants' engagement and quality of work (How?).There is also the potential for future research to further refine or expand the conceptual framework and examine interrelationships between the various dimensions of the framework.

Conclusion
IT-enabled crowdsourcing is an emerging problem-solving approach and a new frontier for IS research that has reached out beyond the traditional organizational boundaries to a much broader context.Since the term crowdsourcing is used for different groups of activities that take on various forms, it is challenging to define what crowdsourcing is and what its characteristics are clearly.We offer a framework to identify the main components involved in an IT-enabled crowdsourcing process.The conceptual model provides for a full, yet parsimonious, consideration of all the essential elements included in the process and their characteristics.We hope that this leads to a more comprehensive understanding of the crowdsourcing projects and sets up the groundwork for future more comprehensive studies.

Appendix
Table A1.Relevance and differences of the existing crowdsourcing models to the proposed conceptual framework.

Description of the study Components integrated from the study into the current conceptual framework
Differences with the current conceptual framework (Rouse, 2009) proposed a taxonomy of crowdsourcing considering three dimensions of supplier capabilities/nature of the task, distribution of benefits, and forms of motivation.
What -Task Who -Benefit, Initiate Why -Motivation This study described a taxonomy of crowdsourcing in order to clarify the meaning of the term in a business setting.In this study, crowdsourcing was seen as an alternative form of outsourcing.The article did not discuss different types of the crowd (Who -Perform) and the crowdsourcing platforms (How -Platform).(Doan, Ramakrishnan, & Halevy, 2011) provided a global picture of crowdsourcing systems on the Web based on the nature of collaboration, architecture, crowd recruitment, and evaluation of users' contributions.

Who -Perform, Benefit, Initiate How -Platform
This study discussed the management of crowd-sourced systems.Although the article provided a general overview of the crowdsourcing system components, they did not discuss motivations and incentive aspects of crowdsourcing (Why -Motivation).The study focused mostly on the process of crowdsourcing and crowd recruitment and retention challenges.(Geiger, Seedorf, Nickerson, & Schader, 2011) identified four types of crowdsourcing mechanisms exclusively within the organizational context based on the following dimensions: preselection of contributors, accessibility of peer contributions, aggregation of contributions, and remuneration for contributions.
What -Task Who -Perform, Benefit, Initiate Why -Motivation How -Platform This study focused exclusively on crowdsourcing from an organizational perspective.It included the aspects that "can be directly influenced by a crowdsourcing organization".Crowdsourcing in a nonbusiness domain was not considered.(Yuen et al., 2011) categorized the literature on crowdsourcing into four groups: research that study the crowdsourcing applications (grouped into four categories of voting system, information sharing system, game and creative system), research on algorithms and system design, research that investigates performance aspect of crowdsourcing (categorized into user participation, quality management and cheating detection), and research that exploit datasets available on the web.
What -Task Who -Perform, Benefit, Initiate Why -Motivation This study aimed to categorize crowdsourcing literature based on the applications, algorithms, performances, and datasets.The article did not include a detailed discussion on different types of technological platforms (How -Platform).(Pedersen et al., 2013) provided a comprehensive review of the crowdsourcing research in the IS field.
What -Task Why -Motivation Who -Perform, Benefit, Initiate How -Platform The authors developed a crowdsourcing conceptual model based on the traditional "Input-Process-Output" model.Elements of this conceptual model include: problem, people (problem owner, individual, and crowd), governance, process, technology, and outcome.Application of the input-process-output model may suggest a one-directional relationship between the building blocks of a crowdsourcing project.Our application of CI genome concept to crowdsourcing can include bi-directional and multi-directional relationships among different components of a crowdsourcing project.(Hosseini, Phalp, Taylor, & Ali, 2014) analyzed the literature and proposed a taxonomy of crowdsourcing to represent the different configurations of crowdsourcing.

Who -Perform, Initiate Why -Motivation What -Task How -Platform
The authors provided a taxonomy of crowdsourcing where they mainly focused on the following components of crowdsourcing: the crowdsourcer, the crowd, the crowdsourced task and the crowdsourcing platform.The model is different from our framework, where motivation is a main component of the model.Also, the beneficiary of the crowdsourced task can be different from the crowdsourcer in our model.(Zhao & Zhu, 2014) did an extensive review of the crowdsourcing research and identified three primary research foci: the conceptualization focus, the system focus, and the application focus.They also identified several crowdsourcing research opportunities for future investigations: participant, organization, and system.
Who -Perform, Benefit, Initiate Why -Motivation What -Task How -Platform This article did a comprehensive review of the crowdsourcing research.We extended the crowdsourcing dimensions mentioned in this study.(Cullina & Morgan, 2015) provided a taxonomy to address the question of how a crowdsourcing process is measured.This research-in-progress paper examined crowdsourcing at the operational level.The authors proposed metrics for the following parts of a crowdsourcing process: crowd membership, crowd Measurements of: Who -Perform, Benefit, Initiate Why -Motivation What -Task How -Platform This article identified the parts of a crowdsourcing process that can be measured and then categorized operational metrics to facilitate deployment in practice.In this study, the focus is on the measurements of the platform, crowd incentivisation, and crowd interactions & outcomes.crowdsourcing process and not identifying its different components.(Ghezzi, Gabelloni, Martini, & Natalicchio, 2017) reviewed the crowdsourcing process applying the "Input-Process-Output" framework: input (problem/task); process (session management; problem management; knowledge management; technology); and outcome (solution/ completed task; seekers' benefits; solvers' benefits).
Who -Perform, Benefit, Initiate Why -Motivation What -Task How -Platform This study applied the input-process-output framework to synthesize the crowdsourcing literature in management journals.Application of the input-process-output model may suggest a one-directional relationship between the building blocks of a crowdsourcing project.Our application of CI genome concept to crowdsourcing can include bi-directional and multi-directional relationships among different components of a crowdsourcing project.(Assis Neto & Santos, 2018) reviewed workflow management in crowdsourcing projects considering task execution, quality management, and platform usage.

Who Perform, Initiate Why -Motivation What -Task How -Platform
This study focused on the quality management of a crowdsourcing workflow.It aimed to understand the concerns of design and execution of crowdsourcing projects, and not necessarily identifying all of its different components.
(Surowiecki, 2004) (2013) (2008)tions of crowdsourcing as crowd creation, crowd labor, crowd wisdom, and crowdfunding.Subsequently,Hossain and Kauranen (2014)classified crowdsourcing applications into six categories of idea generation, microtasking, open source software, public participation, citizen science, citizen journalism, and wiki.Kleemann, Voß, and Rieder (2008)focused on the applications of crowdsourcing in business and listed several categories: product development and configuration, product design, competitive bids on specifically defined tasks or problems, permanent open calls, community reporting, product rating, and customer-to-customer support.Whitla (2009)surveyed a more specific domain in the marketing-related literature and found that there are three areas in which firms actively use crowdsourcing: product development, advertising and promotion, and marketing research.Also,Zhao and Zhu (2012)classified business crowdsourcing functions into four categories of design and development, idea and consultation, test and evaluation, and others.In other study,Ye and Kankanhalli (2013)identified three main crowdsourcing approaches as open call for participation, open call for solutions, and open call for candidates.Based on the literature, we classify the business and non-business crowdsourcing functions listed in the referenced literature into four broad categories: crowd creation, crowd wisdom, crowd labor, and crowdfunding (the advertising, promotion, marketing research introduced by Whitla (2009) fall under two categories of crowd creation and crowd wisdom based on the level of task difficulty).Crowd creation refers to a contribution via a new design, product, concept, or solution.The output from crowd creation is an end-product, intellectual or physical, that has a tangible value to others.These tasks require time, effort, and a high level of specialized skills to complete (e.g., idea generation, product development and configuration, product design, competitive bids on specifically defined tasks or problems, new product development, open call for solutions, consulting problems, research and design projects).Crowd wisdom refers to the projects that utilize individuals' cognition, coordination, and cooperation through internetmediated technologies(Surowiecki, 2004).The key in this category of tasks is the size of the crowd (e.g., public participation, citizen science, citizen journalism, wiki, permanent open calls, community reporting, customer-to-customer support, open call for participation, online challenges by brands).Crowd labor denotes a contribution via activities that range from simple to specialized tasks.Tasks in this category usually do not need very specialized skills to complete.The jobs are crowdsourced to save money and time (e.g., microtasking, open call for candidates, test and evaluation, voting, judging, filtering content, transcribing audio, translating, labeling images, product rating).And finally, crowdfunding refers to asking the crowd to invest in the activities of individuals or groups through online open announcements.
The term crowdsourcing is vague and is used for broad categories of applications and takes on different forms.It is challenging to define what crowdsourcing is and what its characteristics are clearly.The conceptual framework expands our understanding of crowdsourcing phenomenon as a new model for problem-solving.The framework helps to differentiate various applications of crowdsourcing based on the four fundamental dimensions of Who, Why, What, and How.The novelty of the current study is rooted in combing the ideas of the I-model and CI genome, superimposed on the foundation of a comprehensive review of the crowdsourcing literature.