1. case studies
1.1. Ideastorm.com
1.1.1. The goal of this initiative was to hear what new products or services Dell's customer's would like to see Dell develop.
1.2. Threadless
1.3. Adidas, BBC, BMW, Boeing, Ducati, and Muji
1.4. Bamed/MAM Group (http://www.mambaby. com)
1.5. threadless, dell, startbucks
1.6. Ideastorm
1.7. IStockphoto
1.8. Crowdsourcing platforms
1.8.1. Me- chanical Turk, Turkit, Mob4hire, uT- est, Freelancer, eLance, oDesk, Guru, Topcoder, Trada, 99design, Inno- centive, CloudCrowd, and Cloud- Flower. Using
1.9. list of crowdsourcing projects
1.9.1. http://en.wikipedia.org/wiki/List_of_crowdsourcing_projects
1.10. Case studies
1.10.1. has an algorithm to identify the percentage of contribution of each person
2. Research questions
2.1. What are the success factors of a thriving crowdsourcing practice?
2.1.1. Transparency
2.1.2. reward and reputation system
2.1.3. Creating a learning and thriving environment
2.1.3.1. Users would know what the cost structure of the company is and what knowledge feasibility
2.1.4. role of perceived fairness in crowdsourcing communities
2.2. Is crowdsourcing a really powerful method for innovation process?
2.2.1. yes but not in terms of feasibility
2.2.2. some argues that they have low potential
2.2.3. some argues that in terms of novelty, they are good
2.2.4. some argues that in terms of customer benefits, they are okay
2.2.5. some argues that they might be good for the short term but not for the long run
2.2.6. Some argues that UNDER UNCERTAIN SITUATION, they are okay
2.2.7. Some argues that there is a need to a strong internal department that have enough competency in design and technological aspect
2.2.8. Some argues that doesn't worth it compared to its cost
3. it depends on which type of bussiness you want to use it. it geniunely good for the products that have lower front-head cost of production.
4. Thesis plan
4.1. 1. Build a model and test it over a company data
4.2. 2. Build a model through collaboration with a company
4.3. Comprehensive state of the art, Interview or survey
4.4. Set the scope based on SNOVO's (from tangibility to intangibility), do the comprehensive study, recognize the problems related to this field, and improve one or two problems
5. Success criteria of my research We define a criterion as the desired value of the factor the research project sets out to understand and/or influence as described in the research goal. so IMPROVE the crowdsourcing practice is the success criteria.
5.1. improving the crowdsourcing practice
5.2. add more clarity to the exisiting literuature
6. Design problem
6.1. Dislike button
7. Literature review
7.1. Lakhani et al. (2007)
7.1.1. crowdourcing, a good tool for Problem Solving success
7.2. Von hippel (2005)
7.2.1. user innovation with highly commercial success
7.3. Shah et al. (2000,2003, 2006)
7.3.1. user innovation with highly commercial success in 4 different field of sports
7.4. Urban and von Hippel, 1988, Morrison et al., 2000; Franke et al., 2006 and Olson and Bakke, 2001
7.4.1. user innovation with highly commercial success
7.5. Poetz and Schreier (2010)
7.5.1. user innovation in baby products - high in novelty low in feasibility
7.6. Franke and Klausberger (2009)
7.6.1. role of perceived fairness in crowdsourcing communities
7.7. Bayus (2010)
7.7.1. need for a better understanding of the reward and feedback mechanisms in crowdsourcing systems
7.8. haung yan singh (2011)
7.8.1. relation of crowdsourced ideas to cost implementation and potential for novelty
7.8.1.1. Since potential perception is updated by the observation of number of votes that the users’ past ideas receive, users with higher perception about their own potential are by nature those who are proven to have better ability of generating "blockbuster" ideas. In
7.8.1.2. With learning, users can obtain more precise perception of firms’ cost structure, and better understanding of the potential of their own ideas.
7.9. Marion K Poetz (2012)
7.9.1. Can Users Really Compete with Professionals in Generating New Product Ideas?
7.10. Schulze and Hoegl (2008, p. 1742)
7.10.1. limited insights with regard to the “ideal” process of idea generation
7.11. (e.g., Amabile, Barsade, Mueller, and Staw, 2005; Goldenberg, Lehmann, and Mazursky, 2001; Majchrzak, Cooper, and Neece, 2004; Schulze and Hoegl, 2008)
7.11.1. New product development
7.12. Bennett and Cooper (1981, p. 54)
7.12.1. a truly creative idea for a new product “is very often out of the scope of the normal experience of the consumer.”
7.13. (Leonard and Rayport, 1997)
7.13.1. users might be too accustomed to current consumption conditions (i.e., the present), thus preventing them from predicting and shaping the future (Leonard
7.14. (Jeppesen and Frederiksen, 2006)
7.14.1. users might have reasonably good new product ideas
7.15. Bagozzi and Dholakia, 2006; Lakhani and von Hippel 2003; Lerner and Tirole, 2002, 2005; Pitt,Watson, Berthon,Wynn, and Zinkhan, 2006
7.15.1. one of the most extreme and most frequently cited examples of user innovation is open-source software
7.16. Agerfalk and Fitzgerald, 2008; Howe, 2006; Pisano and Verganti, 2008; Surowiecki, 2004
7.16.1. Describing crowdsourcing
7.17. Hienerth, Poetz, and von Hippel, 2007; Lilien, Morrison, Searls, Sonnack, and von Hippel, 2002; von Hippel, 1986
7.17.1. Collaborate with LEAD USER
7.18. Lakhani, Jeppesen, Lohse, and Panetta, 2007; Piller andWalcher, 2006
7.18.1. crowdsourcing relies on a self-selection process among users willing and able to respond to widely broadcast idea generation com- petitions
7.19. Terwiesch and Ulrich (2009)
7.19.1. across industries, about a quarter of innovation opportunities tend to come from interactions with customers and new customer requirements
7.20. Cooper, 2001; Crawford and Di Benedetto 2006; Ulrich and Eppinger, 2008; Urban and Hauser, 1993
7.20.1. CO-CREATION WITH USERS -- needs-based information
7.21. Dahan and Hauser, 2002; Griffin and Hauser, 1993
7.21.1. “voice of the customer”
7.22. von Hippel, 2005
7.22.1. solution-based information
7.23. Amabile (1998);Ulrich and Eppinger (2008) and Ulrich (2007)
7.23.1. A NEED to have a strong internal segment
7.24. Helfat, 1994; March, 1991; Martin and Mitchell, 1998; Stuart and Podolny, 1996; von Hippel, 1994)
7.24.1. firms that rely too heavily on their internal expertise might be blocked from finding alternative, potentially more successful solutions
7.25. Katila and Ahuja (2002)
7.25.1. the extent to which a firm explores external knowledge is positively related to successful new product innovation.
7.26. Baldwin, Hien- erth, and von Hippel (2006)
7.26.1. user innovators can, under certain conditions, can bridge periods of uncertainty in early phases of industry life cycles
7.27. Franke and Shah, 2003; Franke, von Hippel, and Schreier, 2006; Morrison, Roberts, and von Hippel, 2000)
7.27.1. USER Innovation - up to 30% of the user populations
7.28. Lilien et al. (2002)
7.28.1. USER INNOVATIVENESS beats PROFESSIONAL INNOVATIVENSS
7.29. Urban and von Hippel (1988)
7.29.1. USER INNOVATIVENESS beats PROFESSIONAL INNOVATIVENSS
7.30. Kristensson et al. (2004)
7.30.1. professionals are more driven by a convergent thinking style that results in less novel ideas.
7.30.2. users are divergent thinkers. they don't have worry about the cost's implementation or technological feasibility of the ideas
7.31. Ogawa and Piller (2006)
7.31.1. first to provide anecdotal, real-world evidence indicating that user ideas generated in the course of a crowdsourcing process might also hold commercial potential
7.32. Amabile et al., 2005; Franke et al., 2006; Kristensson et al., 2004; Moreau and Dahl, 2005)
7.32.1. the quality of the ideas captured by crowdsourcing should be evaluated based on novelty, value of the idea in terms of its ability to solve the underlying problem, FEASIBILITY
7.33. Poetz, Marion K. Schreier, Martin (2012)
7.33.1. ideas created by profession- als score significantly lower in terms of novelty than ideas created by the users
7.33.2. professional ideas are attributed significantly lower customer benefit compared with user ideas
7.33.3. professional ideas are attributed significantly lower customer benefit compared with user ideas
7.33.4. professional ideas also score significantly lower than user ideas on the overall quality index (the three-way interaction term novelty * customer benefit * feasibility)
7.33.5. users can, in fact, provide valuable needs- based as well as solution-based information in the idea generation stage of the NPD process
7.33.6. the best ideas overall tended to be more heavily concentrated among users compared with a firm’s professionals
7.34. Bayus 2010
7.34.1. LESS clear is the nature of crowd creativity over time
7.34.2. The crowding-out effect occurs when extrinsic rewards undermine intrinsic motivation
7.34.3. individuals in the crowd are unlikely to repeat their early creative success once their ideas are recognized as being creative
7.34.4. Effectively managing crowd creativity thus focuses on continually getting new ideators who then generate creative ideas for a limited time and giving appropriate feedback that does not undermine consumers’ intrinsic motivation for proposing creative ideas.
7.34.5. Three hypotheses involving crowdsourcing and individual creativity over time are developed
7.34.5.1. H1:An individual’s likelihood of proposing a creative idea is positively related to the quantity of ideas they generate.
7.34.5.2. H2An individual’s likelihood of proposing a creative idea is positively related to their exposure to others’ ideas.
7.34.5.3. H3:An individual’s likelihood of proposing a creative idea is negatively related to their past success in generating creative ideas.
7.34.6. Because the feedback from many crowdsourcing systems is more judgmental than developmental, external feedback from these organizations will most likely be viewed as being controlling.
7.34.7. individuals tend to propose less divergent ideas as the number of ideas they submit increases. Thus, prolific ideators are less likely to propose divergent ideas than consumers with only a few ideas.
7.35. Osterloh and Frey 2000; Frey and Jegen 2001
7.35.1. Motivation crowding is a well-established social psychology theory that has wide empirical supporT
7.36. Shalley, et al. 2004
7.36.1. non-monetary payments such as “gold-stars” or external evaluation
7.37. (Audia and Goncalo 2007; Conti, et al. 2010; Goncalo, et al. 2010)
7.37.1. past creative success causes individuals to generate even more ideas, but that these later ideas do not break new ground
7.38. (Shalley, et al. 2004) Amabile 1996; George 2007
7.38.1. Ideas are considered to be novel if they are relatively new compared to other ideas available to the firm and useful if they are potentially valuable to the organization in the short or long run
7.39. Helfat 1994; Sorensen 2002
7.39.1. both incremental and divergent ideas can be important (Helfat
7.40. (Simonton 1997; Huber 2000
7.40.1. the vast majority of (significant) contributions are made by only a few individuals. Importantly, all of these results seem to apply to any form of creativity
7.41. Simonton 2003)
7.41.1. quality is a probabilistic function of quantity (
7.42. Amabile 1988; 1996
7.42.1. an individual’s creative productivity is also a function of their domain-relevant knowledge
7.42.2. “People will be most creative when they feel motivated primarily by the interest, enjoyment, satisfaction, and challenge of the work itself… people who are primarily intrinsically motivated will be more likely to generate truly creative ideas than people who are primarily extrinsically motivated” (Amabile 1988, pp142-143)
7.43. Shane 2000
7.43.1. Individuals that are able to identify novel business opportunities for new technologies also tend to have broad prior knowledge
7.44. March 1991
7.44.1. theory of exploration-exploitation
7.45. Audia and Goncalo (2007)
7.45.1. employ exploration-exploitation theory to understand creativity among an internal staff of professional inventors. They empirically show that past success in obtaining patents leads inventors to be more productive in generating patents, but these patents tend to be less divergent over time.
7.46. Lakhani, et al. 2007; Brabham 2008)
7.46.1. Reasons given for participating in a crowdsourcing community a desire to have fun, develop creative skills, take on a challenge, and solve problems
7.47. Ryan and Deci 2000)
7.47.1. Intrinsic motivation is defined as engaging in an activity for its inherent enjoyment and satisfaction rather than because of external pressures or rewards
7.48. (Deci 1975; Hackman and Oldham 1980)
7.48.1. Intrinsic motivation is of great interest since it can be self-generating and self-perpetuating
7.49. Hackman and Oldham 1980
7.49.1. DEFINITION OF complex tasks
7.49.1.1. those with high levels of variety, significance, identity, autonomy, and feedback;
7.50. Amabile 1988; Ryan and Deci 2000
7.50.1. But creative ideas only occur when an individual finds the task intrinsically interesting and is working in an environment where extrinsic constraints do not undermine intrinsic motivation
7.51. Deci, et al. 1999; Osterloh and Frey 2000; Frey and Jegen 2001
7.51.1. the notion that an external reward can crowd-out intrinsic motivation has wide empirical support
7.52. Deci and Ryan 1985; Ryan and Deci 2000
7.52.1. The social psychology foundation for motivation crowding rests on Cognitive Evaluation Theory (CET)
7.53. Osterloh and Frey 2000)
7.53.1. extrinsic rewards can have two possible aspects: a “controlling” and an “informing” aspect
7.54. Deci, et al. 1999)
7.54.1. virtually every type of expected tangible reward contingent on task performance undermines intrinsic motivation
7.55. Rawsthorne and Elliot 1999
7.55.1. only engaging in a task to reach the goal in crowdsourcing practice. After that, there is no motivation to stay engage and participate
7.56. Sullivan 2010)
7.56.1. these firms have only implemented a few hundred ideas from the several thousand ideas received
7.57. brabham 2008
7.57.1. previous studies showed that crowd likely participate in crowdsourcing ventures to gain peer recognition and to develop creative skills.
7.57.2. Results of this study indicate that the desire to make money, develop individual skills, and to have fun were the strongest motivators
7.57.3. The Internet — specifically given the recent Web 2.0 trend toward massive user–generated online content — is the vehicle for distributed, mass, pleasurable production.
7.57.4. Crowdsourcing communities are new hybrid hobby/work spaces where real money can be made. Friendship and other social networking features are secondary to individual fulfillment and profit in the crowdsourcing context.
7.57.5. The crowd is more likely to be majority white, middle– or upper–class, male, college–educated, and with high–speed Internet connections in the home.
7.58. Surowiecki, 2004
7.58.1. crowd must be diverse in opinion
7.58.2. under the right circumstances, groups are remarkably intelligent, and are often smarter than the smartest people in them
7.59. Brabham (2007b)
7.59.1. theorizes the importance of diversity of identity, diversity of skills, and diversity of political investment as key to a sufficiently diverse crowd, and thus a successful crowdsourcing application.
7.59.2. [D]iversity and independence are important because the best collective decisions are the product of disagreement and contest, not consensus and compromise.”
7.60. Bonaccorsi and Rossi, 2003, 2004; Lancashire, 2001
7.60.1. people participate in opensource practice because of common good
7.61. Ghosh, 1998a, 1998b; 2005
7.61.1. people participate in opensource practice because the think as a hobby. hobbyist
7.62. Mack, 2006
7.62.1. crowdsourcing models add to these factors, from previous studies related to opensource, the existence of a bounty and a more explicit encouragement of the learning of new skills for entrepreneurship. The bounty can sometimes consist of cash and prizes, but it also includes cultural capital and can help people learn skills and develop their portfolios for future work and entrepreneurship
7.63. (Lenhart, et al., 2004; Lenhart and Madden, 2005
7.63.1. the most productive individuals in the crowd are also likely young in age, certainly under 30 and probably under 25 years of age
7.64. Linus Torvalds
7.64.1. “most of the good programmers do programming not because they expect to get paid or get adulation by the public, but because it is fun to program.”
7.65. Lakhani, et al.’s (2007)
7.65.1. found the possibility of monetary reward to be a strong indicator of success in winning InnoCentive challenges, along with intrinsic motivations (e.g., the joy of solving scientific problems) and simply having free time to fill
7.66. Johann Fuler
7.66.1. Crowdsourcing novelty is about consumers are not only asked about their opinions, desires, and needs, but also are asked to contribute their creativity and problem-solving skills. Consum- ers take on the role of co-creators
7.66.2. To date (means 2010), research on virtual co-creation has focused on the co-creation experience9 and the abilities of customers that qualify them for participation in new product development
7.66.3. Contrast to opensource, in crowdsourcing users cannot benefit from their innovation right away.
7.66.4. According to social exchange theory, consumers virtually interact with producers and engage in virtual co-cre- ation activities during new product development because they expect that doing so will be rewarding. 19 For individuals, tangibles such as goods or money, as well as intangibles such as social amenities or friendship, are rewarding.Further, not only the outcomes, but also the interaction experience itself may offer a benefit
7.66.5. FIGURE 1. Virtual Co-Creation Research Framework
7.66.6. RISKS OF OFFERING INCENTIVES
7.66.6.1. “minimax” DILLEMA: “strive to do the least possible of the task for the most possible of the reward.” THEY POST TOO MANY IDEAS BECAUSE THEY RAISE THE ODDS OF WINNING
7.66.6.2. monetary incentives may crowd out users' intrinsic motivations.
7.66.7. For consumers interested in taking part as a means to an end, it may be essential for their exploration experience
7.66.8. Community functionality enables participants to work jointly on problems and create solutions incorporating more than just the summation of each individual’s ideas and knowledge.
7.66.9. According to self-determination theory,36 engaging in leisure activities such as virtual co-creation can be considered a function of intrinsic motivation and self-determined extrinsic motivation
7.66.10. Multiple reasons drive consumers to engage RANGING from purely intrinsic motives (such as fun and alturism) through INTERNALIZED EXTRINSIC MOTIVES ( learning, reputation) to PURELY EXTRINSIC MOTIVES (such as payment and career prospects)
7.66.11. Drawing on the rich body of motivation research found in related fields such as user innovation,40 consumer creativity,41 and open source software,42 10 motive categories —intrinsic playful task, curiosity, self efficacy, skill development, information seeking, recognition (visibility,), community support, making friends, personal need (dissatisfaction), and compensation (monetary reward)
7.66.12. intrinsically motivated consumers tend to prefer experiential- oriented behaviors, while extrinsically driven consumers tend to favor goal-ori- ented behaviors.
7.66.13. TABLE 1. Motive Categories for Engaging in Virtual Co-Creation Projects
7.66.14. TABLE 2. Proposed Impact of Motives on Expectations
7.66.15. FIGURE 2. Proposed Impact of Personal Characteristics on Consumers’ Motives
7.66.16. While consumers’ creativity is important for the quality of their contribution, their web-exploration behavior is significant for the “optimal design” of the virtual interaction experience
7.66.17. TABLE 3. Virtual Co-Creation Projects Consumers where Taking Part
7.66.18. TABLE 4. Impact of Consumers’ Motives On Expectations towards Participation
7.66.19. FIGURE 3. Cluster Means of the Four Different Consumer Types
7.66.20. TABLE 5. Consumer Types
7.66.21. They reveal four different kinds of consumers engaging in co-creation: reward-oriented, need- driven, curiosity-driven, and intrinsically interested.
7.66.22. Our findings also support the idea of going beyond single customer integration projects and forming consumer innovation communities
7.66.23. monetary incentives are not as important for engagement in virtual co-cre- ation.
7.66.24. non-cash prizes can form a bond and relationship with participators
7.66.25. TABLE 6. Design Principles
7.67. Anderson et al
7.67.1. Further, an interaction can be described along three main components: the content—what the individual wants to exchange; the process—how the individ- ual wants to interact; and the people—with whom the individual wants to inter- act.
7.68. Kruglanski et al.’s
7.68.1. DANGER OF OFFERING INCENTIVES: conceptualized “minimax” strategy: “strive to do the least possible of the task for the most possible of the reward.”
7.69. Systems on the World-Wide Web CROWDSOURCING SYSTEMS enlist a multitude
7.69.1. Crowdsourcing systems face four key challenges: How to recruit contributors, what they can do, how to combine their contributions, and how to manage abuse. Many systems “in the wild” must also carefully balance openness with quality.
7.69.2. Definition of crowdsourcing CS system enlists a crowd of users to explicitly collaborate to build a long- lasting artifact that is beneficial to the whole community
7.69.3. NEW DEFINITIOn => we view CS as a gen- eral-purpose problem-solving method. We say that a system is a CS system if it enlists a crowd of humans to help solve a problem defined by the system owners
7.70. Sawhney, Mohanbir Verona, Gianmario Prandelli, Emanuela
7.70.1. one of the most important aspect of collabroation with customer is collaborating to create value through product innovation.
7.70.2. Internet allows firms to transform episodic and one-way customer interactions into a persistent dialogue with customers
7.70.3. the firm tends to be biased towards lis- tening to its current customers, and even among these, to its most important customers
7.70.4. In virtual environments, customer interactions can happen in real-time, and with a much higher frequency
7.70.5. The extended reach, enhanced interactivity, greater persistence, increased speed, and higher flexibility of virtual environments combine to produce three key benefits for collaborative innovation with customers: (a) the direction of communication; (b) the intensity and richness of the interaction; and (c) the size and scope of the audience (Table
7.70.6. Table 1: Key Differences Between Customer Collaboration in Physical and Virtual Environments
7.70.7. FIGURE 1 Mapping Internet-Based Collaboration Mechanisms Based on Nature of Collaboration and Stage of NPD Process
7.70.8. Internet-based collaboration mechanisms can be mapped to the NPD process based on two important dimensions—the nature of customer involvement that is needed, and the stage of the NPD process at which the customer involvement is desired
7.70.9. To facilitate customer participation in virtual communities, the firm may rely on intangible incentives like recognition and opinion leadership in consumer-oriented markets, while it may need to provide economic incentives in business-to-business market settings.
7.70.10. virtual communities are geniunely good for tap into the competencies of lead users
7.70.11. Internet-based mechanisms positively impact both the content and process dimensions of knowledge to support new product development.
7.71. Brandeburger & Nalebuff, 1996; Gulati, Nohria, & Zahere, 2000; Iansiti & Levien, 2004
7.71.1. Collaboration with partners and even competitors has become a strategic imperative for firms in the net- worked world of business
7.72. Prahalad & Ramaswamy, 2004; Thomke & von Hippel, 2002
7.72.1. focused on collaboration with cus- tomers to co-create value
7.73. Ulrich & Eppinger, 2003; Urban & Hauser, 1993
7.73.1. product innovation is generally conceptualized as a five-stage New Product Development (NPD) process—ideation, concept devel- opment, product design, product testing, and product introduction
7.74. Afuha, 2003
7.74.1. The Internet is an open, cost-effective and ubiquitous network
7.75. Cairncross, 1997
7.75.1. These attributes (an open, cost-effective and ubiquitous network) make it a global medium with unprecedented reach, contributing to reduce constraints of geography and distance
7.76. Evans & Wurster, 1999)
7.76.1. Internet potentially allows firms to overcome the trade-off between richness and reach because it is interactive in nature
7.77. Kozinets, 1999
7.77.1. Virtual environments also enhance the firm’s capacity to tap into the social dimension of customer knowledge, by enabling the creation of virtual communities of consumption
7.78. Hagel & Armstrong, 1997; Kozinets, 1999
7.78.1. New product development at the early stages can also benefit from online virtual communities, which bring together users who have common interests and engage in online conversations to share their experi- ences with like-minded people
7.79. Burke, Rangaswamy, & Gupta, 2001
7.79.1. provide validation at the front end of the NPD process, online surveys—the simplest and most traditional use of the Internet for collaborative innovation—are a popular tool
7.80. Marcel boger and Joel west
7.80.1. distributed processes are cumulative innovation, communities or social production, and co-creation
7.80.2. open source software and crowdsourcing as applications of the distributed processes
7.80.3. There are differences in the nature of distributed innovation, as well as its origins and its effects
7.80.4. two major streams of research on these distributed processes of innovation
7.80.4.1. von Hippel (1976, 1988, 2005)
7.80.4.1.1. the importance of user innovation, and how such innovations can be disseminated to others
7.80.4.2. Chesbrough’s (2003a, 2006a)
7.80.4.2.1. open innovation focuses on firms co-operating across firm boundaries to create and commercialize innovations.
7.81. Von Hippel (1994, 2005)
7.81.1. focuses on the ‘sticky information’ held by users that is more effectively developed through user innovation than by transferring that information to producer-innovators.
7.82. Archak and Sundararajan
7.82.1. provides a game theoretic model of a crowdsourcing contest.
7.82.2. all significant outcomes of crowdsourcing contests will be determined by contestants in a small neighborhood (core) of the most efficient contestant type
7.82.3. optimal division of the contest budget among multiple prizes
7.82.3.1. When agents are risk-neutral, the principal should optimally allocate all of its budget to the top prize even if it values multiple submissions. In contrast, if agents are sufficiently risk-averse, the principal may optimally offer more prizes than the number of submissions it desires
7.82.4. The major goal of this paper is in understanding interplay between diversity of expertise and informational asymmetries in the crowdsourcing setting and studying its implications for the optimal design of the crowdsourcing mechanisms.
7.82.5. we focus on a single, most popular form of crowdsourcing: a crowdsourcing contest
7.82.6. our setting the outcome of interest is the quality of the top K solutions.
7.82.7. for web contests and crowdsourcing practice one should consider these factors
7.82.7.1. We argue that these considerations justify the need for a very different game-theoretic model of Web contests, one that incorporates risk-averse and budget- constrained individuals, heterogeneity of expertise in the population, asymmetry of information on the Web and that can focus on investigating structure of the equilibrium when the number of participants is large.
7.82.8. Any good model of open Web contests needs to capture heterogeneity of “skills” or “expertise” across the pool of potential contestants.
7.82.9. “if entry in an expertise-based contest is free and the contest attracts a lot of participants, what is the optimal number of prizes and prize amounts that should be awarded by the profit-maximizing sponsor?”
7.82.9.1. Strikingly, the answer is distribution free and depends only on the utility function and marginal cost of effort.
7.82.10. We show that when participants are risk-averse, the optimal number of prizes can be strictly greater than the number of solutions desired by the sponsor and show that the optimal prize amounts exhibit the exponentially decreasing marginal utility pattern.
7.82.11. 3 distinguishing features of crowdsourcing
7.82.11.1. Open call
7.82.11.2. Large network of contributors
7.82.11.3. Working consumer Howe (2006)
7.82.11.3.1. it blurs boundaries between consumption and production creating a new consumer type: the “working consumer” The proactive nature of “working consumers” and their direct involvement in the production and innovation processes give new meaning to the “long tail” effect (Anderson 2004):
7.83. (Anderson 2004):
7.83.1. The proactive nature of “working consumers” and their direct involvement in the production and innovation processes give new meaning to the “long tail” effect
7.84. Yang, Adamic and Ackerman (2008)
7.84.1. They find significant variation in the expertise and productivity of the participating users: a very small core of successful users contributes nearly 20% of the winning solutions on the site.
7.85. Moldovanu and Sela (2001)
7.85.1. consider a contest in which the sponsor is interested in maximizing the sum of all solutions
7.86. Taylor (1995)
7.86.1. finds that free entry in the contest is not optimal and the organizer should restrict participation by imposing an entry fee, one that extracts all participant surplus
7.87. Terwiesch and Xu (2008)
7.87.1. show that, for expertise-based contests, a free entry in the equilibrium may or may not be optimal depending on the parameters of the expertise distribution
7.88. Huberman, Romero, Wu
7.88.1. t the productivity exhibited in crowd- sourcing exhibits a strong positive dependence on attention
7.88.2. short-term contributors compare their performance to the average contributor’s performance while long-term contributors compare it to their own media.
7.88.3. underlying dilemma of crowdsourcing TRAGEDY OF COMMONS
7.88.4. those contributing to the digital commons perceive it as a private good, inwhich payment for their efforts is in the formof the attention that their content gath- ers in the form of media quotes, downloads or news clicked on.
7.88.5. Tables 2 and 3 suggest that the number of active periods follows a power lawdistribution.
7.88.6. Attention plays a determinant role in the productivity of those uploading vide
7.88.7. these results show that the tragedy of the digital commons is partly overcome by making the uploading of digital content a private good paid for by attention.
7.89. Lampel and A. Bhalla
7.89.1. within online communities, status and recognition have been shown to be very important motiva- tors for contributing
7.90. Raymond
7.90.1. Raymond describes it as a ‘gift culture’ in which participants compete for prestige by giving time, energy and creativity away
7.91. Vuurens Arjen, de Vries, Eickhoff
7.91.1. discusses the importance to remove random judgments, while leaving room for difference in opinion. Random judgments on the other hand are useless for the evaluation of Information Retrieval systems
7.91.1.1. We expect that the quality of relevance judgments can be increased by decreasing the proportion of random judgments
7.91.2. A common approach is to obtain several judgments for each query-document pair, and combine these with a consensus algorithm
7.91.3. Different type of spammers existed in a crowdsourcing platform
7.91.3.1. random spammers
7.91.3.2. uniform spammers
7.91.3.3. semi-random spammers
7.91.4. We also suspect that workers that appear to submit random results, do not always have dishonest intentions.
7.91.4.1. 2 REASONS
7.91.4.1.1. It is possible for workers to have a different understanding of the task, possibly caused by vague or ambiguous instructions
7.91.4.1.2. workers having different frames of reference and abilities
7.91.4.2. there are COUNTERMEASURES
7.91.4.2.1. to prevent these types of random results, such as clear instructions and the use of qualification tests to determine if a worker is capable of performing the task.
7.91.5. ethical annotators are more likely to vote closer to each other than random spammers
7.91.6. If a user accepts a HIT for the first time, the user is asked to enter his/her nickname, which is displayed in the top-right corner. To judge the assignment on the screen the user clicks a label (Figure 1). After judging the last document in the HIT, a thank you message with a submit button will appear, enabling the user to submit the HIT on AMT in order to get paid. The standard AMT interface then shows a screen that enables a user to continue by accepting another HIT from the same batch.
7.91.7. workers are removed if they cast over 80% of their votes on the same labeL
7.91.8. workers below a certain threshold are producing random results and the worker with the lowest inter worker agreement is the most likely random spammer.
7.91.9. workers below a certain threshold are producing random results and the worker with the lowest inter worker agreement is the most likely random spammer.
7.91.10. The average agreement amongst accepted workers, between accepted workers and the TREC participants’ consensus and between accepted workers and the gold standard are all close to 67%
7.91.11. The strength of crowdsourcing does not come from the highest performance of individuals, but from aggregating several annotations obtained from ethical workers.
7.92. According to Crowdsourcing.org
7.92.1. There are 6 types of crodsourcing
7.92.1.1. Crowdfunding (22% of sites)
7.92.1.2. Crowd labour (8% of siteS)
7.92.1.3. Crowd innovation (10% of sites)
7.92.1.4. Distributed knowledge (37% of sites)
7.92.1.5. Crowd Creativity (14% of sites)
7.92.1.6. Tools (9% of sites)