Changes

Jump to navigation Jump to search
1,468 bytes added ,  13:48, 21 September 2020
no edit summary
{{McNair ProjectsProject|Project TitleHas project output=Mechanical Turk (Tool),Guide|Topic AreaHas sponsor=Entrepreneurship Ecosystems,McNair Center|OwnerHas title=Daniel Fehder,Mechanical Turk (Tool)|Start TermHas start date=Summer 2016,|StatusHas keywords=Tool|Has project status=Active,Complete
}}
==Description==
In the steps below, we describe the creation of a Turk project that asks Turk workers to find the twitter handles of companies. It will take as input a series of google search queries in csv form and the workers to enter the search strings into google and look to see if there are google handles that are returned on the first page of the search results.
'''Step 1, Geting Started:'''Click on Create link on the home page. It will bring you to the page that has all the existing projects. 
Figure 1: Existing Projects
[[File: Create.png|2000px]]
[[File: Create.png|800px]] '''Step 2: Choosing Project Type:'''Click on New Project link and you will be brought to a page that asked you to choose a project type. That is, whether you want your project to be a survey or data collection or any other format. There will be an example project in this type once you click on the link. Figure 2 is an example of data collection project. Figure 2: Data Collection Example
[[File: ChooseType.png|800px]]
'''Step 3, Project Info:''' Once you click on the New Project link, you will be brought to an interface with a number of text entry boxes. You want to summarize your project in ways that will be informative for the team as well as potential Turk workers choosing between projects. In the figure below, we describe a HIT Project FINISH.
'''Step 3, Project Info:''' Once you have decided on the type of the project, click on Create Project link. You will be brought to an interface with a number of text entry boxes. You want to summarize your project in ways that will be informative for the team as well as potential Turk workers choosing between projects. In the figure below, we describe a HIT Project FINISH.
 Figure 13: Twitter Project Info{{clear}}
[[File:ProjectDescription.png]]
'''Step 34, Choosing Pay Level:''' Once you have named the project, you have to decide on pay scale (Reward per assignment) and the number of people working on each project (number of assignments per HIT). The higher the pay per HIT, the quicker your work will be completed by turkers, but you obviously do not want to waste money. A good rule of thumb is to work on the tasks you need completed by turkers for 30-60 minutes and then see how many rows you completed. We want the per HIT pay rate to roughly equal $6.00 - $10.00 in hourly wage to get things done efficiently on the system. For example, if 50 rows can be completed in 30 minutes, then the pay level per HIT will be $10/2/50 =$0.1. If you decide to have more than 1 worker per HIT, it will be because you believe that the data task requires a certain amount of human judgement and you want to make sure that you only accept results that have been "verified" by multiple people. The last three parameters in this box determine how each HIT will be completed by each worker and how long the HIT stays in the system. You generally want "Time Allotted" to be 1 dayhour. Expiration of the HIT doesn't matter that much. One of the last important choice in this screen is the "Auto-approve" option. The quicker the auto approve, the more likely that Turkers will take your task. For now, set it to 24 hours, but remember that you are responsible for regularly auditing results when you have a project up on the Turk system.
Figure 24: Cost Parameters in Mturk{{clear}}
[[File:Costing.png]]
 '''Step 45, Design LayoutWorker Requirement:''' At this point, you have to design what the turker sees when they receive your assignment. While it is possible that one turker will complete multiple HITs, it is important to design the HIT so that it can To be easily completed the first (and possibly only) time updated by the worker. In figure 3, below, you can see the initial design layout of the default data acquisition project in the turk system. It is an example HIT that asks turkers to find the website of a restaurant. Please note that this is not a great HIT in terms of the level of clarity of the instructions. We will provide guidelines on creating instructions below. For now, just notice a few features of the HIT. To the right of "Restaurant Name", there is a field called ${name}. This is actually a hook, or a blanks space, that will be populated with the actual name of a restaurant that will come from a spread sheet that you will upload into the turk system. Each HIT will correspond to one row of the spreadsheet. This is the same for the "Address" and "Phone Number" rows. The last key thing to notice is the "Website Address" field with a text entry box right below it. When a turker receives this HIT, they will paste the web address into this text box and you will receive a new spreadsheet with whatever they (add all the other turkers) pasted in the same row as the data you used to populate each HIT.Dan           
    '''Step 6, Design Layout:''' At this point, you have to design what the turker sees when they receive your assignment. While it is possible that one turker will complete multiple HITs, it is important to design the HIT so that it can be easily completed the first (and possibly only) time by the worker. In figure 5, below, you can see the initial design layout of the default data acquisition project in the turk system. It is an example HIT that asks turkers to find the website of a restaurant. Please note that this is not a great HIT in terms of the level of clarity of the instructions. We will provide guidelines on creating instructions below. For now, just notice a few features of the HIT. To the right of "Restaurant Name", there is a field called ${name}. This is actually a hook, or a blanks space, that will be populated with the actual name of a restaurant that will come from a spread sheet that you will upload into the turk system. Each HIT will correspond to one row of the spreadsheet. This is the same for the "Address" and "Phone Number" rows. The last key thing to notice is the "Website Address" field with a text entry box right below it. When a turker receives this HIT, they will paste the web address into this text box and you will receive a new spreadsheet with whatever they (add all the other turkers) pasted in the same row as the data you used to populate each HIT.  Now, How do you modify this HIT to reflect your actual data task? You can actually change the wording of the task directly in the editor screen. Make sure that all of the data element hooks (like ${name}) correspond to the actual names of the columns in the CSV file that you will upload on the turk system. But what if you need the task to look substantially different from the one you are looking at? If you click on the "Source", it will show you the actual html code of your HIT task as displayed in Figure 46. The turk system allows you to display a full website essentially for your HIT task with javascript, CSS, etc. As we develop our system at the McNair center, your will have more existing tasks to choose from, but when you need to actually build your own, some useful HTML references are listed below. When you have completed editing your HIT template, click on the "Save" Button and then move to "Preview". In this last screen, it will show you exactly what the turkers will see (Figure 57). If it looks correct, click "Finish"
;HTML references
Figure 35: Design Layout {{clear}}
[[File:DesignLayout2.png]]
Figure 46: Design Layout, Raw HTML {{clear}}
[[File: DesignLayoutRaw.png]]
Figure 57: HIT Preview {{clear}}
[[File: HITPreview.png]]
'''Step 57, Publishing a Batch:''' Now that you have your HIT Template ready, you can post to the Turk system. To do so, Click the "Publish Batch" button in figure 68. It will then ask you to upload a csv file for that batch. Remember that this CSV file has to have headers that correspond to the data hooks in your template. Once you have successfully uploaded, you should be able to confirm payment and publish the batch. If there is not enough money preloaded on the turk system. Notify either Dan or Ed.
Figure 68: HIT Publishing{{clear}}
[[File: PublishBatch.png]]
 
==Examples in Papers==
*Hubs Paper (Section 2.2.3): [[Hubs (Academic Paper)]]
==Existing HIT Library==
create #Twitter Activity(Data Collection) Input: Search Text: Copy and Paste the search text:CompanyName City "Twitter"(e.g.Capital Factory Austin Twitter) into a list searching engine Company Name: Click on the link that has the company name Data Collect: Twitter Handle: copy and paste the Twitter handle of the company Twitter Activity: Record the date(MM/DD/YY) of existing hits and what they dothe latest 10th tweet/retweet
==TDL with HITS==
* Data validation using javascriptMeeting on 07/08: Dan, Todd, Ariel#Add Step 1 and Step 2 (Ariel)#Update the part on pay level (Ariel)#Step 5: 'worker requirement' to be updated by Dan *07/11#Publish ''Twitter Activity'' project#Work on ''Number of events'' and ''Onsite mentors'' turks *07/12*Evaluate results of Twitter Activity Turk (Todd) *07/13*Try to find more potential Hubs from current incubators<!-- flush flush --><!-- flush flush -->

Navigation menu