Changes

The goal of the Govtrack Webcrawler is to create and automated system in perl ActivePerl by which bills relevant to a certian topic can be pulled from the Govtrack API which can be found [https://www.govtrack.us/api/v2/bill?congress=114&order_by=-current_status_date here].

Now the parameters used to search the api are decided. Currently we are searching for the 107 bills related to entrepreneurship during the 114th congress. my $queryName = "Entrepreneurship''insert query here''"; my $congressNo = "114''insert congress num here''"; my $limit = "107''insert maximum number of bills to search here''";

The useragent object can now retrieve and decode the text from the url into a JSON datastring.

Using the JSON module the JSON string can be converted into a data structure that can be searched through for relevant data. my $JSONcontent = JSON::XS::decode_json($genResponse->decoded_content);After getting the resulting JSON Data an array of all the bills matching the parameters bill id's can now be locatedat @{$JSONcontent->{objects}}Iterating over this array more specific bill information can be found by constructing a url using the bill's ID number as such, my $billurl = "https://www. Using govtrack.us/api/v2/bill/" . $bill->{id};After finding this page the ids another page data can once again be found which contains more specific information about each billparsed from a string into a data structure. my $billresponse = $ua->get($billurl); my $billcontent = JSON::XS::decode_json($billresponse->decoded_content);From this bill specific page the tags of each bill can be used to determine whether or not the bill is relevant and should be reviewed by Mcnair Center staff. An array of these tags can be found at @{$billcontent->{terms}}The data retrieved from this search is then placed into several tab delimited text files containing sets of useful information about bills deemed relevant.Currently tags that are considered relevant: