Changes

[[Shelby Bice]] [[Work Logs]] [[Shelby Bice 2017-12-07 3:15 PM - 4:45 PM (Work Logcame in late due to finals)|(log - finished debugging additions to Oliver's code for the tables that are related to design, reissue, and plant patents, added a troubleshooting section to Oliver's page)]]with instructions on how to deal with issues importing the project.

===Spring 2017 Work===-12-04 2/14/2017 10:00 am 45 pm - 124:00 pm Set up personal wiki, set - continued debugging and started typing up work logtroubleshooting tips for the next person who alters the patent code

2/16/2017 10-12-01 3:30 am 15 pm - 125:00 pm Researched past work - ran code (and ran into errors) which I have been working on databasesfixing. If I don't finish today, discussed I'll continue doing so on Monday. I plan to write up some of the mistakes I made in adding the tables to the database and add them to Oliver's page, with his permission so that people in the future who are not familiar with the code (like I wasn't) hopefully won't fall into the same pitfalls. The main things are 1) how to set up a maven project with Ed(if IntelliJ doesn't automatically set it up for you when you open/import the project, and 2) how to set up the data source so you can run SQL scripts and actually load data into the database on the RDP.

2/21/2017 9-11-30-17 1:00 am 55 pm - 123:00 55 pm Set up work page- continued altering code. Wrote creation tables script in SQL for creating tables for the design, reviewed SQLreissue, researched designing databaseand plant patents, continued going and went through the checklist to make sure I had done everything to create these new tables based on Oliver's Reproducible Patent Data page. Will definitely run code tomorrow and will type up the exact process I went through wikito create new tables.

2017-11-17 2/23/2017 9:30 am 15 pm - 125:00 pm Reviewed Perl- continued altering code to include the special fields for plant, read about database reissue, and design, set up project page for redesigning database, started documenting processpatent. Their models and the changes to XmlParser relevant to those models should be done. I will go through the rest of the code and check where else I need to make alterations next time I am in.

3/2/2017 9-11-16 8:15 45 am - 1210:15 pm Started excel spreadsheet 30 am - continued altering code to document current schema include the special fields for plant, reissue, and design and improvements to be made, updated project pagespatents

3/7/2017 9-11-14 8:30 - 10:30 am - 12:00 pm Continued met with Oliver to go over his code, began working on spreadsheetscripts and finding paths for the fields qunie to design, added relevant page links to project pagereissue, took notes on what I want documentation to look like in the futureand plant patents.

3/9/2017 9-11-10: 2:00 am - 125:00 pm Finished first draft - updated format of worklog, finished researching fields of spreadsheet describing the current schema (new design, utility, and possible changes) to the Patent databasereissue tables, and documenting what each table contains

3/21/2017 9-11-03: 2:30 am - 123:30 pm, 3:45 pm - 5:00 pm (I went to get a snack because I was starving!) - Worked Finished up finding unique fields, added them to database design, and researched what the fields represented - again, this is taking a while since there is little to no documentation online about what fields on determining "core" tables for new patent databasethese XML files actually represent.

3/22/2017 5-11-02: 2:30 15 pm to 6- 4:30 00 pm - Patent Data meetingmade a main data page and continued going through the xpaths Oliver found. Notified Michelle so she can continue adding patent pages

3/23/2017 -10-31: 9:15 00 am - 1210:00 pm 30 am - Narrowed down core tables started trying to determine from Oliver's research what paths are unique to utility, reissue, and fieldsplant patents that we have not yet accounted for in our design of the table

4/4/2017 9-10-27: 2:00 am pm - 113:45 30 pm - Worked on Kept updating documentationtables. For some of the new values I've been adding to tables (I didn't realize we had XPaths for some fields) I'm struggling to find out exactly what the field means (for example, found the date on a citation - is this the date the citedpatent was granted? or is the date that the citingpatent cited the citedpatent?) so the process is slow going. I want to get the documentation on pulling data/making tables right though so in the future someone can look up and see exactly what each field in the table represents and databases, started looking through DTDs to find extra fields not have to pullguess

4/6/2017 9-10-26:30 am 2:00 pm - 114:30 00 pm - Kept looking through DTDsworked on updating the designs for the patent databases tables based on what was discussed on Friday (specifically adding inventors and lawyers tables, kept updating documentationaltering fields on patent tables) - will continue this on Friday

4/11/2017 9-10-20: - 2:15 am 00 pm - 125:00 pm - Worked on trying to update met with Ed and Oliver about patent data through 2016database design

4/13/2017 9-10-19: 2:30 am 00 pm - 124:00 pm - Continued working on trying to update finished up ER diagram and description of all tables for patent data through 2016database, specifically parsing started reading papers concerning creating an inventor's database (looks like other research groups have merged the USPTO data, worked with Ed the Harvard Dataverse data in order to update perl scriptscreate an inventors table)

4/18/2017 9-10-17: 8:45 am - 1210:30 pm am - Cleaned up documentation morecontinued trying to solve twitter issues, kept working worked on ER diagram, skimmed through a paper I found (linked at the process top of parsing the page for Redesign Assignment and Patent Database) to see how they cleaned the data, since I assume that will be the next step to research after finishing the ER diagrams

4/20/2017 -10-12:00 am - 112:30 pm - wrote copy statements for copying data from RDP 5:00 pm write blog post on Grace Hopper Celebration and attempted to database, continued working on documentation.solve Twitter issue

4/25/2017 -10-03: 8:00 45 am - 1210:00 pm 30 am - worked on documentation, tried finished adding descriptions to determine how to clean up the USPTO Assignee Datafields for each table in the patent database, started work on an ER diagram

4/27/2017 1-09-29: 2:00 pm - 35:00 pm - worked on documentation morefinished adding the design for the Patent database to the project page, tried to figure out how added descriptions of the fields for each table to clean citation datathe project page including the datatype that I think the field will be when it's loaded into the database

===Fall 2017 Work===-09-28: 9/15/2017 2:00 pm am - 510:00 pm 30 am - introduced to new continued going over and documenting last semester's patent database projected, reviewed design and took notes on USPTO adding the details to the Redesign Assignment data (notes can be found under McNair/Projects/Redesigning and Patent Database/New project page. Additionally, I began trying to determine how to match up the information in the Document_Info table in Assignment to match up with a patent_id in the Patent table in Patent Database Project as Notes on USPTO Assignment Data Paper).

9/22/2017 8:30 am - 10:30 am - continued looking at paper on USPTO assignment data and Tomorrow I will finish adding the design for the Patent database to the notes on what project page, add descriptions of the fields for each table to the design of that database should look likeproject page, specifically and start working on what I need ER diagrams for different tables and what I don't know yet about the design. Had to set up connection to RDP again due to technical issuestwo databases.

9Links for creating ER diagrams: https://erdplus.com/23#/2017 2:00 pm - 4standalone or https:00 pm - continued working on design of Assignment database and how it will connect to Patent database by writing out what will be in each table in Assignment and questions about different possible structures of tables that we will have to address before finalizing the design - the notes can be found under McNair/Projects/Redesigning Patent Databasecreately.com/New Patent Database Project as Notes on USPTO Assignment Data Paper. Questions are highlighted in yellow throughout the document[[Category:Work Log]]app/?tempID=hqdgwjki1&login_type=demo#

9/2017-09-26/2017 : 8:45 am - 10:00 am - continued worked on design of Assignment database by checking my design against the work done last semester on the assignment data restructure to make sure I didn't miss anything major. Began going over my patent database design from last semester to tweak it. Will need to sync up with Joe Reilly to see if there are any new fields that we are pulling from the data. Additionally, I made a new project page called Redesign Assignment and Patent Database that encompasses the new design for the Assignment database and Patent database redesign and moved some of the notes from McNair/Projects/Redesigning Patent Database/New Patent Database Project/Notes on USPTO Assignment Data Paper to the project page.

92017-09-23: 2:00 pm - 4:00 pm - continued working on design of Assignment database and how it will connect to Patent database by writing out what will be in each table in Assignment and questions about different possible structures of tables that we will have to address before finalizing the design - the notes can be found under McNair/Projects/27Redesigning Patent Database/New Patent Database Project as Notes on USPTO Assignment Data Paper. Questions are highlighted in yellow throughout the document[[Category:Work Log]] 2017 9-09-22:00 8:30 am - 10:30 am - continued going over looking at paper on USPTO assignment data and documenting last semesteradding to the notes on what the design of that database should look like, specifically on what I need for different tables and what I don's t know yet about the design. Had to set up connection to RDP again due to technical issues.  2017-09-15: 2:00 pm - 5:00 pm - introduced to new patent database design projected, reviewed and adding the details to the Redesign took notes on USPTO Assignment and data (notes can be found under McNair/Projects/Redesigning Patent Database/New Patent Database project page. AdditionallyProject as Notes on USPTO Assignment Data Paper) </onlyinclude> ===Spring 2017=== 2017-04-27: 1:00 pm - 3:00 pm - worked on documentation more, tried to figure out how to clean citation data 2017-04-25: 10:00 am - 12:00 pm - worked on documentation, I began trying tried to determine how to match up the information in the Document_Info table in Assignment to match clean up with a patent_id in the Patent table in Patent.USPTO Assignee Data

Tomorrow I will finish adding 2017-04-20: 10:00 am - 11:30 pm - wrote copy statements for copying data from RDP to database, continued working on documentation. 2017-04-18: 9:45 am - 12:30 pm - Cleaned up documentation more, kept working through the process of parsing the data 2017-04-13: 9:30 am - 12:00 pm - Continued working on trying to update patent data through 2016, specifically parsing the design data, worked with Ed to update perl scripts 2017-04-11: 9:15 am - 12:00 pm - Worked on trying to update patent data through 2016 2017-04-06: 9:30 am - 11:30 pm - Kept looking through DTDs, kept updating documentation 2017-04-04: 9:00 am - 11:45 pm - Worked on updating documentation, found documentation on pulling data/making tables and databases, started looking through DTDs to find extra fields to pull 2017-03-23: 9:15 am - 12:00 pm - Narrowed down core tables and fields 2017-03-22: 5:30 pm to 6:30 pm - Patent Data meeting 2017-03-21: 9:30 am - 12:00 pm - Worked on determining "core" tables for new patent database 2017-03-09: 9:00 am - 12:00 pm Finished first draft of spreadsheet describing the current schema (and possible changes) to the Patent database  2017-03-07: 9:30 am - 12:00 pm Continued working on spreadsheet, added relevant page links to the project page, add descriptions of took notes on what I want documentation to look like in the fields for each table future 2017-03-02: 9:15 am - 12:15 pm Started excel spreadsheet to document current schema design and improvements to the be made, updated project pages 2017-02-23: 9:30 am - 12:00 pm Reviewed Perl, read about database design, set up project page for redesigning database, started documenting process 2017-02-21: 9:00 am - 12:00 pm Set up work page, and start working reviewed SQL, researched designing database, continued going through wiki 2017-02-16: 10:30 am - 12:00 pm Researched past work on ER diagrams for the two databases., discussed project with Ed 2017-02-14: 10:00 am - 12:00 pm Set up personal wiki, set up work log [[Category:Work Log]]