Oliver Chang (Work Log) - Revision history

OliverC at 03:19, 3 December 2017

2017-12-03T03:19:15Z

OliverC: /* Fall 2017 */

2017-12-01T18:40:18Z

Fall 2017

OliverC at 19:08, 21 November 2017

2017-11-21T19:08:58Z

OliverC at 19:07, 21 November 2017

2017-11-21T19:07:46Z

OliverC at 19:02, 21 November 2017

2017-11-21T19:02:13Z

OliverC: /* Fall 2017 */

2017-11-15T03:24:19Z

Fall 2017

OliverC: /* Fall 2017 */

2017-11-14T04:23:28Z

Fall 2017

OliverC: /* Fall 2017 */

2017-11-10T22:26:53Z

Fall 2017

OliverC: /* Summer 2017 */

2017-11-09T18:33:14Z

Summer 2017

OliverC: transition to new format

2017-11-09T18:32:55Z

transition to new format

@@ Line 9: / Line 9: @@
 * [[Parallel_Enclosing_Circle_Algorithm|Parallel Enclosing Circle Algorithm]]
 * [[Python_on_the_RDP|Python on the RDP]]
 ''Uploads'':
@@ Line 28: / Line 29: @@
 [[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)|(log page)]]
--12-01: TBD
+-12-02: communicated results of running ECA on 2 stddev table (not the error source); update db and web server software; check if scipy elision is ECA bug (it is not)
 -11-30: stub out implementation, add parsing code and mapping code...just need the meat of the algorithm now

← Older revision		Revision as of 18:40, 1 December 2017
Line 27:		Line 27:

	[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]		[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]
		+
		+	2017-12-01: TBD
		+
		+	2017-11-30: stub out implementation, add parsing code and mapping code...just need the meat of the algorithm now
		+
		+	2017-11-27: documentation & bug finding on the parallel enclosing circle project; research hierarchical linkage approach

	2017-11-14: hand off work on xpathing to Shelby; walked through some program design decisions		2017-11-14: hand off work on xpathing to Shelby; walked through some program design decisions

@@ Line 8: / Line 8: @@
 * [[GPU_Build|GPU Computer Build]]
 * [[Parallel_Enclosing_Circle_Algorithm|Parallel Enclosing Circle Algorithm]]
 ''Uploads'':

@@ Line 11: / Line 11: @@
 ''Uploads'':
-* [[File:Path-example.PNG]]
+* [http://mcnair.bakerinstitute.org/wiki/File:Path-example.PNG File:Path-example.PNG]
 ** Screenshot of how to set the path environment variable
 * [[File:PADX-File-Description-v2_Hague.pdf]]

@@ Line 11: / Line 11: @@
 ''Uploads'':
 * [[File:PADX-File-Description-v2_Hague.pdf]]
 ** Describes patent kind codes (notably, what the hell X0 represents)

← Older revision		Revision as of 03:24, 15 November 2017
Line 24:		Line 24:

	[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]		[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]
		+
		+	2017-11-14: hand off work on xpathing to Shelby; walked through some program design decisions

	2017-11-13: finish javadoc of common/ and some trickier parts about downloading; added descriptions, results to one-off java/python scripts so that they actually make sense in context		2017-11-13: finish javadoc of common/ and some trickier parts about downloading; added descriptions, results to one-off java/python scripts so that they actually make sense in context

← Older revision		Revision as of 04:23, 14 November 2017
Line 24:		Line 24:

	[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]		[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]
		+
		+	2017-11-13: finish javadoc of common/ and some trickier parts about downloading; added descriptions, results to one-off java/python scripts so that they actually make sense in context

	2017-11-10: add javadoc documentation to patent reproducibility project after forgetting half of the stuff myself		2017-11-10: add javadoc documentation to patent reproducibility project after forgetting half of the stuff myself

← Older revision		Revision as of 22:26, 10 November 2017
Line 24:		Line 24:

	[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]		[[Oliver Chang]] [[Work Logs]] [[Oliver Chang (Work Log)\|(log page)]]
		+
		+	2017-11-10: add javadoc documentation to patent reproducibility project after forgetting half of the stuff myself

	2017-10-25: start ingestion of application xml files and deal with all the bugs which accompany that		2017-10-25: start ingestion of application xml files and deal with all the bugs which accompany that

← Older revision		Revision as of 18:33, 9 November 2017
Line 97:		Line 97:

	2017-06-16: create method of getting all data into the database, whether it likes it or not; copy over assignments, granted data using new scheme		2017-06-16: create method of getting all data into the database, whether it likes it or not; copy over assignments, granted data using new scheme
−

	2017-06-15: add more robust error reporting, fix race conditions; build out assignment driver; build out fee event driver; add error logging		2017-06-15: add more robust error reporting, fix race conditions; build out assignment driver; build out fee event driver; add error logging

@@ Line 27: / Line 27: @@
 -10-25: start ingestion of application xml files and deal with all the bugs which accompany that
--10-21: create xml explorer script to mass-inspect xpaths (can be found at <code>E:\McNair\Projects\SimplerPatentData\src\main\java\org\bakerinstitute\mcnair\xml_schema_explorer</code>
+-10-21: create xml explorer script to mass-inspect xpaths (can be found at <code>E:\McNair\Projects\SimplerPatentData\src\main\java\org\bakerinstitute\mcnair\xml_schema_explorer</code>)
 -10-20: re-run assignment import, review xpaths
@@ Line 46: / Line 46: @@
 ===Summer 2017===
-* Aug 4: setup parallel instance python framework for job reporting; begin test run
+-08-04: setup parallel instance python framework for job reporting; begin test run
-* Aug 2: finish up some documentation of the code and for the wiki
-* Aug 1: discuss with Abhi & Ed about alternatives to Java port because of algorithmic constants that would be hard to port; run test batches on python with addition of equality operators and convergence early stopping
+-08-02: finish up some documentation of the code and for the wiki
-* July 31: sketch out parallel enclosing circle algorithm
-* July 28: field questions and data cleanup questions from Kerda & Joe & Adrian
+-08-01: discuss with Abhi & Ed about alternatives to Java port because of algorithmic constants that would be hard to port; run test batches on python with addition of equality operators and convergence early stopping
-* ''travelling''
-* July 19: try to remove duplicated records (esp. those with empty titles) which are preventing the addition of a unique constraint
+-07-31: sketch out parallel enclosing circle algorithm
-* July 18: run correspondent join on properties and correspondents table to match previous project; sync with Adrian and Abhi
-* July 17: redo db operations after cleaning up granted patent number bugs
+-07-18: field questions and data cleanup questions from Kerda & Joe & Adrian
-* July 13: powwow about parallelizing Enclosing Circle Algorithm; sketch out what to do for the rest of the summer; work more on joins
-* July 12: generate some example data illustrating the difficulty of joining different tables
+-07-19: try to remove duplicated records (esp. those with empty titles) which are preventing the addition of a unique constraint
-* July 11: track down some bugs that happen very rarely and were missed in the initial qa phase
-* July 7: catch up on documentation
+-07-18: run correspondent join on properties and correspondents table to match previous project; sync with Adrian and Abhi
-* July 6: try (unsuccessfully) to understand docid mapping...create exploration scripts
-* July 5: add invention title to proper grouping of assignment properties; optimize XML parsing
+-07-17: redo db operations after cleaning up granted patent number bugs
-* June 30: powwow with James, Abhi, Ed about optimization issues; discuss document ids, X0 etc with Ed; pinpoint issues with APS doc numbers (see Repro Pat Dat#Gotchas) for more info
-* June 29: add logging of copy commands, more chattiness to scripts, debug assignment data failure
+-07-13: powwow about parallelizing Enclosing Circle Algorithm; sketch out what to do for the rest of the summer; work more on joins
-* June 28: create examples for expansion to plant, reissue, design patent collection; start optimizing xml
-* June 27: write SQL to replicate assignees, extract postcodes for ongoing projects
+-07-12: generate some example data illustrating the difficulty of joining different tables
-* June 26: speedup code, abstract in-memory file splitters to avoid repetition and some weird edge cases
-* June 25: create mappings for APS, assignment properties, XML 2.5 for data import; run data imports for granted data
+-07-11: track down some bugs that happen very rarely and were missed in the initial qa phase
-* June 23: cleanup hacky models with a better set of abstractions; cleanup IDE warnings; redefine patent-address mapping
-* June 22: create postcode<->patent table
+-07-07: catch up on documentation
-* June 21: document granted patent queries and equivalencies
-* June 20: sketch out APS driver; discuss patent id problem; further document with evidence the zipcode data validity
+-07-06: try (unsuccessfully) to understand docid mapping...create exploration scripts
-* June 19: skim address regular expressions; cursory investigation of patent table
-* June 16: create method of getting all data into the database, whether it likes it or not; copy over assignments, granted data using new scheme
+-07-05: add invention title to proper grouping of assignment properties; optimize XML parsing
-* June 15: add more robust error reporting, fix race conditions; build out assignment driver; build out fee event driver; add error logging
-* June 14: migrate bulk inserts to copy command; refresh on address data and start in on that; convert processor to multi-threaded application
+-06-30: powwow with James, Abhi, Ed about optimization issues; discuss document ids, X0 etc with Ed; pinpoint issues with APS doc numbers (see Repro Pat Dat#Gotchas) for more info
-* June 13: spot check SQL tables; fix broken final case endlessly looping; investigate smarter insert methods
-* June 12: add XML printer, use it to inspect applications; extend BaseScraper to fetch patent application data; add applications documentation to my project page; add CREATE of other tables
+-06-29: add logging of copy commands, more chattiness to scripts, debug assignment data failure
-* June 8: add foreign key inserts; create pretty printer for XML analysis
-* June 7: finalize DB abstraction layer; migrate code to bulk inserts; upgrade webserver software and do optimization on RDP postgres with Ed
+-06-28: create examples for expansion to plant, reissue, design patent collection; start optimizing xml
-* June 6: add jdbc; create basic schema; add db interaction; schedule meeting for later in the week
-* June 5: look into postgresql; refersh on postgis; add some notes to the Enclosing Circle Algorithm page
+-06-27: write SQL to replicate assignees, extract postcodes for ongoing projects
-* June 1: add RDP git remote; add more documentation to wiki page; refactor downloader scripts; start creation of tooling for interacting with data
-* May 31: finish copy-pasting attributes into the wiki page; retroactively fill out work log; meet with Ed to discuss next steps
+-06-26: speedup code, abstract in-memory file splitters to avoid repetition and some weird edge cases
-* May 30: update documentation on wiki, restructure large binary files to have more hierarchy instead of a flat listing at the root
-* May 29: expand to APS; expand to raw assignment data
+-06-25: create mappings for APS, assignment properties, XML 2.5 for data import; run data imports for granted data
-* May 27: expand to maintenance fee data
-* May 26: create models, translate <code>xmlparser*.pl</code> file into Java; start using builder pattern
+-06-23: cleanup hacky models with a better set of abstractions; cleanup IDE warnings; redefine patent-address mapping
-* May 25: sketch out OO design of project; download bulk data
-* May 24: move wiki pages around; start git repository for project
+-06-22: create postcode<->patent table
-* May 21: discuss technical details of previous work with Ed
-* May 8: cleanup dead links on wiki and start reading about previous work; discuss current project status with Ed
+-06-29: document granted patent queries and equivalencies
-* May 4: setup wiki account, rdp account, database training
 [[Category:Work Log]]