227
edits
Changes
Jump to navigation
Jump to search
Listing Page Classifier Progress (view source)
Revision as of 19:37, 17 May 2019
, 19:37, 17 May 2019→Progress Log
***Pytorch: Coding is easy, so it has a flatter learning curve, Supports dynamic graphs so you can adjust on-the-go, Supports GPU acceleration.
***TensorFlow: Flexibility, Contains several ready-to-use ML models and ready-to-run application packages, Scalability with hardware and software, Large online community, Supports only NVIDIA GPUs, A slightly steep learning curve
*Initiate the idea of data preprocessing: create proper input dataset for the CNN model '''5/2/2019'''*Work on data preprocessing '''5/6/2019'''*Keep working on data preprocessing*Generate screenshot '''5/7/2019'''*some issues occurred during screenshot generating (IN PROGRESSWill work on this more tomorrow)*try to set up CNN model**https://www.datacamp.com/community/tutorials/cnn-tensorflow-python '''5/8/2019'''*fix the screenshot tool by switching to Firefox*Data preprocessing '''5/12/2019'''*Finish image data preprocessing '''5/13/2019'''*Set up initial CNN model using Keras**issue: Keras freezes on last batch of first epoch, make sure the following: steps_per_epoch = number of train samples//batch_size validation_steps = number of validation samples//batch_size '''5/14/2019'''*Implement the CNN model *Work on some changes in the data preprocessing part (image data)**place class label in image filename '''5/15/2019'''*Correct some out-of-date data in <code>The File to Rule Them ALL.csv</code>, new file saved as <code>The File to Rule Them ALL_NEW.csv</code>*implement generate_dataset.py and sitmap tool**regenerate dataset using updated data and tool '''5/16/2019'''*implementation on CNN*Some problems to consider:**some websites have more than 1 cohort page: a list of cohorts for each year**class label is highly imbalanced: https://towardsdatascience.com/deep-learning-unbalanced-training-data-solve-it-like-this-6c528e9efea6 '''5/17/2019'''*have to go back with the old plan of separating image data :(*documentation on wiki*test run on the GPU server
