111
edits
Changes
Jump to navigation
Jump to search
Check my work log to see what I have done on a day to day basis
Next steps will be to parse the PDFS, currently running a script to convert them to text Currently running a shell script to download the PDFs. Downloaded most of the PDFs. There were errors download some of the files.
no edit summary
contains the Investigation Title, Investigation No., link to the PDF on the website, Notice description, and date the notice was issued
I have also downloaded the PDFS from the website. These are the pdfs that are in the csv file. Some of the PDFS were no able to tot be downloaded. The PDFs are here E:\McNair\Projects\USITC\pdf_copypdfs_copy
These files were downloaded using the script on the Postgres Server. There are issues downloading PDFs onto the remote windows machine for some reason.
==Status==
Currently the web scraper is able to gather all of the data that I can gather from the HTML.
There are a few cases where the Investigation Number is not listed and I need to test for those and
fix that in the code.
