It appears I do not have the necessary tools to execute shell commands or Python scripts directly, which are required for both deduplicating the CSV file and performing the web scraping task. I am unable to proceed with these actions.

I can provide you with the Python code for both the deduplication and the scraping logic, which you can then execute manually on your system. Please let me know if you would like me to generate this code for you.