Move your USER_AGENT line to the settings.py
file, and not in your scrapy.cfg
file. settings.py
should be at same level as items.py
if you use scrapy startproject
command, in your case it should be something like myproject/settings.py
Move your USER_AGENT line to the settings.py
file, and not in your scrapy.cfg
file. settings.py
should be at same level as items.py
if you use scrapy startproject
command, in your case it should be something like myproject/settings.py