Learning Python and Scrapy 01

Please note that all opinions are that of the author.

In this blog post, I’m going to attempt to use Python and Scrapy to get a basic scrape of the website stlflix.com going.

Step 1: Generating a Python Project

Let’s start by changing into our Python working directory and creating the project:

cd ~/Sync/coding/python
python3 -m venv stlflix

Step 2: Activate it

The next step is to make it active:

cd stlflix
source bin/activate

Step 3: Add a .gitignore

This is a best practices thing. Just find one from the net / github.

Step 4: Add a requirements.txt

Create a file in the root directory of your project and add the name scrapy in this file.

Step 5: Install your package with pip

Use pip to install your packages:

pip install -r requirements.txt

See Also

Here are some of the references I used to write this: