Current location - Recipe Complete Network - Complete cookbook of home-style dishes - How to write a crawler program in python
How to write a crawler program in python
First, you need to install requests and BeautifulSoup4. Then execute the following code. importrequestsfrombs 4 import beautiful soup iurl = '。 Parser') # Title H 1 = soup. Select ('# artibodytitle') [0]. text # Source time _ Source = soup . select('。 Time source') [0]. text # Source origin = soup . select(' # artibodyp ')[0]。 text . strip()# ori title = soup . select(' # artibodyp ')[66 . text . strip()# content raw _ content = soup . select(' # artibodyp ')[2: 19] content =[]for paragraph raw _ content:content . append(paragraph . text . strip())@ '。 Add (Content) # Edit AE = Soup. Select ('. Article Editor') [0]. Text messages are enough.