第一次使用scrapy爬豆瓣top250 报错 AttributeError Requset has no attribute dont_filter 和 meta

关注

原创最新推荐文章于 2025-09-26 18:19:13 发布 784 阅读 AI 写同款· GEO 优化 ›

UFS Host Controller工作流程 contentsUFS架构图HCI接口架构传输请求接口（Transfer Request Interface）UFS 主机控制器寄存器接口（host controller Interface）UTP 层数据传输Host Software与Host Controller的交互Host Controller初始化配置与控制CRYPTOCFG 配置过程该文档描述了UFS Host Controller的主要运作流程以及在开源内核代码hisi平台中的host controller的接口函数设计文档大部分内容参阅读详情

首次学习scrapy框架来爬取豆瓣Top250,在写完代码执行时,报了AttributeError: ‘Request’ object has no attribute ‘meta’ 还有AttributeError: ‘Request’ object has no attribute ‘dont_filter’ .

执行的代码

# -*- coding: utf-8 -*-
from requests import Request
from scrapy.spiders import Spider
from scrapyspider.items import DoubanMovieItem


class DoubanMovieTop250Spider(Spider):
    name = "douban250"
    start_urls = ["https://movie.douban.com/top250"]


    headers ={
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'
    }

    def start_requests(self):
        # url = "https://movie.douban.com/top250"
        url = 'https://movie.douban.com/top250'
        yield Request(url, headers = self.headers)

    def parse(self, response):
        item = DoubanMovieItem()
        movies = response.xpath('//ol[@class="grid_view"]/li')
        for movie in movies:
            item['ranking'] = movie.xpath(
                './/div[@class="pic"]/em/text()').extract()[0]
            item['movie_name'] = movie.xpath(
                './/div[@class="hd"]/a/span[1]/text()').extract()[0]
            item['score'] = movie.xpath(
                './/div[@class="star"]/span[@class="rating_num"]/text()'
            ).extract()[0]
            item['score_number'] = movie.xpath(
                './/div[@class="star"]/span/text()').re(u'(\d+)人评价')[0]
            yield item

跑完代码cmd出现下面的错误情况:

后来经过一番查找才发现原来是因为自己导包错误的问题…

from requests import Request
应该改为
from scrapy import Request

修改完的代码:

from scrapy import Request
from scrapy.spiders import Spider
from scrapyspider.items import DoubanMovieItem


class DoubanMovieTop250Spider(Spider):
    name = "douban250"
    start_urls = ["https://movie.douban.com/top250"]


    headers ={
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'
    }

    def start_requests(self):
        # url = "https://movie.douban.com/top250"
        url = 'https://movie.douban.com/top250'
        yield Request(url, headers = self.headers)

    def parse(self, response):
        item = DoubanMovieItem()
        movies = response.xpath('//ol[@class="grid_view"]/li')
        for movie in movies:
            item['ranking'] = movie.xpath(
                './/div[@class="pic"]/em/text()').extract()[0]
            item['movie_name'] = movie.xpath(
                './/div[@class="hd"]/a/span[1]/text()').extract()[0]
            item['score'] = movie.xpath(
                './/div[@class="star"]/span[@class="rating_num"]/text()'
            ).extract()[0]
            item['score_number'] = movie.xpath(
                './/div[@class="star"]/span/text()').re(u'(\d+)人评价')[0]
            yield item