Duplicate logs from apify logger in Scrapy-Apify integration

[vdusek]

Description

• The Apify logger (apify) is producing duplicate log entries when running a Scrapy-Actor integration.

Log

$ apify run
[apify] INFO  Initializing Actor...
[apify] INFO  Initializing Actor... ({"message": "Initializing Actor..."})
[apify] INFO  System info ({"apify_sdk_version": "2.2.1", "apify_client_version": "1.8.1", "crawlee_version": "0.5.2", "python_version": "3.13.1", "os": "linux"})
[apify] INFO  System info ({"apify_sdk_version": "2.2.1", "apify_client_version": "1.8.1", "crawlee_version": "0.5.2", "python_version": "3.13.1", "os": "linux", "message": "System info"})
[scrapy.addons] INFO  Enabled addons:
[] ({"crawler": "<scrapy.crawler.Crawler object at 0x7fd081db0ec0>"})
[scrapy.middleware] INFO  Enabled extensions:
['scrapy.extensions.corestats.CoreStats',
 'scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats'] ({"crawler": "<scrapy.crawler.Crawler object at 0x7fd081db0ec0>"})
[scrapy.crawler] INFO  Overridden settings:
{'BOT_NAME': 'titlebot',
 'DEPTH_LIMIT': 1,
 'LOG_LEVEL': 'INFO',
 'NEWSPIDER_MODULE': 'src.spiders',
 'ROBOTSTXT_OBEY': True,
 'SCHEDULER': 'apify.scrapy.scheduler.ApifyScheduler',
 'SPIDER_MODULES': ['src.spiders'],
 'TELNETCONSOLE_ENABLED': False,
 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'}
[apify] INFO  ApifyHttpProxyMiddleware is not going to be used. Actor input field "proxyConfiguration.useApifyProxy" is set to False.
[apify] INFO  ApifyHttpProxyMiddleware is not going to be used. Actor input field "proxyConfiguration.useApifyProxy" is set to False. ({"message": "ApifyHttpProxyMiddleware is not going to be used. Actor input field \"proxyConfiguration.useApifyProxy\" is set to False."})
[scrapy.middleware] INFO  Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware',
 'src.middlewares.TitleDownloaderMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats'] ({"crawler": "<scrapy.crawler.Crawler object at 0x7fd081db0ec0>"})
[scrapy.middleware] INFO  Enabled spider middlewares:
['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'src.middlewares.TitleSpiderMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware'] ({"crawler": "<scrapy.crawler.Crawler object at 0x7fd081db0ec0>"})
[scrapy.middleware] INFO  Enabled item pipelines:
['src.pipelines.TitleItemPipeline',
 'apify.scrapy.pipelines.ActorDatasetPushPipeline'] ({"crawler": "<scrapy.crawler.Crawler object at 0x7fd081db0ec0>"})
[scrapy.core.engine] INFO  Spider opened ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[scrapy.extensions.logstats] INFO  Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})

...

[title_spider] INFO  TitleSpider is parsing <200 https://crawlee.dev/blog>... ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[title_spider] INFO  TitleSpider is parsing <200 https://crawlee.dev/docs/upgrading/upgrading-to-v3>... ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[title_spider] INFO  TitleSpider is parsing <200 https://crawlee.dev/api/core>... ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[scrapy.core.engine] INFO  Closing spider (finished) ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[scrapy.statscollectors] INFO  Dumping Scrapy stats:
{'downloader/request_bytes': 13234,
 'downloader/request_count': 49,
 'downloader/request_method_count/GET': 49,
 'downloader/response_bytes': 1384477,
 'downloader/response_count': 49,
 'downloader/response_status_count/200': 49,
 'elapsed_time_seconds': 3.552904,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2025, 2, 6, 15, 53, 14, 193680, tzinfo=datetime.timezone.utc),
 'httpcompression/response_bytes': 8748919,
 'httpcompression/response_count': 49,
 'item_scraped_count': 48,
 'items_per_minute': None,
 'log_count/INFO': 58,
 'memusage/max': 111607808,
 'memusage/startup': 111607808,
 'offsite/domains': 10,
 'offsite/filtered': 20,
 'request_depth_max': 1,
 'response_received_count': 49,
 'responses_per_minute': None,
 'robotstxt/request_count': 1,
 'robotstxt/response_count': 1,
 'robotstxt/response_status_count/200': 1,
 'start_time': datetime.datetime(2025, 2, 6, 15, 53, 10, 640776, tzinfo=datetime.timezone.utc),
 'urllength/request_ignored_count': 1} ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[scrapy.core.engine] INFO  Spider closed (finished) ({"spider": "<TitleSpider 'title_spider' at 0x7fd081db12b0>"})
[apify] INFO  Exiting Actor ({"exit_code": 0})
[apify] INFO  Exiting Actor ({"exit_code": 0, "message": "Exiting Actor"})