Ticket #142 (new defect)

Opened 5 months ago

Last modified 6 weeks ago

Unhandled error in Images Pipeline

Reported by: pablo Owned by: daniel
Priority: major Milestone: 0.10
Component: code Version: 0.8
Keywords: Cc: daniel pablo

Description

Sometimes, when crawling some spiders (with the Images Pipeline enabled) we're getting this error:

2010-03-10 10:41:54-0200 [-] Unhandled error in Deferred:
2010-03-10 10:41:54-0200 [-] Unhandled Error
	Traceback (most recent call last):
	  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 312, in _startRunCallbacks
	    self._runCallbacks()
	  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 328, in _runCallbacks
	    self.result = callback(self.result, *args, **kw)
	  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 243, in callback
	    self._startRunCallbacks(result)
	  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 312, in _startRunCallbacks
	    self._runCallbacks()
	--- <exception caught here> ---
	  File "/usr/lib/python2.6/dist-packages/twisted/internet/defer.py", line 328, in _runCallbacks
	    self.result = callback(self.result, *args, **kw)
	  File "/home/prh/hg/scrapy/scrapy/contrib/pipeline/images.py", line 211, in media_downloaded
	    raise ImageException
	scrapy.contrib.pipeline.images.ImageException: 

Which has several problems:

  • it doesn't log the spider where it happened
  • it doesn't log any message with the reason why it failed, you have to follow the traceback
  • it gets unhandled

We need to sort this out.

Change History

Changed 5 months ago by daniel

Have you anytips on how to reproduce this?

Changed 6 weeks ago by pablo

  • milestone changed from 0.9 to 0.10
Note: See TracTickets for help on using tickets.