Changeset 1785:cbfc4a9daecb

Timestamp:

10/07/09 21:00:34 (10 months ago)

Author:

Pablo Hoffman <pablo@…>

Branch:

default

Message:

improved documentation of http proxy middleware

Location:

docs

Files:

• faq.rst (modified) (1 diff)
• topics/downloader-middleware.rst (modified) (1 diff)
• topics/settings.rst (modified) (1 diff)

docs/faq.rst

@@ -55 +55 @@
 -----------------------------------
 
-No. Support for HTTP proxies is not currently implemented in Scrapy, but it
-will be in the future. For more information about this, follow `this ticket
-<http://dev.scrapy.org/ticket/71>`_. Setting the ``http_proxy`` environment
-variable won't work because Twisted (the library used by Scrapy to download
-pages) doesn't support it. See `this Twisted ticket
-<http://twistedmatrix.com/trac/ticket/2714>`_ for more info.
+Yes. Support for HTTP proxies is provided (since Scrapy 0.8) through the HTTP
+Proxy downloader middleware. See
+:class:`~scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware`.
 
 Scrapy crashes with: ImportError: No module named win32api

docs/topics/downloader-middleware.rst

@@ -254 +254 @@
    :synopsis: Http Proxy Middleware
 
+.. versionadded:: 0.8
+
 .. class:: HttpProxyMiddleware
 
-   This middleware sets proxy to use for requests, it obeys enviroment
-   variables 'http_proxy', 'https_proxy', and 'no_proxy'
-
+   This middleware sets the HTTP proxy to use for requests, by setting the
+   ``proxy`` meta value to :class:`~scrapy.http.Request` objects.
+
+   Like the Python standard library modules `urllib`_ and `urllib2`_ it obeys
+   the following enviroment variables:
+
+   * ``http_proxy``
+   * ``https_proxy``
+   * ``no_proxy``
+
+.. _urllib: http://docs.python.org/library/urllib.html
+.. _urllib2: http://docs.python.org/library/urllib2.html
 
 RedirectMiddleware

docs/topics/settings.rst

@@ -403 +403 @@
         'scrapy.contrib.downloadermiddleware.redirect.RedirectMiddleware': 600,
         'scrapy.contrib.downloadermiddleware.cookies.CookiesMiddleware': 700,
+        'scrapy.contrib.downloadermiddleware.httpproxy.HttpProxyMiddleware': 750,
         'scrapy.contrib.downloadermiddleware.httpcompression.HttpCompressionMiddleware': 800,
         'scrapy.contrib.downloadermiddleware.stats.DownloaderStats': 850,