Timeline


and .

01/23/09:

22:31 ScrapyLogoCandidates edited by pablo
(diff)
20:48 ScrapyLogoCandidates edited by pablo
(diff)
20:46 logo-4b.jpg attached to ScrapyLogoCandidates by pablo
20:46 logo-4a.jpg attached to ScrapyLogoCandidates by pablo
01:42 Changeset [762:80b10cd47bc8] by Daniel Grana <dangra@…>

duplicatesfilter: first version of configurable duplicate requests ...

01/22/09:

16:09 Ticket #56 (investigate redirections handling in cache) created by samus_
Seems that the cache is not handling redirections properly: first run …
15:13 Changeset [761:e60f32d7a382] by elpolilla

Fixed syntax error in test

14:41 Changeset [760:2d5fa9dd722c] by elpolilla

Several modifications done: . attribute method moved from ScrapedItem? to ...

14:40 Changeset [759:2b4774aa9f39] by elpolilla

Added get_base_url to utils.response and tests

14:39 Changeset [758:c4a5eb6df41d] by elpolilla

Changes in HTMLImageLinkExtractor: . Fixed little bug that triggered ...

12:28 Changeset [757:b0cc8e31d024] by Daniel Grana <dangra@…>

tutorial: fix reference to scrapy-ctl. contributed by Patrick Mezard ...

01:19 Changeset [756:4220311c4bb9] by Daniel Grana <dangra@…>

RetryMiddleware?: added ConnectionLost? to retried exceptions

twisted >8.0 ...

01/20/09:

19:49 Changeset [755:9d85b1d71a1e] by Pablo Hoffman <pablo@…>

removed Request.append_callback() method (it was just an alias to ...

19:10 Changeset [754:28c0de8b3d4d] by Pablo Hoffman <pablo@…>

renamed Request url_encoding constructor argument to encoding. added ...

19:00 Changeset [753:59a91b7b1628] by Pablo Hoffman <pablo@…>

added Content-Length header population to Common downloader middleware

16:29 Ticket #55 (Error message when generating a spider with duplicated domain is incorrect) created by ismael
When you try to generate a spider with a new name but its domain is …
12:56 Ticket #54 (ENABLED_SPIDERS_FILE not honoured with project default settings) created by ismael
Right now, after creating a project in its settings the variable …
12:41 Ticket #53 (spider_crawl template should use MyItem instead of ScrapedItem) created by ismael
The startproject command creates an items.py file with MyItem? defined in …

01/19/09:

12:01 Changeset [752:d2cb3d8fa595] by Ismael Carnales <icarnales@…>

make the install scrapy code steps a list, so it doesn't show as sepparate ...

11:51 Changeset [751:9b4ea1027733] by Ismael Carnales <icarnales@…>

removed $ from commands in install it doesn't look so nice but it ...

11:48 Changeset [750:5058dd1607a3] by Ismael Carnales <icarnales@…>

changed (and fixed) download links for windows libraries in install

11:37 Changeset [749:bdd5cef825f8] by Ismael Carnales <icarnales@…>

corrected arch linux install information

01:14 Changeset [748:b4b881a6355c] by Pablo Hoffman <pablo@…>

minor update to topics/settings.rst

01/18/09:

22:35 Changeset [747:af2689d4315a] by Daniel Grana <dangra@…>

docs: fix MailSender? and Settings method references

17:31 Changeset [746:6ec3b69716a4] by Pablo Hoffman <pablo@…>

added additional test to ResponseSoup? extension

17:20 Changeset [745:2b28e85dea41] by Pablo Hoffman <pablo@…>

some minor performance improvements in downloader handlers, added ...

15:52 Changeset [744:cf1d079014f5] by Pablo Hoffman <pablo@…>

added Request.replace method, improved tests for replace/copy method in ...

14:55 Changeset [743:67b4fee0d646] by Pablo Hoffman <pablo@…>

removed 'domain' from Request attributes and constructor arguments

14:38 Changeset [742:5ed91bd4408a] by Pablo Hoffman <pablo@…>

removed domain argument from Response constructor

14:36 Changeset [741:95397dfa714a] by Pablo Hoffman <pablo@…>

removed 'domain' argument from Response objects constructor. besides being ...

01/17/09:

21:57 Changeset [740:6744316304c1] by Pablo Hoffman <pablo@…>

added meta argument to Request & Response constructors

21:09 Changeset [739:6c330b30e39b] by Pablo Hoffman <pablo@…>

removed Request.context attribute (use Request.meta instead)

20:11 Changeset [738:42ca4bf8a9f0] by Pablo Hoffman <pablo@…>

renamed to_string() Request and Response methods to httprepr(). removed ...

19:05 Changeset [737:bd3dbcf6b827] by Pablo Hoffman <pablo@…>

updated request/response reference doc

18:40 Changeset [736:46a16fbe5e40] by Pablo Hoffman <pablo@…>

More Request/Response cleanup:

  • made status attribute an int
  • made ...
13:57 Changeset [735:19e712c8aa34] by Pablo Hoffman <pablo@…>

removed deprecated original_url attribute from Response objects (it can be ...

13:22 Changeset [734:f307ca45712a] by Pablo Hoffman <pablo@…>

changed log message and increased log level, when spiders return objects ...

01/16/09:

09:30 scrapylogo-2.jpg attached to ScrapyLogoCandidates by pablo
09:30 scrapylogo-1.jpg attached to ScrapyLogoCandidates by pablo
09:30 ScrapyLogoCandidates created by pablo

01/15/09:

15:31 Ticket #52 (Get rid of (Twisted) monkey patches) created by pablo
We have been using some Twisted monkey patches (for the HTTPClientFactory …
01:24 Changeset [733:ba2294203e26] by Pablo Hoffman <pablo@…>

Response class:

  • added meta and cache attributes to Response class
  • ...
01:06 Changeset [732:7f65e1565d46] by Pablo Hoffman <pablo@…>

added tests for ResponseSoup? and ResponseLibxml?2 extensions

01/14/09:

22:24 Ticket #51 (Remove response instances references from ItemPipeline) closed by daniel
fixed: closed by r729
22:20 Changeset [731:52829ed5202d] by Pablo Hoffman <pablo@…>

doc; removed referer argument from Request constructor

22:10 Changeset [730:acb44e22984b] by Pablo Hoffman <pablo@…>

removed referer argument from Request constructor. refs #48

21:59 Changeset [729:53450f302622] by Daniel Grana <dangra@…>

Remove response referneces from pipelines. refs #51

21:56 Ticket #51 (Remove response instances references from ItemPipeline) created by daniel
Currently item pipelines receives the response from where an item was …
21:50 Changeset [728:fe73191e792a] by Pablo Hoffman <pablo@…>

removed Request.method magic in Request constructor. refs #48

21:31 Changeset [727:b50b4282ddf4] by Pablo Hoffman <pablo@…>

removed unused (and broken) prepend_callback Request method

21:23 Changeset [726:50a3d3efc869] by Pablo Hoffman <pablo@…>

shortened some line widths

20:02 Changeset [725:0d5838be4209] by Pablo Hoffman <pablo@…>

moved HTTP auth functionality out of Request class and into ...

16:59 Changeset [724:86413a147e93] by Andres Moreira <elkpichico@…>

Added dns cache support for the crawler, improving the performance of the ...

10:12 Changeset [723:2448fba254ba] by samus_

typo

10:09 Changeset [722:4abc45c33208] by samus_

renamed expiration setting to the same used by the image pipeline

09:24 WikiStart edited by pablo
(diff)

01/13/09:

23:17 Ticket #50 (Get fingerprint out of Request class) closed by pablo
fixed: done in r721
23:17 Changeset [721:fd1bb0a980f0] by Pablo Hoffman <pablo@…>
  • moved request fingerprinting from Request class to scrapy.utils.request ...
22:19 Changeset [720:bddca0f5c714] by Pablo Hoffman <pablo@…>

added documentation for settings: ENGINE_DEBUG, DOWNLOADER_DEBUG

20:19 Ticket #50 (Get fingerprint out of Request class) created by pablo
We should get the fingerprinting code out of the Request class and into a …
19:47 Changeset [719:3886a63c94f5] by Pablo Hoffman <pablo@…>

decreased logging level of RequestLimitMiddleware? to DEBUG

19:05 Ticket #49 (Move duplicate filtering into a scheduler middleware) created by pablo
We need to move the duplicate filtering stuff out of the scheduler and put …
13:44 WikiStart edited by pablo
(diff)
12:43 Changeset [718:3500a840fcdf] by Pablo Hoffman <pablo@…>

added DOWNLOAD_DELAY to default_settings and documentation, fixed some ...

11:50 Changeset [717:97e64f76f6f2] by Pablo Hoffman <pablo@…>

removed my email from CLOSEDOMAIN_NOTIFY setting

09:55 Changeset [716:df0dd67df9c2] by Pablo Hoffman <pablo@…>

updated doc: ref/emails.rst and topics/downloader-middleware.rst

08:10 Changeset [715:2f7cfa5487ea] by Pablo Hoffman <pablo@…>

removed unused imports

00:49 Changeset [714:500d31ccbdf2] by Pablo Hoffman <pablo@…>

renamde old SchedulerStats? web console module to ScheduleQueue? and made it ...

01/12/09:

23:46 Changeset [713:0cae7d31b1e5] by Pablo Hoffman <pablo@…>

removed unused (Django) classes from scrapy.utils.datatypes: MergeDict?, ...

23:37 Changeset [712:22bc03ef3d7c] by Pablo Hoffman <pablo@…>

added len to PriorityQueue/Stack?, and changed iter implementation ...

23:14 Changeset [711:ad66f44836ae] by Pablo Hoffman <pablo@…>
20:43 Changeset [710:0c399efe1023] by samus_

removed extra return

15:18 Changeset [709:62f7aabe2b13] by Andres Moreira <elkpichico@…>

Small change in canonicalize_url improved its performance a bit

14:53 Ticket #48 (Refactor Request and Response classes) created by pablo
Refactor the Request and Response classes and update the documentation
14:50 Ticket #47 (Refactor scrapy.http.Headers class) created by pablo
The scrapy.http.Headers class needs some refactoring and cleanup, as (as …

01/11/09:

22:53 Changeset [708:075ade8e2286] by Pablo Hoffman <pablo@…>

added settings: REQUEST_HEADER_ACCEPT, REQUEST_HEADER_ACCEPT_LANGUAGE. ...

21:04 Changeset [707:ece24bc8a741] by Pablo Hoffman <pablo@…>

ported MailSender? class to use twisted non-blocking IO

19:27 Changeset [706:53d27232e923] by Pablo Hoffman <pablo@…>

renamed StackTraceDebug? extension to StackTraceDump?

18:04 Changeset [705:0f6914fa100d] by Pablo Hoffman <pablo@…>

improved settings doc

17:49 Changeset [704:bb0e42df4a8f] by Pablo Hoffman <pablo@…>

some email doc improvments

17:48 Changeset [703:540937bb86fc] by Pablo Hoffman <pablo@…>

added logging doc

17:14 Changeset [702:8a12123648e5] by Pablo Hoffman <pablo@…>

moved email doc to reference (instead of topics)

17:11 Changeset [701:8e5c7c97c715] by Pablo Hoffman <pablo@…>

added doc for scrapy.mail

04:35 Ticket #33 (Write documentation for Web Console) closed by pablo
fixed: added in r700
04:35 Ticket #29 (Write documentation for Extensions) closed by pablo
fixed: added in r700
04:34 Changeset [700:858dbac0f84b] by Pablo Hoffman <pablo@…>

added doc for extensions and web console (closes #29 and #33). also ...

04:31 Changeset [699:84b4d4050f7e] by Pablo Hoffman <pablo@…>

minor (and inoffensive) code improvements and fixes found while ...

01/09/09:

20:45 Changeset [698:cb9164227535] by Pablo Hoffman <pablo@…>

some minor doc improvements here and there

20:45 Changeset [697:d59aa7b74e5d] by Pablo Hoffman <pablo@…>

added misc section to doc

19:18 Changeset [696:41fd5f3217a8] by Pablo Hoffman <pablo@…>

added FAQ entry about Django

15:35 Changeset [695:c790fb8a8e40] by elpolilla

Bugfix in AWSMiddleware regarding requests from local files

11:45 Changeset [694:8c55eea1d4ad] by elpolilla

Improved adaptors documentation

08:49 Changeset [693:bc5dcfab8b94] by elpolilla

Added needed conversion from unicode to string before using twisted's ...

08:35 Changeset [692:f71a0ab6b10c] by Pablo Hoffman <pablo@…>

added docstrings to unicode_to_str and str_to_unicode

01/08/09:

23:33 Changeset [691:30ea2e47d4c3] by samus_

small improvement to Response.init testcase

22:45 Changeset [690:220dd20acbe3] by samus_

added test for Response.init

22:42 Changeset [689:a87aa809be53] by samus_

refactored ResponseBody?'s encoding test

16:29 Changeset [688:a2add5bda313] by samus_

a bit of performance

16:29 Changeset [687:b08f43c75590] by samus_

fixed bug

14:08 Changeset [686:d9298fdf5d23] by samus_

fixed bug in copy method of Response (tests coming soon)

13:42 Changeset [685:62d7229789bc] by elpolilla

Changed method process_spider_output name to process_results in crawl and ...

13:03 Ticket #46 (Adapt Sphinx default template for Scrapy colors) closed by pablo
fixed: New CSS added in r684. Thanks Ezequiel.
13:02 Changeset [684:0c3037e8a416] by Pablo Hoffman <pablo@…>

added custom CSS for scrapy doc with minor modifications (colors only)

13:01 Ticket #44 (Improve Overview documentation) closed by pablo
fixed: improved overview doc added in r667
12:21 Changeset [683:5d592ed8524f] by Pablo Hoffman <pablo@…>

removed scrapyengine import from downoader code, minor improvements to ...

11:46 Changeset [682:56a4aea5142c] by elpolilla

Disabled test until memory leak in libxml is fixed

11:30 Changeset [681:50435f8b6ead] by Pablo Hoffman <pablo@…>

reverted r680 until we there are tests and documentation available about ...

09:05 Changeset [680:614e90bdfe94] by elpolilla

. Added strip() to link texts in LinkExtractor? in order to avoid ...

08:11 Changeset [679:c6c26ff84bbc] by elpolilla

Changed process_results name in feed spiders to process_spider_output and ...

01/07/09:

19:25 Ticket #35 (Improve parse command) closed by elpolilla
fixed: This was already finished some time ago in several revisions from r450, to …
16:06 Changeset [678:d2b72b70d970] by Pablo Hoffman <pablo@…>

reverted r675 which broke precedence for environment variables

16:04 Changeset [677:f0d5a5e662c5] by Pablo Hoffman <pablo@…>

updated settings doc

15:39 Changeset [676:a76079a8ec01] by elpolilla

Fixed bug in RegexLinkExtractor?. Encoding was not being specified

15:31 Changeset [675:38f788678898] by elpolilla

. Modified loading of command-specific settings (were loaded as defaults, ...

12:18 Changeset [674:5fdf0d0e60e5] by elpolilla

Removed non-scrapy import in shell command

12:02 Changeset [673:3b6f8a227972] by Pablo Hoffman <pablo@…>

minor update to overview doc

12:00 Changeset [672:23fff245c864] by elpolilla

Modified extract adaptor to make use of "adaptor_args" (as it should), and ...

10:56 Changeset [671:20644bfc6afe] by Pablo Hoffman <pablo@…>

minor corrections to overview doc

10:48 Changeset [670:99f7bf30711a] by elpolilla

Refactored extract adaptor

10:09 Changeset [669:c43279016338] by elpolilla

Small bugfix in selectors constructor regarding strings and unicodes

09:47 Ticket #21 (Write documentation for Spiders) closed by elpolilla
fixed: Done in r635.
09:47 Ticket #24 (Write documentation for Items) closed by elpolilla
fixed: Finished in r648.
09:46 Ticket #25 (Write documentation for Selectors) closed by elpolilla
fixed: This was finished in r636 and r645 already.
09:41 Ticket #46 (Adapt Sphinx default template for Scrapy colors) created by pablo
Right now Scrapy documentation is using Sphinx default templates and style …
09:38 Ticket #45 (Create logo for Scrapy) created by pablo
Please create a nice logo for Scrapy. We were thinking something like a …
01:59 Changeset [668:071698a330ac] by Pablo Hoffman <pablo@…>

somes fixes and updates to scrapy documentation

01:58 Changeset [667:cfd68de77fba] by Pablo Hoffman <pablo@…>

improved overview doc. closes #44

01:18 Changeset [666:df523c32b720] by elpolilla

Added documentation for Adaptors

01/06/09:

23:02 Changeset [665:1f1ce51ad1ed] by Pablo Hoffman <pablo@…>

made url_is_from_spider work for tuples in extra_domain_names

21:07 Changeset [664:0d72760baae2] by Pablo Hoffman <pablo@…>

doc: minor grammar correction

21:05 Ticket #31 (Write documentation about Scrapy Architecture) closed by pablo
fixed: added in r663
21:05 Changeset [663:0816a8c30e07] by Pablo Hoffman <pablo@…>

added doc about scrapy architecture. closes #31

17:43 Changeset [662:e73749ff8cc7] by olveyra

fix to revision 660

15:34 Changeset [661:a9dc484ad04e] by Pablo Hoffman <pablo@…>

updated robotstxt, spidermw and downloadermw docs

15:30 Changeset [660:5b2db2efa6cc] by Pablo Hoffman <pablo@…>

renamed spider middleware methods to more consistent ones: ...

14:54 Ticket #39 (Add example bot used in the tutorial to scrapy/examples) closed by pablo
fixed: added by elpolilla in r656
14:54 Ticket #28 (Write documentation for Spider Middlewares) closed by pablo
fixed: added in r659
14:53 Changeset [659:5f12ddff334e] by Pablo Hoffman <pablo@…>

added spider middleware documentation. closes #28

14:47 Ticket #44 (Improve Overview documentation) created by pablo
Improve the Overview documentation adding some diagrams of Scrapy …
14:45 Ticket #43 (Write documentation for Stats) created by pablo
Write documentation for Stats
14:44 Changeset [658:0501acf32c0a] by Pablo Hoffman <pablo@…>

removed unused import

14:42 Changeset [657:349516afcd44] by Pablo Hoffman <pablo@…>

added DEFAULT_SPIDER to default_settings and fixed formatting in ...

14:08 Changeset [656:da6da3a1d49a] by elpolilla

Added the example project from the tutorial

12:34 Changeset [655:6d97a6a456a8] by Pablo Hoffman <pablo@…>

doc: fixed intro-install xref

01/05/09:

22:19 Changeset [654:1f0cb4e6c933] by Pablo Hoffman <pablo@…>

updated INSTALL file to point to docs/intro/install. also, utf-8 default ...

22:15 Changeset [653:6ab3a00a7b42] by Pablo Hoffman <pablo@…>

updated installation doc

22:15 Changeset [652:ddc3ff9ea616] by Pablo Hoffman <pablo@…>

updated documentation index

21:49 Changeset [651:7b344f963882] by Pablo Hoffman <pablo@…>

added note to scrapy version

21:29 Changeset [650:738fac959e1c] by Pablo Hoffman <pablo@…>

updated some doc configuration - version, release and html_copy_source

16:11 Changeset [649:8178448cd1f3] by olveyra

added DEFAULT_SPIDER setting help

14:55 Changeset [648:ff2a8bfd21e6] by elpolilla

Updated items documentation

14:54 Changeset [647:df3f0042ce44] by elpolilla

. Moved init and repr methods from RobustScrapedItem? to ScrapedItem? . ...

01:37 Changeset [646:4cdf212ea86f] by Pablo Hoffman <pablo@…>

updated scrapy version

00:49 Changeset [645:3c9f87a0cd9b] by Pablo Hoffman <pablo@…>

some improvements to selectors doc structure, added literalincludes for ...

01/04/09:

21:28 Changeset [644:32259c50c0f4] by Pablo Hoffman <pablo@…>

disabled docs app (redirecting to doc.scrapy.org instead)

20:34 WikiStart edited by pablo
cleaned up Trac homepage to be more friendly for newcomers (diff)
17:03 Changeset [643:53e56ee57457] by Pablo Hoffman <pablo@…>

fixed wrong xref name

16:23 Changeset [642:2a88e4734b66] by samus_

reconfigured scrapy Response in order to disallow direct use of ...

13:37 Changeset [641:e94a980b0f48] by elpolilla

Added Items documentation

12:46 Changeset [640:39a1bff5f6dd] by elpolilla

Fixed unquote adaptor's test

12:25 Changeset [639:a8bc182936e5] by elpolilla

Removed unappropiate unquote adaptor's default behaviour

08:50 Changeset [638:a58cce3c24ee] by elpolilla

Removed already-deprecated response decompression tool

02:54 Changeset [637:2610902c838b] by Pablo Hoffman <pablo@…>

updated some docs paths

01/03/09:

23:15 Changeset [636:2266d68f70f3] by elpolilla

Updated Selectors documentation

22:36 Changeset [635:d032eb886483] by elpolilla

Updated Spiders documentation

07:22 Ticket #42 (Add robots.txt support) closed by pablo
fixed: functionality added in r629, settings in r630, and documentation in r632.
07:20 Ticket #42 (Add robots.txt support) created by pablo
Add support for obeying robots.txt rules, probably using a downloader …
07:14 Changeset [634:3aba84220fe3] by Pablo Hoffman <pablo@…>

doc: several more improvements

05:54 Changeset [633:629276cd8045] by Pablo Hoffman <pablo@…>

doc: added README

05:41 Changeset [632:7f0bd73218c5] by Pablo Hoffman <pablo@…>

doc: added topic about robots.txt, added ROBOTSTXT_OBEY setting, added ...

05:40 Changeset [631:f91b722f346b] by Pablo Hoffman <pablo@…>

minor typo and grammar corrections to docs/faq

05:36 Changeset [630:ac4bf6febf4a] by Pablo Hoffman <pablo@…>

updated default_settings with robots.txt related settings

05:35 Changeset [629:cf2ce29fd148] by Pablo Hoffman <pablo@…>

finished working version of robots.txt downloader middleware, and renamed ...

04:15 Changeset [628:1130691ca184] by Pablo Hoffman <pablo@…>

removed deprecated Request.getitem method

01:11 Changeset [627:87b9bde35903] by Pablo Hoffman <pablo@…>

updated downloader-middleware to link to Request and Response classes

01:10 Changeset [626:7032d029cb9b] by Pablo Hoffman <pablo@…>

added (incomplete) request-response doc

01:05 Changeset [625:d1dd97ef557e] by Pablo Hoffman <pablo@…>

renamed itempipeline.rst to item-pipeline.rst

01/02/09:

23:30 Ticket #27 (Write documentation for Downloader Middlewares) closed by pablo
fixed: added in r624
23:29 Changeset [624:ef193677f817] by Pablo Hoffman <pablo@…>

added documentation for downloader middleware. closes #27

23:25 Changeset [623:3e39e0fb8552] by Pablo Hoffman <pablo@…>

docs: more updates to Makefile and conf.py

23:24 Changeset [622:edfbf47ef60c] by Pablo Hoffman <pablo@…>

added FAQ to docs

22:58 Changeset [621:462c7d78de72] by Pablo Hoffman <pablo@…>

more stripping down of doc Makefile

22:43 Changeset [620:81060eb86f40] by Pablo Hoffman <pablo@…>

added stripped down version of Python documentation Makefile

18:45 Changeset [619:042dc72041d5] by Pablo Hoffman <pablo@…>

simplified obfuscated implementation of duplicated links removal and made ...

18:44 Changeset [618:b805019f2783] by Pablo Hoffman <pablo@…>

removed unused private variable

18:42 Changeset [617:aa76637abf3e] by Pablo Hoffman <pablo@…>

added key argument to unique function

18:17 Changeset [616:23a6310c43e5] by Pablo Hoffman <pablo@…>

minor text changes to home page

18:07 Changeset [615:4b14dcf56861] by Pablo Hoffman <pablo@…>

changed scrapy.org title to include more key words

18:06 Changeset [614:c2280b77ce53] by Pablo Hoffman <pablo@…>

removed ugly apostrophe

17:59 Changeset [613:7583d86263ae] by Pablo Hoffman <pablo@…>

renamed ItemPipeline? class to ItemPipelineManager? to avoid confusions

17:55 Changeset [612:b59a2d4dffd6] by Pablo Hoffman <pablo@…>

added comment about new docs

17:49 Ticket #41 (Write documentation for Exceptions) closed by pablo
fixed: done in r611
17:49 Ticket #40 (Write documentation for Signals) closed by pablo
fixed: done in r611
17:49 Ticket #26 (Write documentation for Item Pipelines) closed by pablo
fixed: done in r611
17:48 Changeset [611:f3f90a8ac178] by Pablo Hoffman <pablo@…>

added documentation for item pipeline, signals and exceptions. changed ...

17:45 Changeset [610:ff02dd6ac253] by Pablo Hoffman <pablo@…>

added -E to sphinx-build so source rst's are be always reloaded and ...

16:59 Changeset [609:156ad49e6497] by elpolilla

. Modified unicode conversion functions to accept the conversion encoding ...

16:48 Ticket #41 (Write documentation for Exceptions) created by pablo
Write documentation for Exceptions
16:47 Ticket #40 (Write documentation for Signals) created by pablo
Write documentation for Signals
16:32 Changeset [608:b95205f49ce0] by elpolilla

. Added tests for RegexLinkExtractor? . Modified LinkExtractor? in order to ...

16:31 Changeset [607:62885e9e08f6] by elpolilla

Modified url_safe_string in order to accept strings encoded in any ...

16:24 Changeset [606:ae03ec3d477d] by Pablo Hoffman <pablo@…>

removed domain_initialized signal which was defined but never used

15:15 Changeset [605:ee32c47aeeae] by Pablo Hoffman <pablo@…>

removed border-bottom on link hover which made trac behave strangly

14:34 Changeset [604:117ef9324c33] by Pablo Hoffman <pablo@…>

more rearrangements to scrapy doc

14:28 Changeset [603:20ae844b74a4] by samus_

created test for body_or_str (forgot to add the file to the repo at r601)

14:25 Changeset [602:0da539c7020b] by Pablo Hoffman <pablo@…>

rearranged doc according to what we agreed on a meeting with elpolilla and ...

14:21 Changeset [601:2478ecc44aa6] by samus_

created test for body_or_str and made small improvement to ...

14:08 Changeset [600:fdeae3898f35] by Pablo Hoffman <pablo@…>

added topics section to docs, changed reference to ref

12:57 Changeset [599:751ba328b7dc] by samus_

reverted generator approach because it conflicts with unique parameter ...

12:06 Changeset [598:dc6d8a891037] by elpolilla

. Modified LinkExtractors? extract_links for being inconsistent. Moved ...

11:07 Changeset [597:f4371169c39a] by elpolilla

Reverted change in r590 for inconsistency in the addition operation ...

06:42 Changeset [596:af3053413268] by samus_

this changeset improves the extractors' implementation:

* moved ...

02:12 Changeset [595:13d4c3e3c6f5] by elpolilla

Disabled ImageLinkExtractor? test for triggering mysterious leaks in ...

00:36 Changeset [594:4d2c22fb70f0] by elpolilla

Added test's missing sample file

00:34 Changeset [593:2d514fd5caa8] by elpolilla

Renamed LinkExtractors? extract_urls method to extract_links

00:25 Changeset [592:9642649066f7] by elpolilla

Added ImageLinkExtractor?'s missing docstring

00:18 Changeset [591:7fa76d69e1a8] by elpolilla

- Added repr method to Link objects - Added ImageLinkExtractor? and tests

12/31/08:

11:23 Changeset [590:e2e7f750b61e] by elpolilla

Modified XPathSelectorLists: adding them should return a new ...

12/30/08:

17:38 Changeset [589:6ec616690ba2] by olveyra

doc string fix

13:04 Changeset [588:8630837d0c4d] by Ismael Carnales <icarnales@…>

corrected code blocks

12:51 Changeset [587:f300ae2734a2] by olveyra

fix to commit (r586)

11:49 Changeset [586:aea8be5017c7] by olveyra

Allow to load a default spider when no spider was found for a given url. ...

11:42 Ticket #12 (Define a mechanism for editing homepage pages) closed by ismael
fixed
11:41 Ticket #8 (Write scrapy tutorial for creating your first spider) closed by elpolilla
fixed: Okay, completed, although many things can (and will) be added in the …
11:35 Ticket #20 (Refactor default settings) closed by pablo
fixed: done in r585
11:34 Changeset [585:1313b5caf2c7] by Pablo Hoffman <pablo@…>

rearranged and sorted out default scrapy settings. closes #20

11:29 Ticket #30 (Write documentation for Settings) closed by pablo
fixed: added in r584
11:28 Changeset [584:fd2e6842258d] by Pablo Hoffman <pablo@…>

added settings documentation topic, and completed available settings ...

11:05 Ticket #13 (Write homepage introduction) closed by pablo
fixed: I've written a small introduction describing the project features and …
11:04 Changeset [583:c286534858ac] by olveyra

Added doc line in RegexLinkExtractor? for the case when no allow/deny ...

09:27 Changeset [582:215f1dac70a5] by Ismael Carnales <icarnales@…>

the badge is back to green ... hulk is angry

09:25 Changeset [581:5469e1897dff] by Ismael Carnales <icarnales@…>

using small gray badge

09:16 Changeset [580:bb93f0e8f944] by Ismael Carnales <icarnales@…>

removed menu from footer, added django badge

08:56 Changeset [579:f6ec9997e192] by Ismael Carnales <icarnales@…>

replicating header menu in footer

08:45 Changeset [578:adba14c43287] by Ismael Carnales <icarnales@…>

fixed settings indentation and added reference

12/29/08:

17:02 Changeset [577:65f793725322] by Ismael Carnales <icarnales@…>

changed settings from description unit to crossreference

16:46 Changeset [576:818f86bfe5a6] by Ismael Carnales <icarnales@…>

added docs breadrumb

16:35 Changeset [575:41eeedee3799] by Ismael Carnales <icarnales@…>

fixed caps in news, reordered menu items

16:28 Changeset [574:d1df78140cc4] by Ismael Carnales <icarnales@…>

make sidebar bigger

16:27 Changeset [573:747c6032e907] by Ismael Carnales <icarnales@…>

removed django.contrib.comments requirement

16:20 Changeset [572:39cc49a130dd] by Ismael Carnales <icarnales@…>

moved blog to news

14:50 Changeset [571:6bf0dad8c19c] by Ismael Carnales <icarnales@…>

using a modified version of django simple blog

14:13 Changeset [570:0281dab846fd] by Ismael Carnales <icarnales@…>

moved blog templates to backup folder

13:54 Changeset [569:a381cd35c39a] by elpolilla

Fixed minor encoding issues in adaptors

13:17 Changeset [568:af126336fc79] by elpolilla

Fixed typo in test

13:10 Changeset [567:84dd03375989] by elpolilla

Fixed lots of encoding issues, and improved some adaptors tests

10:10 Changeset [566:845ca3841fe1] by Pablo Hoffman <pablo@…>

fixed grammar error

09:38 Changeset [565:7cc054340590] by Pablo Hoffman <pablo@…>

started writing settings documentation

12/28/08:

06:45 Changeset [564:168ec97be1d3] by Pablo Hoffman <pablo@…>

fixed cyclic import between scrapy.core.engine and scrapy.utils.db

12/27/08:

19:38 Changeset [563:8aa29060265f] by Pablo Hoffman <pablo@…>

moved scrapy-docs svn:external where it belongs

19:35 Ticket #39 (Add example bot used in the tutorial to scrapy/examples) created by pablo
Please add the example bot used in the scrapy tutorial to a …
19:32 Changeset [562:446ac2d9ddd9] by Pablo Hoffman <pablo@…>

moved dia diagram to docs/media

19:27 Changeset [561:8fa15b20e2c3] by Pablo Hoffman <pablo@…>

moved scrapy docs from website source to scrapy source, since it makes ...

19:20 Changeset [560:02037e934bf2] by Pablo Hoffman <pablo@…>

moved docs to docs-old

17:57 Changeset [559:7d753e3b59bd] by Pablo Hoffman <pablo@…>

added AUTHORS file

17:16 Changeset [558:46bde4252f77] by Pablo Hoffman <pablo@…>

improved text

17:15 Changeset [557:5775aa7e8a51] by Pablo Hoffman <pablo@…>

some updates to home and download page

16:21 Changeset [556:fb9e2e54646a] by Pablo Hoffman <pablo@…>

removed tagline from scrapy logo

15:37 Changeset [555:0465d0391775] by Pablo Hoffman <pablo@…>

added spider_exceptions to scrapy stats

00:06 Changeset [554:2f04f2a8f3f5] by Pablo Hoffman <pablo@…>

fixed minor bug in scrapy manager

00:05 Changeset [553:9c1bf6932800] by Pablo Hoffman <pablo@…>

fixed minor bug in scrapy manager

12/26/08:

23:07 Changeset [552:68ee64eb5bd2] by Pablo Hoffman <pablo@…>

added start_requests method to BaseSpider?, made start_urls empty by ...

16:53 Changeset [551:491478f52b80] by elpolilla

- Improved unquote_markup by using generators instead of lists - Added ...

14:10 Changeset [550:3a04545079f2] by elpolilla

Added items_to_csv function

12:03 Changeset [549:d49f543ba250] by elpolilla

Updated scrapy tutorial

10:21 Changeset [548:5cd9323db8d6] by elpolilla

Updated scrapy overview

09:51 Changeset [547:8bf5988e13af] by elpolilla

Modified XMLFeedSpider in order to support parsing with HtmlXPathSelector

12/24/08:

13:48 Ticket #17 (ImagePipeline: unable to refresh updated images with old filenames) closed by pablo
invalid: You can use the IMAGES_EXPIRE setting to adjust the expiration interval. …
12:14 Changeset [546:f3d326c5878e] by elpolilla

Added tests for str_to_unicode and unicode_to_str

11:02 Changeset [545:53a3706e5284] by elpolilla

Added str_to_unicode and unicode_to_str functions, and used them in ...

Note: See TracTimeline for information about the timeline view.