Timeline
09/14/08:
- 23:34 Ticket #4 (Deploy scrapy.org homepage) created by
- Deploy scrapy.org Django homepage written by Matias.
- 23:32 Ticket #3 (Implement adaptor infrastructure) created by
- Implement the adaptors infrastructure and write some common basic …
- 23:30 Ticket #2 (Implement media pipeline) created by
- Implement media pipeline as discussed in our meetings, and based on the …
- 22:23 Changeset [229:03643a610439] by
-
removed reference to guid (decobot specific) attribute in item pipeline ...
09/12/08:
- 18:55 Changeset [228:4ff78c39d1bd] by
-
mediamiddleware: bugfixes and remove prints
- 17:48 Changeset [227:e1f13a264f52] by
-
- Support for not output in cluster status - schedule option by default ...
- 16:58 Changeset [226:ef90a278007e] by
-
add media pipeline skeleton
09/11/08:
- 13:35 Changeset [225:8a4c9af757a4] by
-
Fixed condition of new option --quiet.
- 12:33 Changeset [224:91a28bac3a72] by
-
- leave in adaptors.py only the elemental adaptors pipeline code. - moved ...
- 11:59 Changeset [223:8110e2ab3e4d] by
-
Fixed bug in the total items count. Added new option --quiet to the action ...
- 11:50 Changeset [222:661fa930e1aa] by
-
Small piece of code moved for not complying with conventions
- 09:13 Changeset [221:e7d9bc1c41d8] by
-
some small semantics errors fixed in xpath test
- 08:14 Changeset [220:e933ec04f26d] by
-
implemented extract_unquote method over xpath selectors
09/10/08:
- 11:06 Changeset [219:344b703d1a4b] by
-
- AdaptorPipe? compilation feature to resolve the problem of efficience and ...
09/09/08:
- 23:49 Changeset [218:cc0524c61580] by
-
bugfix in decompressor tool
- 12:05 Changeset [217:4aad6de30ddb] by
-
Added getd command, similar to get but first decompress the ...
- 11:12 Changeset [216:a91709634c32] by
-
pylinted decompressor
- 11:10 Changeset [215:a963dca75d67] by
-
decompressor tool improved
- 11:10 Changeset [214:01c82b133da1] by
-
Added more flexibility to childclasses who override gespider command
- 10:02 Changeset [213:8d99a8e1e0eb] by
-
added sample data for decompression tool's test
- 09:56 Changeset [212:28e574f3afa3] by
-
added response decompression tool
09/08/08:
- 09:06 Changeset [211:d548356aea33] by
-
minor code fix
09/06/08:
- 14:03 Changeset [210:3aa8466a6e75] by
-
Removed PRIORITY constants, added DEFAULT_PRIORITY setting
09/05/08:
- 18:51 Changeset [209:4d14b487026a] by
-
adaptors with generic matching function
- 11:45 Changeset [208:b28725f595cc] by
-
Added an histogram plot to simpages group to the report. Added quantities ...
09/04/08:
- 16:50 Changeset [207:77a2b78feee7] by
-
- added negative attribute name match
- 16:33 Changeset [206:17e135e92087] by
-
second security fix
- 16:11 Changeset [205:7f9a7d633607] by
-
- removed contrib/adaptors.py - display adaptor name when an exception ...
- 15:15 Changeset [204:ccaf3e6b507d] by
-
security fix
09/03/08:
- 16:11 Changeset [203:45d4a81624ec] by
-
Remove old code.
- 16:06 Changeset [202:07f3bf9afcb7] by
-
Add rule engine to the framework. Rules are executed in a pipeline.
- 10:53 Changeset [201:e44bbd830172] by
-
- Improved attribute name checks - added support to tuple definition of ...
09/02/08:
- 15:52 Changeset [200:ee7a9a955845] by
-
more efficient name attribute check in adaptors pipeline
- 14:39 Changeset [199:da57acac74f3] by
-
- assure deferred_degenerate will take an iterable (bug raised when ...
- 12:41 Changeset [198:121c59a886b7] by
-
removed unused import
- 12:20 Changeset [197:50a2a1215472] by
-
removed canonicalize from get function in shell
- 11:12 Changeset [196:78fc9a1e7002] by
-
fix get function (strip and canonicalize url)
- 09:38 Changeset [195:2c288cdc5c2b] by
-
updated settings template
09/01/08:
- 17:06 Changeset [194:ec036753723c] by
-
- Fixes in adaptors code, after testing - added attrs_list param to ...
- 16:28 Changeset [193:9e6871dbebef] by
-
removed unneeded exception code
- 01:28 Changeset [192:9642e3db1251] by
-
changed Referer middleware class name
- 01:18 Changeset [191:c1171f2b19b1] by
-
improved SpiderMiddleware?'s docstrings
- 01:16 Changeset [190:f816847935bd] by
-
added UrlFilterMiddleware?
- 01:09 Changeset [189:f9741bf0ce15] by
-
added update_fingerprint method to Request
- 00:34 Changeset [188:8dbdf1f41358] by
-
fixed some documentation errors
- 00:33 Changeset [187:800ec953dce1] by
-
changed remove_fragments argument to keep_fragments, for consistency with ...
- 00:31 Changeset [186:beae8d9369aa] by
-
added canonicalize_url function to scrapy.utils.url, along with a complete ...
08/31/08:
- 22:19 Changeset [185:bc8b1458d930] by
-
some functions were added to scrapy.utils.url without following our ...
08/30/08:
- 21:25 Changeset [184:5c7083673eac] by
-
Improved Adaptors code
08/29/08:
08/27/08:
- 14:37 Changeset [183:4d759176d456] by
-
moved some url utils from decobot to scrapy
- 14:21 Changeset [182:6f7ebde1eee0] by
-
- avoid to raise an exception when no arg is given to replay command
- 10:52 Changeset [181:461f51ffc6ba] by
-
- moved scrape command to shell - fixes - get and scrapehelp functions ...
08/24/08:
- 21:00 Changeset [180:b3fefc7972f6] by
-
cleaned up simpages code a bit, added some documentation
- 16:10 Changeset [179:4c49114a6af9] by
-
added prototype page similarity code, to detect different layouts
08/23/08:
- 15:21 Changeset [178:8567f75929cd] by
-
Added a synchronous get method which also updates console user namespace.
08/22/08:
- 10:38 Changeset [177:6ccd28237d68] by
-
allow to use scrape command without an url
08/21/08:
- 14:12 Changeset [176:e63e6e9f6b4f] by
-
reverted clean_markup code movement
- 12:07 Changeset [175:18ca1a39ea1d] by
-
moved clean_markup to scrapy.utils.markup
08/20/08:
- 09:52 Changeset [174:9b3422bcb0cf] by
-
fixed a clean code movement error: forget to apply remove tags when text ...
08/19/08:
- 16:52 Changeset [173:c11852054b75] by
-
added some validation to new spider module names
- 10:42 Changeset [172:6d9a20492cd6] by
-
removed temporal fix in 171
08/18/08:
- 12:53 Changeset [171:065a7b5a3981] by
-
temporal fix to avoid exceptions before commit in decobot
- 12:18 Changeset [170:c1910ec268f7] by
-
- Added generic clean adaptors - removed attribute name from adaptor ...
08/15/08:
- 14:04 Changeset [169:6ca3d16ba58c] by
-
minor fixes
- 11:59 Changeset [168:fb92e69a4e18] by
-
Added support to replay update to crawl again all the pages downloaded in ...
- 09:35 Changeset [167:d3f4063e8abc] by
-
improved explanation comment of the RequestLimitMiddleware?
