Timeline
10/06/08:
- 15:12 Changeset [309:316e231b660e] by
-
Modified ItemDeltas? to work with RobustScrapedItems? instead of ...
- 15:00 Ticket #9 (Define Scrapy core components) closed by
- fixed: In the last meeting with the team we decided that the Scrapy core will …
- 13:19 Ticket #13 (Write homepage introduction) created by
- We need a introduction to Scrapy in our homepage. It should describe …
- 13:18 Ticket #12 (Define a mechanism for editing homepage pages) created by
- Define a mechanism to edit homepage pages. We're currently thinking …
- 13:15 Ticket #11 (Write usable basic spider class) closed by
- fixed: Added BasicSpider? in r301. It will probably need need some further …
- 13:12 Ticket #11 (Write usable basic spider class) created by
- Right now we only have the BaseSpider? class which doesn't provide any …
- 13:08 Ticket #10 (Add some more usable Link Extractors) closed by
- fixed: Added RegexLinkExtractor? in r302
- 13:06 Ticket #10 (Add some more usable Link Extractors) created by
- Add a more usable Link Extractor to define how a spider should following …
- 13:04 Ticket #9 (Define Scrapy core components) created by
- Define the core components of Scrapy and sort of the code so they remain …
- 12:59 Ticket #8 (Write scrapy tutorial for creating your first spider) created by
- Write a simple tutorial for a simple first spider. Perhaps scraping Google …
- 12:56 Ticket #7 (Write some basic adaptors) created by
- Write some basic adaptors for the most common tasks like extraction (from …
- 09:58 Changeset [308:29d0c1bbf719] by
-
Fixed bug for unicode support.The empty string () in some platforms is ...
- 08:14 Changeset [307:f87b1a2f1f70] by
-
Added ItemDelta? objects and modified replays to make use of them
- 01:23 Changeset [306:d9d6230095ab] by
-
removed unneeded DEFAULT_DATA_ENCODING and commented COMMANDS_MODULE in ...
- 01:22 Changeset [305:4597b9e1c258] by
-
added setadaptors method to ScrapedItem?, removed incorrect constructor
- 00:36 Changeset [304:53e7d3f55944] by
-
removed unused imports and minor bug fix to media/image pipeline
10/05/08:
- 16:32 Changeset [303:e01a349cffcc] by
-
removing utf-16 xpathselector_iternodes testcase since the problem comes ...
- 05:57 Changeset [302:001204823c87] by
-
added RegexLinkExtractor? in new scrapy.link.extractors module
- 05:45 Changeset [301:dad9a5b25bd2] by
-
added scrapy.contrib.spiders module
- 05:39 Changeset [300:1fbebb30a107] by
-
added scrapy.utils.response module
- 05:37 Changeset [299:393ca306f33f] by
-
improved scrapy-admin.py script and default project template
- 05:35 Changeset [298:507eaefc2123] by
-
removed old comment
- 04:58 Changeset [297:2c3961f304ea] by
-
added nonzero method to XPathSelector
10/03/08:
- 16:50 Changeset [296:7c46a4dc7259] by
-
added NotSupported? Exception
- 11:37 Changeset [295:2a219c647af0] by
-
Added test for utils/markup.py. Added support to unicode to the new markup ...
- 08:57 Changeset [294:7b8bf2670ca1] by
-
Added new functions to parse html.
- 01:14 Changeset [293:e8036f5f34ca] by
-
added another test for safe_url_string function
10/02/08:
- 19:41 Changeset [292:8d11ca2543c2] by
-
reverted an experimental code that should have been commited
- 17:39 Changeset [291:1f203db25492] by
-
added DOWNLOAD_DELAY comment in settings template
- 16:59 Changeset [290:1e28d7fdd693] by
-
added support for global DOWNLOAD_DELAY setting
- 16:22 Changeset [289:206880fae84a] by
-
allow to directly specify which domain corresponds to a given request
09/30/08:
- 16:57 Changeset [288:6a7cc5a534c8] by
-
simplified test without loosing functionality
- 16:24 Changeset [287:f2736aab761e] by
-
added test for r284
- 13:56 Changeset [286:ea4294de0b24] by
-
better management of some redirection loops.
- 13:19 Changeset [285:688f07ba54d5] by
-
small fix
- 11:15 Changeset [284:87ac2d7adc10] by
-
safe_url_string should not escape unreserved marks (see RFC 2396, sec 2.3)
- 10:44 Changeset [283:fff1d4064041] by
-
- copied original request to response.request in get_url method - deleted ...
09/29/08:
- 15:14 Changeset [282:e7af2d20d017] by
-
- Added the posibility of knowing the decompressed response's format, in ...
- 10:21 Changeset [281:149b8173258a] by
-
rolled back public ent_re to private and added a function has_entities ...
- 09:52 Changeset [280:9716ad99f4b9] by
-
changed private html entity regex to public
09/25/08:
- 09:49 Changeset [279:d1dd66016ea2] by
-
added comment
- 09:45 Changeset [278:16ad4995eca2] by
-
small fix to the regex
- 09:22 Changeset [277:9fb775f438c6] by
-
images: use brief exception logging
09/24/08:
- 23:14 Changeset [276:6abe09ae4b61] by
-
allow to override item class adaptor in constructor
- 15:19 Changeset [275:13f5b2b0e14e] by
-
added support for xml-declared encodings
- 09:40 Changeset [274:fd20e0ee6a63] by
-
added test for remove entities
- 09:25 Changeset [273:43df3a99077a] by
-
html entities regexp improvement
09/23/08:
- 19:18 Changeset [272:798b3b5f1234] by
-
moved scrapy.core.log module to scrapy.log
- 18:52 Changeset [271:d1670998ba5c] by
-
moved scrapy.core.mail module to scrapy.mail
- 18:48 Changeset [270:26c4d9d1c6a8] by
-
added scrapy.utils.defer, moved deferred functions from scrapy.utils.misc ...
- 18:34 Changeset [269:fcde243f2228] by
-
removed location_str function incorrectly added to this module
- 17:36 Changeset [268:b69cb7ddaa8c] by
-
removed duplicated function convert_entity from scrapy.utils.misc
- 08:46 Ticket #5 (Remove scrapy.utils.misc.unquote_html) closed by
- fixed: done in r267
09/22/08:
- 15:04 Changeset [267:cbd270dc632b] by
-
removed unquote_html
09/19/08:
- 17:03 Ticket #4 (Deploy scrapy.org homepage) closed by
- fixed
- 17:00 Changeset [266:77249aed5c4c] by
-
Change site-media with static in css file
- 16:57 Changeset [265:66f8d1a5c4e6] by
-
Comment blog templatetags calls
- 16:53 Changeset [264:88099d4cc7f1] by
-
Remove blog url an installed app setting
- 16:19 Changeset [263:f2e96b328f4e] by
-
Remove blog application which is not django 1.0
- 16:17 Changeset [262:44fdb4735206] by
-
Django 1.0 support over download application
- 13:57 Changeset [261:65dd393b0e25] by
-
remove decobot reference and fix test_plugin
- 13:57 Changeset [260:1da99f84f1f7] by
-
remove decobot reference
- 12:21 Changeset [259:7f8da8cfd8a3] by
-
grrr.. stupid bugfix
- 11:11 Changeset [258:d5a3b33c4d65] by
-
removed references to python2.5
- 09:58 Ticket #6 (Add spider argument to settings) created by
- Sometimes spiders need to override settings, but there is currently no way …
- 09:50 Changeset [257:a5f74b6dd472] by
-
reverted changeset 254
09/18/08:
- 17:19 Changeset [256:f1307eefb3f4] by
-
ouch! reverted a failed commit and now yes, removed an unneeded None
- 17:13 Changeset [255:01a553c266ca] by
-
removed unneeded None
- 17:11 Changeset [254:aa6d49861e0f] by
-
allow to set download delay as a scrapy setting
- 16:21 Changeset [253:4948d9208447] by
-
bump version
- 14:48 Changeset [252:aa5a86414c05] by
-
include templates
- 13:08 Changeset [251:dd31b26716e4] by
-
new setup.py
- 11:58 Changeset [250:89558d4c4488] by
-
in the way of setuptooling
09/17/08:
- 09:34 Changeset [249:a114d4344f4f] by
-
use request.deferred for cached valued too
09/16/08:
- 20:40 Changeset [248:2cab2509abaf] by
-
media: use deferred of request to allow adding callback from ...
- 20:40 Changeset [247:ffa384206046] by
-
add image pipeline that stores and can thumbs images in different sizes
- 20:39 Changeset [246:63ab4724fa90] by
-
move media_to_download hook to allow caching and prevent downloading
- 16:31 Changeset [245:19ac29207c2c] by
-
mediapipeline: change cache variable by domaininfo
- 11:18 Ticket #2 (Implement media pipeline) closed by
- fixed: implemented at …
- 11:03 Changeset [244:af3a21fc4eea] by
-
messy typo adding errback
- 08:56 Changeset [243:5cbce03f57d0] by
-
Scrapy architecture diagram 1st version, dia source
- 08:30 Changeset [242:30d43e30d5e7] by
-
added docs dir, guess for what? :)
09/15/08:
- 12:37 Changeset [241:dabac5c48c3f] by
-
renamed scrapy.lib to scrapy.xlib
- 12:29 Changeset [240:eca8f9ca9e6d] by
-
removed simplejson from scrapy code
- 11:55 Ticket #5 (Remove scrapy.utils.misc.unquote_html) created by
- Remove unquote_html which whoose functionality is a duplicate of the …
- 10:55 Changeset [239:e23aaa70e373] by
-
ername methods and enforce extraction of Request objects as item media to ...
- 10:08 Changeset [238:e197360ad0fa] by
-
improved fix to support IE7 (thanks bill\!)
- 09:58 Changeset [237:004a147ccabe] by
-
fixed hack for IE6
- 09:56 Changeset [236:43994444805d] by
-
added hack for IE6
- 09:46 Changeset [235:272c0f22bca2] by
-
move bugtraps to avoid silence of bugs in media_downloaded or media_failed
- 05:57 Changeset [234:92302b2b8373] by
-
small bugfix and add bugtraps
- 05:27 Changeset [233:e8b316f3ec50] by
-
remove item.image_urls from get_urls_from_item
- 04:08 Changeset [232:986eeef678e0] by
-
add info object to public methods
- 03:01 Changeset [231:9efda0c9d303] by
-
Adds docstrings to public methods, remove unused imports, normalize urls ...
- 01:33 Changeset [230:04d10118f899] by
-
add media_to_download hook support, and return cached result if available
09/14/08:
- 23:34 Ticket #4 (Deploy scrapy.org homepage) created by
- Deploy scrapy.org Django homepage written by Matias.
- 23:32 Ticket #3 (Implement adaptor infrastructure) created by
- Implement the adaptors infrastructure and write some common basic …
- 23:30 Ticket #2 (Implement media pipeline) created by
- Implement media pipeline as discussed in our meetings, and based on the …
- 22:23 Changeset [229:03643a610439] by
-
removed reference to guid (decobot specific) attribute in item pipeline ...
09/12/08:
- 18:55 Changeset [228:4ff78c39d1bd] by
-
mediamiddleware: bugfixes and remove prints
- 17:48 Changeset [227:e1f13a264f52] by
-
- Support for not output in cluster status - schedule option by default ...
- 16:58 Changeset [226:ef90a278007e] by
-
add media pipeline skeleton
09/11/08:
- 13:35 Changeset [225:8a4c9af757a4] by
-
Fixed condition of new option --quiet.
- 12:33 Changeset [224:91a28bac3a72] by
-
- leave in adaptors.py only the elemental adaptors pipeline code. - moved ...
- 11:59 Changeset [223:8110e2ab3e4d] by
-
Fixed bug in the total items count. Added new option --quiet to the action ...
- 11:50 Changeset [222:661fa930e1aa] by
-
Small piece of code moved for not complying with conventions
- 09:13 Changeset [221:e7d9bc1c41d8] by
-
some small semantics errors fixed in xpath test
- 08:14 Changeset [220:e933ec04f26d] by
-
implemented extract_unquote method over xpath selectors
09/10/08:
- 11:06 Changeset [219:344b703d1a4b] by
-
- AdaptorPipe? compilation feature to resolve the problem of efficience and ...
09/09/08:
- 23:49 Changeset [218:cc0524c61580] by
-
bugfix in decompressor tool
- 12:05 Changeset [217:4aad6de30ddb] by
-
Added getd command, similar to get but first decompress the ...
- 11:12 Changeset [216:a91709634c32] by
-
pylinted decompressor
- 11:10 Changeset [215:a963dca75d67] by
-
decompressor tool improved
- 11:10 Changeset [214:01c82b133da1] by
-
Added more flexibility to childclasses who override gespider command
- 10:02 Changeset [213:8d99a8e1e0eb] by
-
added sample data for decompression tool's test
- 09:56 Changeset [212:28e574f3afa3] by
-
added response decompression tool
09/08/08:
- 09:06 Changeset [211:d548356aea33] by
-
minor code fix
09/06/08:
- 14:03 Changeset [210:3aa8466a6e75] by
-
Removed PRIORITY constants, added DEFAULT_PRIORITY setting
