Scrapy source-code repository, wiki and bug-report system
Welcome!. This is the development site of Scrapy, an open source web crawling and screen scraping framework written in Python.
Interested in trying Scrapy?
This site is intended for Scrapy development. If you're only interested in using Scrapy we recommend you visit:
Getting the code
Scrapy uses Mercurial (hg) for managing its code.
Assuming you have Mercurial installed, the following command in a terminal will fetch the most recent code for you:
hg clone http://hg.scrapy.org/scrapy
Browsing the code online
- Browse source: See the full code via a Web interface.
- Timeline: Keep track of recent changes.
Getting involved
The Scrapy project have only recently gone public, so we're a relatively young open source project, but we welcome any kind of contributions or feedback. At the moment, we're focused on finishing the documentation and preparing the first official release (namely, 0.7).
If you want to get involved, feel free to check out the code and start playing with it, and take a look at the community resources if you want to talk with other Scrapy developers. We welcome any kind of feedback.
