Scrapy Release Procedure

1. Make sure all unit tests pass

  • in *nix: install Scrapy using python setup.py install and make sure all unittests pass by running trial scrapy.tests
  • in win32: install Scrapy using windows installer and make sure all unittests pass by running c:\python26\scripts\trial scrapy.tests

2. Update version and tag hg repo

  1. Update version accordingly in setup.py
  2. tag hg repo - for example: hg tag 0.8-rc1

3. Build source tarball

hg purge --all
python setup.py sdist

4. Build win32 release

IMPORTANT: The following commands must be run in a Windows box due to a bug in bdist_wininst command 

hg purge --all
python setup.py bdist_wininst

5. Sign release files

Put Scrapy-X.Y.Z.tar.gz and Scrapy-X.Y.Z.win32.exe in dist/ and run: 

md5sum dist/Scrapy-* > dist/MD5SUMS
sha1sum dist/Scrapy-* > dist/SHA1SUMS
gpg -ba dist/MD5SUMS
gpg -ba dist/SHA1SUMS

6. Upload release files

Upload release and signature files to http://scrapy.org/releases/

7. Branch hg repos and doc (only for major releases)

Assume OLD_VER is the old stable version, and NEW_VER is the new stable version.

  1. cd /home/hg/repos
  2. cp -r scrapy-OLD_VER scrapy-NEW_VER
  3. cd /home/hg/repos; ln -sf scrapy-NEW_VER scrapy
  4. cd /srv; hg clone /home/hg/repos/scrapy-NEW_VER
  5. edit /home/hg/repos/hgadmin/hgweb/hgweb.config and add a line with the new version
  6. edit /home/hg/repos/scrapy-NEW_VER/.hg/hgrc and:
    • update description
    • update changegroup.inotify accordingly
  7. edit /home/hg/repos/scrapy-OLD_VER/.hg/hgrc and:
    • update description
    • update changegroup.inotify accordingly
  8. edit /etc/lighttpd/conf-enabled/30-insophia.conf and adjust doc aliases
  9. edit /etc/incron.d/scrapy-update and
    • add a line for the new version
    • update the version used on scrapy-social script
  10. update /etc/cron.d/scrapy

7. Publish new release on PyPI

  1. Register the new release version with: python setup.py register
  2. Login to PyPI, go to edit Scrapy project, and upload the new release files

Need to enter the URL manually, like: http://pypi.python.org/pypi?:action=files&name=Scrapy&version=0.9

8. Update home page

  1. Update Download page to point to the new release
  2. Update home page News section notifying about the new release

9. Update default version in Trac

Edit /var/trac/scrapy/conf/trac.ini and update default_version value.

10. Send announcement

  • send announcement to Scrapy users group
  • announce in Scrapy twitter
  • change #scrapy IRC channel topic:
    • /msg ChanServ topic #scrapy Scrapy 0.9 released! http://scrapy.org/download/
  • publish new packages to AptRepos
  • edit AptRepos, add NEW_VER, remove OLD_VER (only for major updates)