Blog

Create a Django development environment on 64-bit Windows for Heroku deployment

Getting Started with Django on Heroku

This weekend I tried to follow Heroku’s Getting Started with Django article. Unfortunately for me, this article is clearly aimed at developers working in a “unix-style environment”, and I develop with a 64-bit Windows machine both at work and at home. Given that we are working with a Python-based framework, I foolishly assumed that it would be trivial to translate these instructions to Windows. How wrong I was.

This article summarises all the issues I encountered and how I eventually worked around them. I found most of the individual answers I needed via Google, but I couldn’t find a single resource that explains this whole process. I hope it saves you some time. The specific error-messages and console output were generated with 64-bit Windows 8.1, but I expect things to be basically the same with 64-bit Windows 7.

Installing Python and Virtualenv

Standard Python Distro

I started off by installing the standard Python v2.7.6 64-bit distro, followed by Pip. I then used Pip to install the Virtualenv package. Everything seemed to be fine until I tried to create a new Virtualenv:

E:\Develop\Heroku\test_project>virtualenv venv --distribute
New python executable in venv\Scripts\python.exe
Installing setuptools, pip...
Complete output from command E:\Develop\Heroku\te...v\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip:
Ignoring indexes: https://pypi.python.org/simple/
Downloading/unpacking setuptools
Cleaning up...
Exception:
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\basecommand.py", line 122, in main
status = self.run(options, args)
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\commands\install.py", line 2
78, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\req.py", line 1177, in prepare_files
url = finder.find_requirement(req_to_install, upgrade=self.upgrade)
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\index.py", line 209, in find_requirement
file_locations, url_locations = self._sort_locations(locations)
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\index.py", line 128, in _sort_locations
sort_path(os.path.join(path, item))
File "C:\Python27\lib\site-packages\virtualenv_support\pip-1.5.4-py2.py3-none-any.whl\pip\index.py", line 109, in sort_path
if mimetypes.guess_type(url, strict=False)[0] == 'text/html':
File "C:\Python27\Lib\mimetypes.py", line 297, in guess_type
init()
File "C:\Python27\Lib\mimetypes.py", line 358, in init
db.read_windows_registry()
File "C:\Python27\Lib\mimetypes.py", line 260, in read_windows_registry
with _winreg.OpenKey(hkcr, subkeyname) as subkey:
TypeError: must be string without null bytes or None, not str

Storing debug log for failure in C:\Users\iantho\pip\pip.log
----------------------------------------
...Installing setuptools, pip...done.
Traceback (most recent call last):
File "C:\Python27\lib\runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "C:\Python27\lib\runpy.py", line 72, in _run_code
exec code in run_globals
File "C:\Python27\Scripts\virtualenv.exe\__main__.py", line 9, in
File "C:\Python27\lib\site-packages\virtualenv.py", line 824, in main
symlink=options.symlink)
File "C:\Python27\lib\site-packages\virtualenv.py", line 992, in create_environment
install_wheel(to_install, py_executable, search_dirs)
File "C:\Python27\lib\site-packages\virtualenv.py", line 960, in install_wheel
'PIP_NO_INDEX': '1'
File "C:\Python27\lib\site-packages\virtualenv.py", line 902, in call_subprocess
% (cmd_desc, proc.returncode))
OSError: Command E:\Develop\Heroku\te...v\Scripts\python.exe -c "import sys, pip; sys...d\"] + sys.argv[1:]))" setuptools pip failed with error code 2Code language: Access log (accesslog)

The underlying error suggests there’s a problem reading a registry key. I found a lot of discussions about issues to do with Python-related registry entries needing to be manually copied to an alternative location on 64-bit Windows for things to work properly. I didn’t like the sound of that, so I tried installing the 32-bit version of Python v2.7.6 instead. That gave exactly the same error message.

NOTE – please see my follow-up post for the reason behind this error and how to fix it.

On the basis that I was having an unknown registry-related issue with Virtualenv on 64-bit Windows, I decided to try using the ActivePython distro instead, since that comes with Pip and Virtualenv pre-installed, along with some other useful packages. This was a good move…

ActivePython Installation

I uninstalled the standard Python distro and installed ActivePython v2.5.7.6 (64-bit) in its place, and then tried running Virtualenv again. This time I got a different error message:

E:\Develop\Heroku\test_project>virtualenv venv --distribute
New python executable in venv\Scripts\python2.7.exe
Not overwriting existing python script venv\Scripts\python.exe (you must use venv\Scripts\python2.7.exe)
Cannot find sdist setuptools-*.tar.gz
Cannot find sdist pip-*.tar.gzCode language: Access log (accesslog)

I did some digging and found that the two “Cannot find…” errors were caused by the absence of the necessary archive files that Virtualenv wants to use to install Pip and Setuptools in the new Virtualenv environment. I found two possible ways to resolve this:

Option 1 – Manual Repair

  • Download Pip package.
  • Download Setuptools package.
  • Copy both of these packages to the virtualenv_support directory of your active Python installation, e.g.: C:\Python27\Lib\site-packages\virtualenv_support

Virtualenv is now working correctly:

E:\Develop\Heroku\test_project>virtualenv venv
New python executable in venv\Scripts\python2.7.exe
Also creating executable in venv\Scripts\python.exe
Installing Setuptools.....................................................................................................................................................................................................................................................done.
Installing Pip.......................................................................................................................................................................................................................................................................................................................................done.Code language: Access log (accesslog)

However, there is another nicer way to fix this issue…

Option 2 – Upgrade Virtualenv

I found a StackOverflow answer that suggested this issue could be resolved by simply upgrading the existing version of Virtualenv in-place from v1.10.1 to v1.11.4:

  • pip install --upgrade virtualenv

Virtualenv is now working correctly:

E:\Develop\Heroku\test_project>virtualenv venv
New python executable in venv\Scripts\python2.7.exe
Also creating executable in venv\Scripts\python.exe
Installing setuptools, pip...done.Code language: Access log (accesslog)

This configuration gives you Pip v1.5.4 and Setuptools v2.2 in each Virtualenv that you create.

Working with Virtualenv on Windows

Activate a Virtualenv

  • Go to the parent directory of the Virtualenv you want to activate, e.g. E:\Develop\Heroku\test_project
  • Run the Virtualenv activate script: venv\Scripts\activate.bat
  • The Command Prompt should display the name of the Virtualenv folder that is currently active, e.g. (venv) E:\Develop\Heroku\test_project

Confirm which Python interpreter is active

  • Start the Python interpreter from a Command Prompt: python
  • Run the following two Python commands:
    • import sys
    • sys.prefix
  • This will print the location of the currently active Python installation, which should be within your current Virtualenv.

Deactivate a Virtualenv

  • Run the equivalent deactivate script from the same Virtualenv location: venv\Scripts\deactivate.bat
  • The command prompt will return to normal (i.e. the name of your previously active Virtualenv will disappear).

django-admin.py error in Virtualenv

If you try to run django-admin.py from within a Virtualenv at this point, you will see the following error:

Traceback (most recent call last):
File "E:\Develop\Heroku\test_project\venv\Lib\site-packages\django\bin\django-admin.py", line 2, in
from django.core import management
ImportError: No module named django.coreCode language: JavaScript (javascript)

Thanks to this StackOverflow post, I discovered the solution – you must use the %VIRTUAL_ENV% variable to invoke the script within your current Virtualenv directly:

  • python %VIRTUAL_ENV%\scripts\django-admin.py startproject my_django_project

Foreman Error

I then found that the Foreman package included in the Heroku Toolbelt would not run:

(venv) E:\Develop\Heroku\test_project>foreman start
Bad file descriptor
C:/Program Files (x86)/Heroku/ruby-1.9.2/lib/ruby/gems/1.9.1/gems/foreman-0.63.0/lib/foreman/engine.rb:372:in `read_nonblock'
C:/Program Files (x86)/Heroku/ruby-1.9.2/lib/ruby/gems/1.9.1/gems/foreman-0.63.0/lib/foreman/engine.rb:372:in `block (2 levels) in watch_for_output'
C:/Program Files (x86)/Heroku/ruby-1.9.2/lib/ruby/gems/1.9.1/gems/foreman-0.63.0/lib/foreman/engine.rb:368:in `loop'
C:/Program Files (x86)/Heroku/ruby-1.9.2/lib/ruby/gems/1.9.1/gems/foreman-0.63.0/lib/foreman/engine.rb:368:in `block in watch_for_output'
18:42:02 web.1 | exited with code 1
18:42:02 system | sending SIGKILL to all processesCode language: Access log (accesslog)

This can be fixed by following these steps:

  • Add your Ruby bin directory to the Windows PATH, e.g. C:\Program Files (x86)\Heroku\ruby-1.9.2\bin
  • As per this Foreman bug report, you should downgrade your Foreman from v0.63 to v0.61:
    • gem uninstall foreman
    • gem install foreman -v 0.61

Gunicorn Compatibility Issues

The final nail in the coffin – it turns out Gunicorn is not compatible with Windows. Although you will still need to configure it in order to make use of it in your Django deployment on Heroku, in order to run Django locally in your Windows development environment, just start it directly via the manage.py script:

(venv) E:\Develop\Heroku\test_project>python manage.py runserver
Validating models...

0 errors found
February 23, 2014 - 18:56:53
Django version 1.6.2, using settings 'my_django_project.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
[23/Feb/2014 18:57:11] "GET / HTTP/1.1" 200 1757Code language: Access log (accesslog)

These instructions should enable you to successfully reach the “specify dependencies with pip” section of the Heroku guide on a 64-bit Windows PC. Obviously it would be nice if Windows users didn’t have to ski uphill like this, but this is the world we live in!

Tags: , , , , , , ,

Leave a Reply