Category Archives: Development

One Note: Using PyDev for Django with virtualenv

In addition to my last post on Python development environments, here’s a note on using the PyDev plug-in for Eclipse with our virtualenv set-up.

Assuming everything went swimmingly — virutalenv running, Django installed with a project folder (let’s call it ‘myproject’) having done something like this:

$ mkdir dev
$ cd dev
$ virtualenv --no-site-packages env
$ source env/bin/activate
(env) $ pip install django
(env) $ django-admin.py startproject myproject

First install PyDev in Eclipse under “Install New Software” with the URL: http://pydev.org/updates

Once that’s done, configure the Python interpreter. In your Preferences, under PyDev and Interpreter – Python, hit the Auto Config option to find all your libraries and populate the Python Path.

Otherwise, manually add Python and locate your interpreter, mine is under /usr/bin/python2.7.

For now, this only includes your globally-installed system-wide libraries, not those you’ve installed within the virtualenv environment.

To create a new “PyDev Django project” however, you’ll need to have Django installed globally (or otherwise configured in the above Python Path settings) so PyDev can see it. Right now ours isn’t, so we have an extra step.

Instead, we’ll create a “New PyDev project” (non-Django), add our virtualenv location containing the libraries in our local site-packages directory, then convert that to a Django Project once PyDev is satisfied we have the goods.

This method means we don’t have to install Django globally just for the sake of using this IDE.

To do this, from the File menu and New PyDev Project, I un-tick ‘Create src folder and add it to the PYTHONPATH’, instead selecting ‘Don’t configure PYTHONPATH (to be done manually later on)’.

Right-click the project folder, go to Properties and PyDev – PYTHONPATH and add a Source Folder pointing to your virtualenv site-packages. In this instance:

dev/env/lib/python2.7/site-packages

Having found Django, PyDev now let’s us convert this to a Django project. Right-click again and under the PyDev menu select Set as Django Project.

Now everything can be performed within Eclipse, rather than by the command line.

For example, to run the server we’ll add a Custom Command. Under the Django menu select Custom Command and add the following:

runserver --noreload

You may be asked to select which manage.py to run from, choose the one within your project, i.e. myproject/manage.py.

Hit Run and test http://localhost:8000/ within your browser.

Note the --noreload option allows Eclipse to maintain control over the process, rather than spawning a new thread. This function usually allows the server to reload automatically when changes are made to your code, at your convenience.

Here’s to Taking It Easy: Simple Python environments and package management

I mentioned in my previous post that I borked my system meddling with Python. Having reset my workspace, I’ve now set up a solid system that makes handling projects and multiple development environments super simple.

The new set up easily handles multiple Python projects, without compatibility or version conflicts. The installation is equally straightforward.

Before switching to a desktop Linux, I used to sing the praises of VMware and developing with virtual machines when dealing with unique environments. By “unique”, I rather mean any odd project out of the ordinary LAMP set-up I usually work with, or something that requires a specific version of a piece of software.

Since then however, I’ve found no need. So long as you think before you leap.

Virtual boxes (as closed, single-piece software) are good and all, you can be as venturous as you wish without risk of damaging your native system. Plus, if you screw one of these you can restore a saved state in a few clicks. However, the VM safety net allows you to proceed without caution, perhaps recklessly, at the expense of fully comprehending the commands you’re executing and tasks you’re running.

In that sense, they’re great for beginners uncertain of how (or if) they should install software, e.g. Apache, PHP, Python etc — appliances and virtual stacks are helpful.

Otherwise they can convolute your workspace — and more often than not, won’t be configured exactly how you want or need them. Running software natively is simple and as intended, it also allows you to configure your entire environment without any assumptions made by distributors.

Virtualenv

Virtualenv is quite the revelation. It facilitates multiple isolated Python environments on a single system, dynamically handling your Python Path so packages are install within an enclosed local directory, rather than in amongst your top-level system packages.

This means you can create project-by-project virtual environments avoiding compatibility and version conflicts. When an environment is created (and activated) libraries are thereafter installed within discreet directories that aren’t shared with other virtualenv environments.

This means nothing is installed “system-wide”, so libraries don’t accrue over time, there’s no balancing of versions. It also means you can work with different version of Python simultaneously.

Python packages should be installed with a package manager. The latest of which is pip.

Prior to this, easy_install was the manager du jour (part of Setuptools, both now out-dated), but we’ll only be using that to install pip:

$ sudo easy_install pip

Pip is a direct replacement for easy_install, improving on a few things (a comparison can be found on the installer site). Packages that are available with easy_install should be pip-installable and the installation method is the same — the following installs virtualenv:

$ sudo pip install virtualenv

With virtualenv installed we can create an environment within your workspace, all it needs is the environment directory name, here ‘env’:

$ virtualenv env

There are a few options you have with this command. In the following example, the --no-site-packages flag means that the new environment will not inherit any system-wide global site packages. The --distribute flag will install Distribute rather than Setuptools:

$ virtualenv --no-site-packages --distribute env

Distribute is to setuptools as Pip is to easy_install. Distribute and pip are the new hotness, Setuptools and easy_install are old and busted — for now.

Anyway, activate your environment:

$ source env/bin/activate

You’ll see from your shell prompt that the environment is activated, with the name prepended.

Then we’ll install something with pip. Yolk is a tool for querying the packages currently installed on your system, so we’ll install that and grab a list:

(env) $ pip install yolk
(env) $ yolk -l

Then you’ll see everything the environment can see in the output (this will depend on your global site packages and how you created the environment, as above).

Note that you don’t need to sudo whilst in the activated environment.

As a test, we’ll deactivate the environment and run the same command, which gets the following error (unless you have yolk installed globally):

(env) $ deactivate
$ yolk -l
yolk: command not found

If installed within an environment, a package is only available whilst it is activated. This is the means to install whatever you wish, without worrying about cross-project conflicts.

More pip

Another good feature of pip is to generate a list of requirements for your working set of packages. The command is called freeze and generates a text file as follows:

(env) $ pip freeze > requirements.txt

This will create a list of all installed packages with specific versions for each library. This is in a custom syntax and looks something like this:

distribute==0.6.19
wsgiref==0.1.2
yolk==0.4.1

This list can then be distributed (e.g. to a team of developers) and used to install those packages on other systems, like so:

(env) $ pip install -r requirements.txt

Note, this isn’t couple with virtualenv, which actually has it’s own method of bootstrapping — see “Creating Your Own Bootstrap Scripts”.

Home Security: Apache VirtualHosts within an encrypted Ubuntu home directory

Since deciding to work exclusively in a Linux environment at the beginning of the year, I’ve been more than pleasantly surprised not to have found myself needing to reset my system as a result of the frequent changes of set-up and numerous installations and removals of software that I’ve needed to perform in order to work on various projects.

The inevitable day, however, came a couple of weeks ago when I royally screwed my system messing around with Python (solution in another blog post. Update: here it is).

Once Ubuntu was reinstalled, I encountered a problem attempting to recreate my workspace having opted to encrypt my home directory during user setup.

Running the normal LAMP-server setup, Apache is unable to access files within the encrypted home.

I was tying to duplicate my previous configuration, using individual VirtualHosts locating directories within my user home, for example:

/home/marc/sites/dev/

I’m pretty sure my home directory was encrypted last time too, but this problem was new for me — perhaps something from an update in between?

The permissions problem occurs as only my user, marc, has access to the home and Apache’s user, www-data, does not. This results in a HTTP 403 Forbidden when attempting to serve files.

Having a look around, I found a convoluted method using symlinks and Apache’s UserDir then a far simpler solution, on AskUbuntu, as follows.

It’s unsafe to change your home ownership (to www-data, for example) but Apache needs execute permissions there. So selectively chmod the directory:

sudo chmod 751 /home

This grants execute access to others, who can only read files with correct knowledge of names and locations. It also removes your user’s read access to /home, so you’ll have to sudo for that.

Another precaution benefiting those on development-only machines, is to restrict IP listening within Apache’s ports.conf, so only local connections get any attention:

Listen 127.0.0.1:80

Alternatives

As for alternatives, you could encrypt your whole drive rather than just the home directory. You shouldn’t see any problems then.

Or you could just ignore encryption all together.

You could, of course, just work out of the traditional /var/www/ location, which is the Apache default. Simply create a directory there and chown to your user so you don’t have to always sudo changes.

sudo mkdir /var/www/dev/
sudo chown marc /var/www/dev/

If you’re directories are elsewhere on your system, for example in SVN repositories such as /srv/svn/ or /usr/local/svn/ then you’ll need to chown those to www-data so they’re readable, similar to our method of reading from within /home above.

The Ubuntu docs on Subversion offer the best solution for handling user permissions for SVN over HTTP.

Create a new user group, subversion, add the users marc and www-data to it and chown the repo to www-data:subversion, giving read/write access to the group (granting privileges to marc). Finally chmod with -s so that new files inherit that group ID, like so:

cd /srv/svn/
sudo chown -R www-data:subversion dev/
sudo chmod -R g+rws dev/

The -s flag means that all files created inside that directory will inherit the group of the directory, otherwise files takes on the primary group of the user. New subdirectories will also inherit this.

The -R option applies the changes recursively (i.e. existing subdirectories).

Better Half: Dual-booting Ubuntu and Windows

With a new venture, comes a new set-up. I’m going to be working with a new start-up company, a small outfit that will require some working from home, some working on site, plenty of time spent in odd Wi-Fi spots around London presumably — all amounting to a good enough excuse to purchase a brand new laptop and establish a new development environment.

For some time I’ve been keen to try working with Linux full-time, particularly Ubuntu, the most popular flavour of Linux. So with a brand new system I have an opportunity to try it.

I usually work with Unix-like systems for Web development, but on the server-side and using a virtualisation such as VMware, but infrequently at author time or on the desktop. I figured this is going to be a development machine, prone to be tarnished by experiment (or more likely ignorance), perhaps in need of frequent reset, so why not give it a roll.

Rather than go solo with only an installation of Ubuntu, I decided to dual-boot with Windows 7. I already own a genuine copy of Windows 7 and a few other PC-based applications, particularly Creative Suite for Web, plus this meant I could save some cash on buying a laptop without a pre-installed operating system. The dual-boot also gifts me with a fallback option if I really can’t get on with Linux.

So far as installation goes, there’s really not much to report. Installing Windows came first, a set-up as normal, straightforward but tiresome probably hitting double figures in the number of reboots required to get going. Around seventy updates recommended therein.

Ubuntu was downloaded and burned as a Live CD, running a super-simple installer with a couldn’t-be-easier draggable horizontal slider to determine the partition size.

There are many ways to partition drives, recommended schemes such as those on the Ubuntu docs, but I figured a halfway split would be fine. I can always readdress the partitioning if Linux doesn’t work out or if I end up never touching Windows and need the extra space.

After a few days to playing with Ubuntu, so far so good. Needless to say, now at version 10.10 it’s been a fully realised operating system for some time.

The two flavours, for desktop and netbook, are both fully featured, true alternatives to Windows and Mac OS. No longer only options for developers and the computer-savvy.

They both come with a number of applications pre-installed — browsers, mail and chat clients, image editors, a range of software you’d expect. Anything else you need can be found in the Ubuntu Software Centre, an App Store-like directory of open-source and proprietary software from Ubuntu and trusted third-parties and partners.

Firefox and Thunderbird are amongst the familiar names that come packaged with the basic install and a huge number of common day-to-day applications are widely supported — Chrome, Opera, the Flash player, Skype, all AIR-based apps (such as TweetDeck).

There’s support for iPhone and Android devices and cameras. OpenOffice provides your alternative to Word, Excel and PowerPoint.

I’m currently getting to grips with Evolution Mail (email apps are always fun) and Empathy IM, a chat client that integrates Google Chat, MSN, Jabber — all the usual suspects — akin to Pidgin or Adium.

And the icing on the cake — Spotify have a Linux preview. With that installed, it feels like home. Now I just need to get used to a whole new set of shortcuts and hotkeys.

The Ubuntu docs have a lengthy article about the various ways of dual booting with Windows, on the whole the Community Docs look like a very helpful resource.

25 Minutes to Go: Investigating the Pomodoro Technique

The Pomodoro Technique is a time management method developed in the late 1980s by a man named Francesco Cirillo.

The method is simple, you take a 25-minute timer (or anything capable of timing 25 minutes) and work uninterrupted until that time is up. Then you take a 5 minute break, then you repeat. At the end of every fourth working interval, take a longer break – say 15-20 minutes. Sounds easy.

The technique has gained a kind of quasi-cult status online, heralded as one of the most effective modern lifehacks – productivity tricks devised to cut through information overload, frequently adopted by programmers.

So what’s the big deal? Skeptically, you may ask – why do you need to employ this or any technique? Why can’t you just work normally? Particularly of one so straightforward.

Whilst the method is simple, such tricks can be hugely effective. The very process of planning, recording and reflecting on progress and work completed creates a sense of motivation and adds to your feelings of accomplishment. The ‘Pomodoro Timer’, a tomato-shaped kitchen timer (pomodoro is the Italian word for tomato) produces an audible ticking during the working interval, triggering some subconscious drive to get things done.

Cirillo’s view adds that the physical act of winding up the timer confirms the user’s determination to start the task, the kick we need to get going. To overcome the all-too-familiar deadline anxiety.

So over the past month I’ve been working with the Pomodoro Technique. Adopted at the beginning of a new one-month contact, in a new office at a new desk, a clean opportunity to try it out.

Firstly I should say I am usually somewhat skeptical to any kind of technique promising some revolutionary effect, on any level be it relating to money, work or health. I guess I already consider to days to be pretty productive, thanks. I probably only considered the Pomodoro Technique because of the amount of positive testimonials I’ve heard, from a wealth of online sources and more recently from a few people whose opinions I respect.

But I don’t particularly suffer from deadline anxiety, motivation isn’t a problem for me when I’m given a heap of work – in fact I’m the opposite, irritably frustrated when I have nothing to do.

I hoped this would improve my time management, rather than boost my productivity. Too often I’m distracted by checking emails, chat clients or Twitter. By “distracted”, I don’t particularly mean going off-task and browsing intermittently, so much as perhaps in a lull of attention, wonder whether an email response has arrived or message been returned, so I’ll switch windows hit refresh.

For me the technique structured my breaks. It offered the reassurance that I will have the chance to check for that email or tweet in no longer than 25 minutes time. It meant that there was no possibility that I could get caught up in a task that might stretch an hour and therefore cause me to fall behind in anything else.

Ultimately, that I could devote my full attention to the current task and I’ll be reminded of when it’s time for a break, when my timer tells me so.

An essential aim of the technique is to cut down on interruptions, both internal and external. If asked “Do you have time for a quick chat?”, I could confidently say “Yes, in 5 minutes”, or however much time I had left on the clock. I knew that if that ‘quick chat’ became a 10 minute meeting, my work wouldn’t be any more interrupted.

The timer offers an impartial and impersonal structure, an obedience to which relieves the distraction of continual desktop beeps and alerts whilst sharpening your focus on the task in hand.

There are a number of free unofficial Pomodoro applications for desktop and mobile platforms to save you ordering a tomato timer or testing your colleagues patience with a bell ringing every half hour.

I used the Pomodoro Lite iPhone app, which is very straightforward (it’s just a preset interval timer after all), though eats up the battery for use of a whole day.

Focus Booster and Pomodairo are cross-platform AIR-based apps. Both look pretty slick and have additional features such as task lists.

ChromoDoro is an extension for Google Chrome, adding a tomato timer to your toolbar with pop-ups to alert you when you’re done.

Whether I’ll continue to use the technique, I’m not sure. I think as a method of concentrating your focus, or retraining your attention span even, it’s very effective. I didn’t find any revelation, but it didn’t disrupt my normal working day either.

Getting into the hang of it, completing tasks within a 25 minute cycles does result a great sense of accomplishment, feeding a higher level of motivation. It might also surprise you how much you can achieve in 25 uninterrupted minutes. I found myself attempting tasks I would usually estimate taking longer. Perhaps that’s the competitive part of me, but it felt great beating the clock.

The most positive outcome as I say, put a stop to my tendency to allow distractions to creep into working time. Though it would be nice to think that I could achieve that with just a bit more self-discipline instead.

The wise men were all fools, what to do?