Welcome to OpenREM’s documentation!¶

Install Python 2.7.x¶

• Linux – likely to be installed already
• Windows – https://www.python.org/downloads

;C:\Python27\;C:\Python27\Scripts\

Setuptools and pip¶

Install setuptools and pip – for details go to http://www.pip-installer.org/en/latest/installing.html. The quick version is as follows:

Linux

Download the latest version using the same method as for Windows, or get the version in your package manager, for example:

sudo apt-get install python-pip

Windows

Download the installer script get-pip.py and save it locally – right click and Save link as... or equivalent.

Open a command window (Start menu, cmd.exe) and navigate to the place you saved the get‑pip.py file:

python get-pip.py

pip -V

Install RabbitMQ¶

• Linux - Follow the guide at http://www.rabbitmq.com/install-debian.html
• Windows - Follow the guide at http://www.rabbitmq.com/install-windows.html

For either install, just follow the defaults – no special configurations required.

Install NumPy¶

(New for version 0.6.0)

Numpy is required for charts. OpenREM will work without NumPy, but charts will not be displayed.

For linux:

sudo apt-get install python-numpy
# If using a virtualenv, you might need to also do:
pip install numpy

For Windows, there are various options:

1. Download executable install file from SourceForge:

   • Download a pre-compiled Win32 .exe NumPy file from http://sourceforge.net/projects/numpy/files/NumPy/. You need to download the file that matches the Python version, which should be 2.7. At the time of writing the latest version was 1.9.2, and the filename to download was numpy-1.9.2-win32-superpack-python2.7.exe. The filename is truncated on SourceForge, so you may need to click on the i icon to see which is which. It’s usually the third superpack.
   • Run the downloaded binary file to install NumPy.

2. Or download a pip installable wheel file:

   • Download NumPy from http://www.lfd.uci.edu/~gohlke/pythonlibs/#numpy - numpy‑1.9.2+mkl‑cp27‑none‑win32.whl is likely to be the right version, unless you have 64bit Python installed, in which case use the numpy‑1.9.2+mkl‑cp27‑none‑win_amd64.whl version instead.

   • Install using pip:

     pip install numpy‑1.9.2+mkl‑cp27‑none‑win32.whl
     

Install pynetdicom¶

(New for version 0.6.0)

Pynetdicom is used for the new DICOM store SCP function that is available as a preview in this release. See DICOM Networking in OpenREM for details.

pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

Configure¶

Locate install location

• Linux: /usr/local/lib/python2.7/dist-packages/openrem/ or /usr/lib/python2.7/site-packages/openrem/
• Windows: C:\Python27\Lib\site-packages\openrem\

There are two files that need renaming:

• openremproject/local_settings.py.example to openremproject/local_settings.py
• openremproject/wsgi.py.example to openremproject/wsgi.py

In the local_settings.py file, set the database details, the MEDIA_ROOT path, the secret key and the ALLOWED_HOSTS.

'ENGINE': 'django.db.backends.sqlite3',
'NAME': '/ENTER/PATH/WHERE/DB/FILE/CAN/GO',

MEDIA_ROOT = "/var/openrem/media/"

MEDIA_ROOT = "C:/Users/myusername/Documents/OpenREM/media/"

ALLOWED_HOSTS = [
    '192.168.56.102',
    '.doseserver.',
    'localhost',
]

Create the database¶

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py syncdb

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py syncdb

Answer each question as it is asked, do setup a superuser. This username and password wil be used to log into the admin interface to create the usernames for using the web interface. See the Start using it! section below.

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp

python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp

Start test web server¶

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py runserver --insecure

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py runserver --insecure

If you are using a headless server and need to be able to see the web interface from another machine, use python /usr/lib/python2.7/dist-packages/openrem/manage.py runserver x.x.x.x:8000 --insecure (or Windows equivalent) replacing the x with the IP address of the server and 8000 with the port you wish to use.

Open the web addesss given, appending /openrem (http://localhost:8000/openrem)

Start the Celery task queue¶

Celery will have been automatically installed with OpenREM, and along with RabbitMQ allows for asynchronous task processing for imports and exports.

In a new shell:

Linux:

cd /usr/local/lib/python2.7/dist-packages/openrem/
celery -A openremproject worker -l info

Windows:

cd C:\Python27\Lib\site-packages\openrem\
celery -A openremproject worker -l info

For production use, see Daemonising Celery below

Preview feature: Start the DICOM Store SCP¶

See DICOM Networking in OpenREM

Start using it!¶

Add some data!

openrem_rdsr.py rdsrfile.dcm

Add some users (New in version 0.4.0)

• Go to the admin interface (eg http://localhost:8000/admin) and log in with the user created when you created the database (syncdb)

• Create some users and add them to the appropriate groups (if there are no groups, go to the OpenREM homepage and they should be created).

  • viewgroup can browse the data only
  • exportgroup can do as view group plus export data to a spreadsheet
  • admingroup can delete studies and import height and weight data in addition to anything the export group can do

• Return to the OpenREM interface (eg http://localhost:8000/openrem) and log out of the superuser in the top right corner and log in again using one of the new users you have just created.

Database options¶

SQLite is great for getting things running quickly and testing if the setup works, but is really not recommended for production use on any scale. Therefore it is recommended to use a different database such as PostgreSQL or MySQL.

Here are instructions for installing PostgreSQL on linux and on Windows:

Database migrations¶

South is a django application to manage database migrations. Using South means that future changes to the database model can be calculated and executed automatically with simple commands when OpenREM is upgraded.

Production webservers¶

Unlike the database, the production webserver can be left till later and can be changed again at any time.

For performance it is recommended that a production webserver is used instead of the inbuilt ‘runserver’. Popular choices would be either Apache or you can do as the cool kids do and use Gunicorn with nginx.

The django website has instructions and links to get you set up with Apache.

Daemonising Celery¶

In a production environment, Celery will need to start automatically and not depend on a particular user being logged in. Therefore, much like the webserver, it will need to be daemonised. For now, please refer to the instructions and links at http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html.

Virtualenv and virtualenvwrapper¶

If the server is to be used for more than one python application, or you wish to be able to test different versions of OpenREM or do any development, it is highly recommended that you use virtualenv or maybe virtualenvwrapper

Virtualenv sets up an isolated python environment and is relatively easy to use.

If you do use virtualenv, all the paths referred to in the documentation will be changed to:

• Linux: lib/python2.7/site-packages/openrem/
• Windows: Lib\site-packages\openrem

In Windows, even when the virtualenv is activated you will need to call python and provide the full path to script in the Scripts folder. If you call the script (such as openrem_rdsr.py) without prefixing it with python, the system wide Python will be used instead. This doesn’t apply to Linux, where once activated, the scripts can be called without a python prefix from anywhere.

Advanced guides for developers¶

Installing Apache on Windows Server 2012 with auto-restart¶

Note

Author JA Cole

These instructions are for installing OpenREM under Apache on Windows as a developers alternative to the built-in HTTP server. They have been written using Windows Server 2012, and feature automatic restarts of the Apache server when the code changes, much as the built-in server does.

Get and Install Apache¶

• Download the zip of the appropriate version from https://www.apachelounge.com/
• Extract the zip somewhere useful. For this guide we will assume C:\apache24\

Get and Install MOD_WSGI¶

• Download mod_wsgi that matches your Windows, Apache and Python versions from http://www.lfd.uci.edu/~gohlke/pythonlibs/#mod_wsgi

• Extract the mod_wsgi.so file to C:\apache24\modules\

• Add the following module C:\apache24\conf\httpd.conf:

  LoadModule wsgi_module modules/mod_wsgi.so
  

• At the end of C:\apache24\conf\httpd.conf add the following:

  WSGIScriptAlias / "c:/Python27/Lib/site-packages/openrem/openremproject/wsgi.py"
  WSGIPythonPath "c:/Python27/Lib/site-packages/openrem"
  
  <Directory "c:/Python27/Lib/site-packages/openrem/openremproject">
  <Files wsgi.py>
  Order deny,allow
  Require all granted
  </Files>
  </Directory>
  

Get and Install wsgi.py and monitor.py¶

Detailed instructions are available here: https://code.google.com/p/modwsgi/wiki/ReloadingSourceCode

• Change wsgi.py in the openrem/openremproject folder to the following

"""
WSGI config for OpenREM project.

This module contains the WSGI application used by Django's development server
and any production WSGI deployments. It should expose a module-level variable
named ``application``. Django's ``runserver`` and ``runfcgi`` commands discover
this application via the ``WSGI_APPLICATION`` setting.

Usually you will have the standard Django WSGI application here, but it also
might make sense to replace the whole Django WSGI application with a custom one
that later delegates to the Django one. For example, you could introduce WSGI
middleware here, or combine a Django application with an application of another
framework.

"""
import os
import sys

path = 'C:/Python27/Lib/site-packages/openrem'
if path not in sys.path:
sys.path.append(path)

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openremproject.settings")

# Apply WSGI middleware here.

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

import openremproject.monitor
openremproject.monitor.start(interval=1.0)

• Create a file monitor.py in the openrem/openremproject folder with the following contents

# Code from the modwsgi wiki at https://code.google.com/p/modwsgi/wiki/ReloadingSourceCode
# Copyright 2007-2011 GRAHAM DUMPLETON
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#     http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#


import os
import sys
import time
import signal
import threading
import atexit
import Queue

_interval = 1.0
_times = {}
_files = []

_running = False
_queue = Queue.Queue()
_lock = threading.Lock()

def _restart(path):
    _queue.put(True)
    prefix = 'monitor (pid=%d):' % os.getpid()
    print >> sys.stderr, '%s Change detected to \'%s\'.' % (prefix, path)
    print >> sys.stderr, '%s Triggering Apache restart.' % prefix
    import ctypes
    ctypes.windll.libhttpd.ap_signal_parent(1)

def _modified(path):
    try:
        # If path doesn't denote a file and were previously
        # tracking it, then it has been removed or the file type
        # has changed so force a restart. If not previously
        # tracking the file then we can ignore it as probably
        # pseudo reference such as when file extracted from a
        # collection of modules contained in a zip file.

        if not os.path.isfile(path):
            return path in _times

        # Check for when file last modified.

        mtime = os.stat(path).st_mtime
        if path not in _times:
            _times[path] = mtime

        # Force restart when modification time has changed, even
        # if time now older, as that could indicate older file
        # has been restored.

        if mtime != _times[path]:
            return True
    except:
        # If any exception occured, likely that file has been
        # been removed just before stat(), so force a restart.

        return True

    return False

def _monitor():
    while 1:
        # Check modification times on all files in sys.modules.

        for module in sys.modules.values():
            if not hasattr(module, '__file__'):
                continue
            path = getattr(module, '__file__')
            if not path:
                continue
            if os.path.splitext(path)[1] in ['.pyc', '.pyo', '.pyd']:
                path = path[:-1]
            if _modified(path):
                return _restart(path)

        # Check modification times on files which have
        # specifically been registered for monitoring.

        for path in _files:
            if _modified(path):
                return _restart(path)

        # Go to sleep for specified interval.

        try:
            return _queue.get(timeout=_interval)
        except:
            pass

_thread = threading.Thread(target=_monitor)
_thread.setDaemon(True)

def _exiting():
    try:
        _queue.put(True)
    except:
        pass
    _thread.join()

atexit.register(_exiting)

def track(path):
    if not path in _files:
        _files.append(path)

def start(interval=1.0):
    global _interval
    if interval < _interval:
        _interval = interval

    global _running
    _lock.acquire()
    if not _running:
        prefix = 'monitor (pid=%d):' % os.getpid()
        print >> sys.stderr, '%s Starting change monitor.' % prefix
        _running = True
        _thread.start()
_lock.release()

Install Micosoft C++ Distributable¶

Install the microsoft C++ distributable making sure the version number matches the version number for the apache and mod_wsgi downloads. http://www.microsoft.com/en-us/download/details.aspx?id=30679#

Optional: Install apache as a service¶

Run a terminal as administrator.:

c:\apache24\bin\httpd -k install

Setup the URLs¶

Add the following to the openrem urls.py file:

from django.conf import settings
if settings.DEBUG:
    urlpatterns += patterns('django.contrib.staticfiles.views',
        url(r'^static/(?P<path>.*)$', 'serve'),
    )

Collect the static files¶

Collect your static files by running:

python manage.py collectstatic

If this fails because openrem lacks a static folder either copy the static folder from remapp to the openrem directory, adjust the openrem settings or set up a link. To setup a link run:

mklink /D c:\python27\lib\site-packages\openrem\static c:\python27\lib\site-packages\openrem\remapp\static

Changes for 0.6.2¶

Minor update due prevent new installs from installing a non-compatible version of django-filter. The link to OpenSkin has also been updated in the fluoroscopy detail page.

There is no advantage to updating to this version over 0.6.0

Release 0.6.1 was just a documentation only change to update the link to OpenSkin.

Convert to South¶

Make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py convert_to_south remapp

# Windows, assuming no virtualenv
python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp

Additional installs¶

OpenREM requires two additional programs to be installed to enable the new features: Numpy for charts, and pynetdicom for the DICOM Store Service Class Provider. Note that the version of pynetdicom must be later than the current pypi release!

sudo apt-get install python-numpy
# If using a virtualenv, you might need to also do:
pip install numpy

pip install https://bitbucket.org/edmcdonagh/pynetdicom/get/default.tar.gz#egg=pynetdicom-0.8.2b2

Upgrading from versions prior to 0.5.1¶

You must upgrade to 0.5.1 first. Instructions for doing this can be found in the OpenREM Release Notes version 0.5.1.

Upgrading from version 0.5.1¶

• Back up your database

  • For PostgreSQL you can refer to Backing up a PostgreSQL database
  • For a non-production SQLite3 database, simply make a copy of the database file

• The 0.6.0 upgrade must be made from a 0.5.1 (or later) database, and a schema migration is required:

pip install openrem==0.6.0

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
# Windows:
python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp

• Restart the services

  • Restart the webserver
  • Restart Celery

Charts¶

Release 0.6.0 has a range of charting options available for CT and radiographic data. These charts allow visualisation of trends and frequencies to inform surveys and monitor performance. For more information, please see Charts.

DICOM Store Service Class Provider¶

OpenREM can now act as the DICOM Store service, allowing direct sending of DICOM objects from modalities to OpenREM without needing to use Conquest or any other DICOM Store SCP. This feature is a preview as it hasn’t been extensively tested, but it is expected to work. For more information, please see DICOM Networking in OpenREM.

Exports for OpenSkin¶

Fluoroscopy studies can now be exported in a format suitable for importing into Jonathan Cole’s OpenSkin software. The export link is on the fluoroscopy study detail page. The software for creating the exposure incidence map can be downloaded from https://bitbucket.org/jacole/openskin/downloads (choose the zip file), and information about the project can be found on the OpenSkin wiki. The software allows the user to choose between a 2D phantom that would represent the dose to a film laying on the couch surface, or a simple 3D phantom made up of a cuboid and two semi-cylinders (these can be seen on the Phantom design section of the wiki). For both options the output is an image of the dose distribution in 2D, along with calculated peak skin dose information.

Automatic hiding of unused modality types¶

A fresh install of OpenREM will no longer show any of the four modality types in the tables or in the navigation bar at the top. As DICOM objects are ingested, the appropriate tables and navigation links are created.

Therefore a site that has no mammography for example will no longer have that table or navigation link in their interface.

Mammography import compression force change¶

Prior to version 0.6, the compression force extracted from the mammography image header was divided by ten before being stored in the database. This was because the primary author only had access to GE Senograph DS units, which store the compression force in dN, despite claiming using Newtons in the DICOM conformance statement.

The code now checks for the term senograph ds contained in the model name. If it matches, then the value is divided by ten. Otherwise, the value is stored without any further change. We know that later GE units, the GE Senograph Essential for example, and other manufacturer’s units store this value in N. If you have a case that acts like the Senograph DS, please let us know and we’ll try and cater for that.

If you have existing non-GE Senograph mammography data in your database, the compression force field for those studies is likely to be incorrect by a factor of ten (it will be too small). Studies imported after the upgrade will be correct. If this is a problem for you, please let us know and we’ll see about writing a script to correct the existing data.

Import of Toshiba Planar RDSRs fixed¶

Toshiba include Patient Orientation and Patient Orientation Modifier information in their cath lab RDSRs. The extractor code was deficient for this as the RDSRs previously used didn’t have this information. This has now been fixed. There might however be an issue with Station Name not being provided - it is not yet clear if this is a configuration issue.

Version specific information¶

OpenREM Release Notes version 0.5.1¶

Headline changes¶

• Major database modification to remove table name length errors
• Extended the field value lengths to better incorporate all possible values and decimal places
• Improved import of grid and filter information from DX images
• Improved DX summary and detail web pages
• Any item in a row can now be clicked to move between the home and filtered pages

Upgrades: Convert to South¶

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already

# Linux: Debian/Ubuntu and derivatives
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
python /usr/local/lib/python2.7/site-packages/openrem/manage.py convert_to_south remapp

# Windows, assuming no virtualenv
python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp

Upgrading from before 0.5.0¶

Upgrading from version 0.3.9 or earlier¶

• Back up your database

  • For PostgreSQL you can refer to Backing up a PostgreSQL database
  • For a non-production SQLite3 database, simply make a copy of the database file

• pip install openrem==0.4.2

• Migrate the schema

  # Linux: Debian/Ubuntu and derivatives
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
  # Windows:
  python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
  

  When South has considered the changes to the schema, you will see the following message:

  ? The field 'Observer_context.device_observer_name' does not have a default specified, yet is NOT NULL.
  ? Since you are making this field nullable, you MUST specify a default
  ? value to use for existing rows. Would you like to:
  ?  1. Quit now.
  ?  2. Specify a one-off value to use for existing columns now
  ?  3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
  ? Please select a choice: 3
  

  • As per the final line above, please select option 3, and then execute the migration:

  # Linux: Debian/Ubuntu and derivatives
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
  
  # Windows, assuming no virtualenv
  python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  

• Create and populate the database settings in the new local_settings.py file

  The openrem/openrem folder can be found at:

  # Linux: Debian/Ubuntu and derivatives
  /usr/lib/python2.7/dist-packages/openrem/openrem
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  /usr/lib/python2.7/site-packages/openrem/openrem
  # Windows:
  C:\Python27\Lib\site-packages\openrem\openrem
  

  In the openrem/openrem folder, create a new file called local_settings.py and copy the contents of this link into a the file and save it. Alternatively, rename local_settings.py.example to local_settings.py - this is an older version of the file.

  Copy the database details from settings.py into local_settings.py

• Change the secret key - you can use http://www.miniwebtool.com/django-secret-key-generator/ to generate a new one

• Move the existing settings.py out of the python directories (delete or move somewhere as a backup)

• Rename the settings.py.new to settings.py

• Restart your webserver to check everything looks ok

• Add some users

  • Go to the admin interface (eg http://localhost:8000/admin) and log in with the user created when you originally created the database (the manage.py syncdb command - Do you want to create a superuser)

  • Create some users and add them to the appropriate groups (if there are no groups, go to the OpenREM homepage and they should be there when you go back to admin).

    • viewgroup can browse the data only
    • exportgroup can do as view group plus export data to a spreadsheet, and will be able to import height and weight data in due course (See Issue #21)
    • admingroup can delete studies in addition to anything the export group can do

Upgrading from versions 0.4.0 - 0.4.2¶

• Back up your database

  • For PostgreSQL you can refer to Backing up a PostgreSQL database
  • For a non-production SQLite3 database, simply make a copy of the database file

• Install version 0.5.0

  • pip install openrem==0.5.0

• Install RabbitMQ

  • Linux - Follow the guide at http://www.rabbitmq.com/install-debian.html
  • Windows - Follow the guide at http://www.rabbitmq.com/install-windows.html

• Move local_settings.py details from openrem to openremproject

  The inner openrem Django project folder has now been renamed openremproject The customised local_settings.py file and the wsgi.py file have remain in the old openrem folder. The openrem/openrem folder can be found as detailed in the upgrade from ‘0.3.9 or earlier’ instructions above, and the new openrem/openremproject folder is in the same place.

  • Move local_settings.py to openremproject. If you have kept the older local_settings file, you may like to instead rename the local_settings.py.example file instead, then transfer the database settings and change the secret key.
  • Set the path for MEDIA_ROOT. The webserver needs to be able to write to this location - it is where OpenREM will store export files etc so that they can be downloaded. For suggestions, see the main _install instructions.
  • Set ALLOWED_HOSTS. For details see the Django docs A '*' allows any host - see the Django docs for the risk of this.

• Move wsgi.py from openrem to openremproject or rename wsgi.py.example in openremproject

  If you haven’t edited it, simply rename the new version in openremproject. Otherwise, move the old version and edit the following line as follows:

  # Old:
  os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openrem.settings")
  # New:
  os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openremproject.settings")
  

• Tidying up - you should delete the old openrem folder - you might like to take a backup first!

• Update web server configuration

  The configuration of the webserver will need to be updated to reflect the new location for the settings.py file and the wsgi.py file.

  If you are using the built-in test webserver, static files will no-longer be served unless you use the insecure option:

  python manage.py runserver x.x.x.x:8000 --insecure
  

• Migrate the schema

  # Linux: Debian/Ubuntu and derivatives
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
  # Windows:
  python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
  python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  

After restarting the webserver, you should now have OpenREM 0.5.0 up and running. If you wish to test export functionality at this stage, start the Celery task queue - instructions in the Installation instructions docs or at the end of this guide.

Now move to Upgrading from version 0.5.0.

Upgrading from version 0.4.3¶

• Back up your database

  • For PostgreSQL you can refer to Backing up a PostgreSQL database
  • For a non-production SQLite3 database, simply make a copy of the database file

• The 0.5.1 upgrade must be made from a 0.5.0 database, so a schema migration is required:

  pip install openrem==0.5.0
  
      # Linux: Debian/Ubuntu and derivatives
      python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
      python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
      # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
      python /usr/local/lib/python2.7/site-packages/openrem/manage.py schemamigration --auto remapp
      python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
      # Windows:
      python C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
      python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  

Upgrading from version 0.5.0¶

• Back up your database

  • For PostgreSQL you can refer to Backing up a PostgreSQL database
  • For a non-production SQLite3 database, simply make a copy of the database file

• Install 0.5.1:

  pip install openrem==0.5.1
  

• Find out how many migration files you have

  Method 1:

  Use a file browser or terminal to list the contents of the migrations folder, eg:

  # Linux: Debian/Ubuntu and derivatives
  ls /usr/local/lib/python2.7/dist-packages/openrem/remapp/migrations/
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  ls /usr/local/lib/python2.7/site-packages/openrem/remapp/migrations/
  # Windows (alternatively use the file browser):
  dir C:\Python27\Lib\site-packages\openrem\remapp\migrations\
  

  Method 2:

  Use the Django manage.py program to list the existing migrations:

  # Linux: Debian/Ubuntu and derivatives
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate --list remapp
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate --list remapp
  # Windows
  python C:\Python27\Lib\site-packages\openrem\manage.py migrate --list remapp
  

  The output should look something like this - there can be any number of existing migrations (though 0001_initial will always be present):

  remapp
  (*) 0001_initial
  (*) 0002_auto__chg_field_ct_accumulated_dose_data_ct_dose_length_product_total_
  (*) 0003_auto__chg_field_general_equipment_module_attributes_station_name
  (*) 0004_auto__chg_field_ct_radiation_dose_comment__chg_field_accumulated_proje
  (*) 0005_auto__add_exports__add_size_upload
  (*) 0006_auto__chg_field_exports_filename
  (*) 0007_auto__add_field_irradiation_event_xray_detector_data_relative_xray_exp
  ( ) 000x_051datamigration
  ( ) 000x_051schemamigration
  

• Rename the two 051 migration files to follow on from the existing migrations, for example 0008_051schemamigration.py and 0009_051datamigration.py for the existing migrations above, or 0002_051schemamigration.py and 0003_051datamigration.py if the only migration is the initial migration. The 051schemamigration must come before the 051datamigration.

  If you are using linux, you might like to do it like this (from within the openrem folder):

  mv remapp/migrations/000{x,8}_051schemamigration.py
  mv remapp/migrations/000{x,9}_051datamigration.py
  

• If you now re-run migrate --list remapp you should get a listing with the 051schemamigration and the 051datamigration listed at the end:

  remapp
   (*) 0001_initial
   (*) 0002_auto__chg_field_ct_accumulated_dose_data_ct_dose_length_product_total_
   (*) 0003_auto__chg_field_general_equipment_module_attributes_station_name
   (*) 0004_auto__chg_field_ct_radiation_dose_comment__chg_field_accumulated_proje
   (*) 0005_auto__add_exports__add_size_upload
   (*) 0006_auto__chg_field_exports_filename
   (*) 0007_auto__add_field_irradiation_event_xray_detector_data_relative_xray_exp
   ( ) 0008_051schemamigration
   ( ) 0009_051datamigration
  

  The star indicates that a migration has already been completed. If you have any that are not completed apart from the 051schemamigration and the 051datamigration then please resolve these first.

• Now execute the migrations:

  # Linux: Debian/Ubuntu and derivatives
  python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
  # Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
  python /usr/local/lib/python2.7/site-packages/openrem/manage.py migrate remapp
  # Windows
  python C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp
  

Restart the web server¶

If you are using the built-in test web server (not for production use):

python manage.py runserver x.x.x.x:8000 --insecure

Otherwise restart using the command for your web server

Restart the Celery task queue¶

For testing, in a new shell:

# Linux: Debian/Ubuntu and derivatives
cd /usr/local/lib/python2.7/dist-packages/openrem/
# Linux: other distros. In a virtualenv replace all up to lib/ as appropriate
cd /usr/local/lib/python2.7/site-packages/openrem/
# Windows
cd C:\Python27\Lib\site-packages\openrem\

# All
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.5.0¶

Headline changes¶

• Import, display and export of CR/DX data from image headers
• Export of study data from fluoroscopy to xlsx files
• Importing data from Windows using *.dcm style wildcards
• Hologic tomography projection images are no longer excluded if part of a Combo exposure

Specific upgrade instructions¶

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already:

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp

Upgrading from versions before 0.4.3¶

If you are upgrading from 0.3.9 or earlier, you will need to upgrade to version 0.4.2 first. See the OpenREM Release Notes version 0.4.3.

If you are upgrading from 0.4.0 or later, the instructions in OpenREM Release Notes version 0.4.3 still need to be followed to install/setup RabbitMQ and Celery and to update the configuration files, but you can go straight to 0.5.0 rather than installing 0.4.3.

Upgrading from version 0.4.3¶

pip install openrem==0.5.0

(Will need sudo or equivalent if using linux without a virtualenv)

Database migration¶

Assuming no virtualenv

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp

Windows:

C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp

Restart the web server¶

If you are using the built-in test web server (not for production use):

python manage.py runserver x.x.x.x:8000 --insecure

Otherwise restart using the command for your web server

Restart the Celery task queue¶

For testing, in a new shell: (assuming no virtualenv)

Linux:

cd /usr/local/lib/python2.7/dist-packages/openrem/
celery -A openremproject worker -l info

Windows:

cd C:\Python27\Lib\site-packages\openrem\
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.4.3¶

Headline changes¶

• Export of study information is now handled by a task queue - no more export time-outs.
• Patient size information in csv files can now be uploaded and imported via a web interface.
• Proprietary projection image object created by Hologic tomography units can now be interrogated for details of the tomosynthesis exam.
• Settings.py now ships with its proper name, this will overwrite important local settings if upgrade is from 0.3.9 or earlier.
• Time since last study is no longer wrong just because of daylight saving time!
• Django release set to 1.6; OpenREM isn’t ready for Django 1.7 yet
• The inner openrem Django project folder is now called openremproject to avoid import conflicts with Celery on Windows
• DEBUG mode now defaults to False

Specific upgrade instructions¶

Always make sure you have converted your database to South before attempting an upgrade

Quick reminder of how, if you haven’t done it already:

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py convert_to_south remapp

Windows:

python C:\Python27\Lib\site-packages\openrem\manage.py convert_to_south remapp

Upgrading from 0.3.9 or earlier¶

It is essential that you upgrade to at least 0.4.0 first, then upgrade to 0.4.3. Otherwise the settings file will be overwritten and you will lose your database settings. There is also a trickier than usual database migration and instructions for setting up users. Fresh installs should start with the latest version.

Upgrade to version 0.4.2

pip install openrem==0.4.2

(Will need sudo or equivalent if using linux without a virtualenv)

Then follow the instructions in OpenREM Release Notes version 0.4.0 from migrating the database onwards, before coming back to these instructions.

Upgrading from 0.4.0 or above¶

Install OpenREM version 0.4.3¶

pip install openrem==0.4.3

(Will need sudo or equivalent if using linux without a virtualenv)

RabbitMQ¶

The message broker RabbitMQ needs to be installed to enable the export and upload features

• Linux - Follow the guide at http://www.rabbitmq.com/install-debian.html
• Windows - Follow the guide at http://www.rabbitmq.com/install-windows.html

Move and edit local_settings.py file and wsgi.py files¶

The inner openrem Django project folder has now been renamed openremproject to avoid import confusion that prevented Celery working on Windows.

When you upgrade, the local_settings.py file and the wsgi.py file will remain in the old openrem folder. Both need to be moved across to the openremproject folder, and edited as below.

The new and old folders will be found in:

• Linux: /usr/local/lib/python2.7/dist-packages/openrem/
• Linux with virtualenv: /home/myname/openrem/lib/python2.7/site-packages/openrem/
• Windows: C:\Python27\Lib\site-packages\openrem\

Edit the local_settings.py file¶

The MEDIA_ROOT path needs to be defined. This is the place where the study exports will be stored for download and where the patient size information csv files will be stored temporarily whilst they are bing processed.

The path set for MEDIA_ROOT is up to you, but the user that runs the webserver must have read/write access to the location specified because it is the webserver than reads and writes the files. In a debian linux, this is likely to be www-data for a production install. Remember to use forward slashes in the config file, even for Windows.

Linux example:

MEDIA_ROOT = "/var/openrem/media/"

Windows example:

MEDIA_ROOT = "C:/Users/myusername/Documents/OpenREM/media/"

The ALLOWED_HOSTS needs to be defined, as the DEBUG mode is now set to False. This needs to contain the server name or IP address that will be used in the URL in the web browser. For example:

ALLOWED_HOSTS = [
    '192.168.56.102',
    '.doseserver.',
    'localhost',
]

A dot before a hostname allows for subdomains (eg www.doseserver), a dot after a hostname allows for FQDNs (eg doseserver.ad.trust.nhs.uk). Alternatively, a single '*' allows any host, but removes the security the feature gives you.

Edit the wsgi.py file with the new project folder name¶

If you aren’t using the wsgi.py file as part of your webserver setup, you might like to simply rename the wsgi.py.example file in the openremproject folder.

If you are using it, edit the line:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openrem.settings")

to read:

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "openremproject.settings")

Tidying up¶

Finally, you should delete the old openrem folder - you might like to take a backup first!

Database migration¶

Assuming no virtualenv

Linux:

python /usr/local/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
python /usr/local/lib/python2.7/dist-packages/openrem/manage.py migrate remapp

Windows:

C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp
C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp

Web server¶

If you are using a production webserver, you will probably need to edit some of the configuration to reflect the change in location of settings.py, for example DJANGO_SETTINGS_MODULE = openremproject.settings, and then restart the web server. You may need to do something similar for the location of wsgi.py.

If you are using the built-in test web server (not for production use), then the static files will not be served unless you add --insecure to the command. This is one of the consequences of setting DEBUG to False:

python manage.py runserver x.x.x.x:8000 --insecure

Start the Celery task queue¶

Note

The webserver and Celery both need to be able to read and write to the MEDIA_ROOT location. Therefore you might wish to consider starting Celery using the same user or group as the webserver, and setting the file permissions accordingly.

For testing, in a new shell: (assuming no virtualenv)

Linux:

cd /usr/local/lib/python2.7/dist-packages/openrem/
celery -A openremproject worker -l info

Windows:

cd C:\Python27\Lib\site-packages\openrem\
celery -A openremproject worker -l info

For production use, see http://celery.readthedocs.org/en/latest/tutorials/daemonizing.html

OpenREM Release Notes version 0.4.2¶

Headline changes¶

• This release fixes a major bug introduced in 0.4.0 regarding the import scripts.

Specific upgrade instructions¶

Upgrading from 0.3.9 or earlier¶

Follow the instructions in OpenREM Release Notes version 0.4.0

Upgrading from 0.4.0 or above¶

Move straight to version 0.4.3 and follow the instructions in OpenREM Release Notes version 0.4.3

OpenREM Release Notes version 0.4.1¶

Headline changes¶

• This release is exacly the same as 0.4.1 bar some documentation corrections

Specific upgrade instructions¶

Please use the 0.4.0 release notes for upgrades from 0.3.9

OpenREM Release Notes version 0.4.0

OpenREM Release Notes version 0.4.0¶

Headline changes¶

• User authentication has been added
• Studies can be deleted from the web interface
• Import scripts can now be passed a list of files, eg python openrem_rdsr.py *.dcm
• Date of birth no longer retained for mammography (bug fix - correct behaviour already existed for other imports)
• General bug fixes to enable import from wider range of sources
• Improved user documentation

Specific upgrade instructions¶

• pip install openrem==0.4.2 Go straight to 0.4.2

• Migrate the database

  Warning

  A database migration is required that will need a choice to be made

  • Linux: python /usr/lib/python2.7/dist-packages/openrem/manage.py schemamigration --auto remapp
  • Windows: C:\Python27\Lib\site-packages\openrem\manage.py schemamigration --auto remapp

  When South has considered the changes to the schema, you will see the following message:

  ? The field 'Observer_context.device_observer_name' does not have a default specified, yet is NOT NULL.
  ? Since you are making this field nullable, you MUST specify a default
  ? value to use for existing rows. Would you like to:
  ?  1. Quit now.
  ?  2. Specify a one-off value to use for existing columns now
  ?  3. Disable the backwards migration by raising an exception; you can edit the migration to fix it later
  ? Please select a choice: 3
  

  As per the final line above, the correct choice is 3. The fields that are now nullable previously weren’t. Existing data in those fields will have a value, or those tables don’t exist in the current database. The problem scenario is if after the migration these tables are used and one of the new nullable fields is left as null, you will not be able to migrate back to the old database schema without error. This is not something that you will want to do, so this is ok.

  Do the migration:

  • Linux: python /usr/lib/python2.7/dist-packages/openrem/manage.py migrate remapp
  • Windows: C:\Python27\Lib\site-packages\openrem\manage.py migrate remapp

• Update the settings files

  Warning

  The settings file has changed and will need to be manually edited.

  Changes need to be made to the settings.py file where the database details are stored. Normally upgrades don’t touch this file and the copy in the upgrade has a .example suffix. This upgrade and potentially future ones will need to change this file, so the format has been changed. The settings.py file will now be replaced each time the code is upgraded. In addition, there is a new local_settings.py file that contains things that are specific to your installation, such as the database settings.

  This upgrade will include a file called settings.py.new and the local_settings.py.example file. You will need to do the following:

  • Copy the database settings from your current settings.py file to the local_settings.py.example file and rename it to remove the .example. Both of these files are in the openrem/openrem directory, which will be somewhere like
    • Linux: /usr/lib/python2.7/dist-packages/openrem/openrem/
    • Windows: C:\Python27\Lib\site-packages\openrem\openrem\
  • Move the existing settings.py out of the python directories
  • Rename the settings.py.new to settings.py

• Create a new secret key

  All versions of openrem ship with the same secret key. This key is used for web security checks, and should be unique (and secret) for each installation.

  • Generate a new secret key - http://www.miniwebtool.com/django-secret-key-generator/ is a suitable method of creating a new key.
  • Copy the new key and use it to replace the default key in the local_settings.py file

• Restart your webserver

• Add some users

  • Go to the admin interface (eg http://localhost:8000/admin) and log in with the user created when you originally created the database (manage.py syncdb)

  • Create some users and add them to the appropriate groups (if there are no groups, go to the OpenREM homepage and they should be created).

    • viewgroup can browse the data only
    • exportgroup can do as view group plus export data to a spreadsheet, and will be able to import height and weight data in due course (See Issue #21)
    • admingroup can delete studies in addition to anything the export group can do

Radiation Dose Structured Reports¶

openrem_rdsr.py filename.dcm

You can use wildcards to process a number of files at once, ie:

openrem_rdsr.py *.dcm

For mammography DICOM images¶

openrem_mg.py filename.dcm

The facility for extracting dose information from mammography DICOM images has been designed and tested with images created with the GE Senographe DS. It has now been tested in a limited fashion with the images generated by the following systems:

• GE Senographe Essential
• Hologic Selenia
• Siemens Inspiration

See Mammography module for testing restrictions.

For radiographic DICOM images¶

openrem_dx.py filename.dcm

It is now possible with OpenREM version 0.5.0 to import dose information from DX and CR images from digital radiography units. As yet, it hasn’t been tested extensively.

For CT dose summary files from Philips CT scanners¶

openrem_ctphilips.py filename.dcm

Patient height and weight information¶

If height and weight data is not available in the DICOM data, but is available from another source, then it can be imported into the database using the openrem_ptsizecsv.py function. Normally the key to match the size information with the studies in the database will be the accession number; however in some situations this isn’t available and the Study Instance UID can be used instead.

usage:

openrem_ptsizecsv.py [-h] [-u] [-v] csvfile id height weight

-h, --help

Print the help text.

-u, --si-uid

Use Study Instance UID instead of Accession Number.

-v, --verbose

New in 0.3.7 Print to the standard output the success or otherwise of inserting each value.

csvfile

csv file containing the height and/or weight information and study identifier. Other columns will be ignored. Use quotes if the filepath has spaces.

id

Column title for the accession number or study instance UID. Use quotes if the title has spaces.

height

Column title for the patient height (DICOM size) - if this information is missing simply add a blank column with a suitable title. Use quotes if the title has spaces.

weight

Column title for the patient weight - if this information is missing simply add a blank column with a suitable title. Use quotes if the title has spaces.

STORE_AET¶

This is the AET you use when configuring send nodes from your modalities.

Set this to your chosen value - any combination of letters and numbers up to 16 characters. No spaces or other characters allowed.

In the current implementation, the actual value is not actually important - the AET is not checked when a DICOM object is received.

STORE_PORT¶

This is the port you send DICOM objects to. The standard port for DICOM servers is port 104. However, on many operating systems starting a service on a port lower than 1025 requires additional privileges. That is why the suggested port is 8104.

Depending on your network setup you may have to configure the firewall accordingly.

RM_DCM_NOMATCH¶

When DICOM objects are received they are checked for suitability to have dose related data extracted using any of the current extraction routines.

If you want the DICOM object to be deleted if it can’t be used, set RM_DCM_NOMATCH to True. Otherwise set this to False.

Setting this to True is advisable as otherwise your disk can fill up very quickly if enture CT studies get sent through for example.

RM_DCM_RDSR, RM_DCM_MG, RM_DCM_DX and RM_DCM_CTPHIL¶

Set these to True to delete the DICOM objects once the dose related data had been extracted. Otherwise set them to False, and the DICOM objects will be stored in a folder called dicom_in in the MEDIA_ROOT folder.

The RDSR setting is for Radiation Dose Structured Reports (usually from CT or fluoroscopy), MG is for mammography images, DX is for radiography images and CTPHIL is for Philips CT dose screen images.

Setting these to True is advisable, especially for the images as again the disk can fill up quickly.

DICOM Store¶

• Configuration will move to the database with a web interface
• Web interface view of activity and logs

DICOM Query-Retrieve¶

• Function to query retrieve the PACS or modality
• Ad hoc or scheduled
• Web interface for configuration, activating, monitoring success and logs

Results text file¶

It should look something like this:

File created    : 04/21/15 17:42:45
Data file       : C:/Users/[...]/exports-2015-04-21-OpenSkinExport20150421-162805246134.csv
Phantom         : 90.0x70.0 3d phantom
Peak dose (Gy)  :                  0.50844405521
Cells > 3 Gy    :                              0
Cells > 5 Gy    :                              0
Cells > 10 Gy   :                              0

The peak dose is the peak incident dose delivered to any one-cm-square area. If any of these 1 cm² areas (referred to as cells) are above 3 Gy, then the number of cells in this category, or the two higher dose categories, are listed in the table accordingly.

Incidence map image file¶

The image file will be a small 70x90 px PNG image if you used the 3D phantom, or 150 x 50 px PNG if you used the 2D phantom. With both, the head end of the table is on the left.

The image is scaled so that black is 0 Gy and white is 10 Gy. For most studies, this results in an incidence map that is largely black! However, if you use GIMP or ImageJ or similar to increase the contrast, you will find that the required map is there.

A native and ‘colour equalised’ version of the same export are shown below: